most significant changes n.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
“Most Significant Changes” PowerPoint Presentation
Download Presentation
“Most Significant Changes”

Loading in 2 Seconds...

play fullscreen
1 / 43

“Most Significant Changes” - PowerPoint PPT Presentation


  • 141 Views
  • Uploaded on

“Most Significant Changes”. Adapted from:. Rick Davies - MandENEWS Jessica Dart – Clear Horizon.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about '“Most Significant Changes”' - ulysses


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
adapted from
Adapted from:
  • Rick Davies - MandENEWS
  • Jessica Dart – Clear Horizon

MSC @ Exchange

slide3
“If you knew what was going to happen in advance every day you could do amazing things. You could become insanely wealthy, influence the political process et cetera. Well, it turns out that most people don't even know what happened yesterday in their own business. So, a lot of businesses are discovering they can take tremendous competitive advantage simply by finding out what happened yesterday as soon as possible“ (Steve Jobs, Fortune, 1994:23)

MSC @ Exchange

slide5

Why stories?

  • People tell stories naturally - indigenous
  • Stories can deal with complexity and context
  • People remember stories
  • Stories can carry hard messages /undiscussables
  • But stories not known for accuracy/truth

MSC @ Exchange

slide6

Use of stories in MSC

  • Collection of stories + systematic, collective interpretation = storytelling can be effectively harnessed for participatory evaluation
  • Because interpretations tell another story & process has beneficial outcomes for evaluation utilisation

MSC @ Exchange

slide7

MSC

  • Form of qualitative, participatory M&E
  • Based on ‘stories’ of significant change
  • Developed by Rick Davies 1996 - Bangladesh
  • Now used in numerous development programs and in the public sector

MSC @ Exchange

slide8

MSC

  • Creates space for stakeholders to reflect, to make sense of complex changes
  • Provides dialogue to help make sense of each other’s values
  • Facilitates dynamic dialogue ie. “what do we really want to achieve and how will we produce more of it?”
  • Excellent for participatory programs with diverse, complex outcomes, & multiple stakeholders

MSC @ Exchange

slide9

Purpose of MSC in M&E

  • Primary purpose to facilitate improvement by:
    • focusing direction of work towards explicitly valued directions
    • eg. what do we really want to achieve and how will we produce more of it?
  • Contributes to summative evaluation:
    • Information about unexpected outcomes
    • Performance information concerning very best success stories
    • Can inform criteria used to judge projects

MSC @ Exchange

slide10

Qualitativevs quantitative monitoring

Quantitative

  • Focus on measurement
  • Closed questions
  • About ‘proving’
  • Easy to aggregate
  • Deductive
  • Static
  • Goal displacement can be a problem

Qualitative

  • Focus on questioning
  • Open questions
  • About learning
  • Hard to aggregate
  • Inductive
  • Dynamic
  • Goal displacement is not an issue

MSC @ Exchange

slide11

Qualitative monitoring

  • Can be used in conjunction with conventional output monitoring
  • Is usually more aimed at learning than accountability

MSC @ Exchange

slide13

Limitations of indicator based monitoring

  • Goal displacement
  • Not about learning
  • Don’t tell you what you don’t know you need to know

MSC @ Exchange

how to aggregate complex experience
How to aggregate complex experience?
  • Example:
    • a bowl containing 2 oranges, 3 apples, 4 bananas, and 1 mango
  • Summary-by-inclusion
    • There are 10 pieces of fruit in the bowl
      • Find lowest common denominator = fruit
        • Cost: Loss of interesting detail
  • Summary-by-selection
    • The mango is rotten, it will spoil the rest of the fruit. Remove it, please

MSC @ Exchange

practice now
Practice Now!
  • Turn to your neighbour and ask them
    • What was the most significant change that took place as a result of the workshop this week? (Get the details)
    • Then ask them why they thought this was the most interesting.
    • Document:
      • description (who, what, where, when)
      • explanation (why is it significant)
      • who documented the story (name, position location, date)
    • Then let your neighbour ask the same questions to you.

MSC @ Exchange

slide16
Then…
  • Come to a decision about which of the two stories you think is most interesting, and identify why you both think so.
    • You may have a number of reasons.
  • When asked to, tell a group of 8 people the story you chose, and why you did so

MSC @ Exchange

slide17

Then…

  • Come to a decision about which of the 4 stories you think is most interesting, and identify why you all think so.
    • You may have a number of reasons.
  • When asked to, tell the large group people the story you choose, and why you did so

MSC @ Exchange

the core of msc
The core of MSC
  • A question:
    • “In your opinion what was the most significant change that took place in ….over the … months”
      • [describe the change and explain why you think it is significant]
  • Re-iteration of the same kind of question
    • “Which of these SC stories do you think is the most significant of all?”
      • [describe the change and explain why you think it is significant]

MSC @ Exchange

explaining msc
Explaining MSC
  • The first challenge when introducing it
    • Can be difficult because it is very different to conventional methods
  • Make use of direct experience
  • Use metaphors
  • Highlight the key differences
  • Explain in terms of stages

MSC @ Exchange

using metaphors
Using metaphors
  • Organisations as newspapers,
    • with journalists, sub-editors, editors, senior editors, etc
      • Stories get passed up the hierarchy, but only a few make it to the front page, and only one to the top of the front page
  • Organisations as amoeba,
    • sensing positive and negative experiences and moving to and away from those respectively.

MSC @ Exchange

how is msc different
How is MSC different?
  • Participants have a choice about what sort of information to collect
  • Uses diverse rather than standard data
  • Information is analysed by all participants, not simply by a central unit
  • Subjectivity is used rather than avoided

MSC @ Exchange

slide23

MSCvs quantitative monitoring

Quantitative

  • Focus on measurement
  • Closed questions
  • Project out
  • About ‘proving’
  • Deductive
  • Static
  • Inclusive
  • Central tendencies

MSC

  • Focus on questioning
  • Open questions
  • Context in
  • About learning
  • Inductive
  • Dynamic
  • Selective
  • Outer edges of experience

MSC @ Exchange

slide24

Program out

Context in

Goal-based evaluation

Goals

Extent to which they were achieved

From the view point of the program staff + consultation

Program

From the viewpoint

of the Participants

MSC @ Exchange

explaining msc in stages
Explaining MSC in stages
  • Defining Domains of Change
  • Define reporting period
  • Collecting SC stories
  • Selection of collected SC stories
  • Feedback of the choices made
  • Verification
  • Quantification
  • Meta-monitoring and secondary analysis
  • Re-settings of MSC system

MSC @ Exchange

1 defining domains
1. Defining “domains”
  • Opposite of SMART indicators?
    • Like newspaper sections: sports, finance, leisure, business, etc
    • Defined by how people use them
  • Examples:
    • “changes in peoples’ lives”
    • “changes in relationships with our partners”
    • “changes in government policy on HIV/AIDS

MSC @ Exchange

defining domains
Defining domains…
  • Not essential but
    • Can help structure the selection process
    • Can help focus on goals of concern
    • Their use tells us how what goals mean to participants
  • Options
    • Open window domain
    • Negative changes domain

MSC @ Exchange

2 set the reporting period
2. Set the reporting period
  • “In your opinion what was the most significant change that took place in ….over the … months
  • Period used by NGOs varies from 2 weekly, to monthly, to three monthly, and yearly.
    • Three monthly is most common
  • Time demands on staff is the main constraint on frequency

MSC @ Exchange

3 collecting sc stories
3. Collecting SC stories
  • From those closest to the event’s of concern.
    • But do not exploit people’s unpaid time
  • Basic format:
    • Description (who, what, where, when)
    • Explanation (why is it significant)
    • Who documented the story (name, position location, date)
    • Option: Recommendation

MSC @ Exchange

collecting sc stories
Collecting SC stories…
  • Reminder: Key parts of the question
    • “Looking back over the last month…”
    • “…what do you think was…”
    • “…the most significant…”
    • “…change…”
    • “…in the quality of people’s lives…”
    • “…in this community?”

MSC @ Exchange

slide32

Funder meeting

State meetings

flow of

stories

feedback

Region1

Region 2 Region 3 Region 4

Story tellers

MSC @ Exchange

selecting sc stories
Selecting SC stories…
  • Task is to read through and identify the most significant of all the submitted SC stories. Take one domain at a time
  • Need to decide who to involve: story providers, their superiors, their peers,..
  • Need to decide whether to predefine selection criteria, or let them emerge through discussion of SC stories

MSC @ Exchange

selecting sc stories1
Selecting SC stories…
  • Must (not optional)
    • Document what SC was selected
    • Why it was selected
    • Process used to make the selection
      • Participants
      • Their preferences
  • [Subjectivity is made accountable through transparency]

MSC @ Exchange

5 feedback
5. Feedback
  • To immediate providers of SC stories, on what was selected, why selected and process used
  • Enables adjustment of focus of MSC next time around
  • A motivational factor
  • Weakest point in all M&E systems, including MSC

MSC @ Exchange

6 verification of sc stories
6. Verification of SC stories
  • What
    • Factual content & interpretation of facts
  • Why
    • Encourages some discipline in reporting
    • Enables elaboration and further learning
  • When
    • When SC story first enters system
    • When selected as MS of all SC
    • When SC stories are publicly used

MSC @ Exchange

7 quantification
7. Quantification
  • Within the SC story
    • Number of people, events, etc involved
  • As once–off follow-up to SC story
    • How many other cases like this known
  • Within meta-monitoring (see next)
    • How many other SC stories like this

MSC @ Exchange

8 meta monitoring and secondary analysis
8. Meta-monitoring and secondary analysis
  • Keep all SC stories on record
  • Meta-monitoring (Recommended) of
    • Changes in numbers of SC stories, who provides them, whose SC stories are selected, changes in percentage of negative stories
  • Secondary analysis (Optional) by
    • Categorising and counting of types of changes reported, and types of explanations given, at different levels

MSC @ Exchange

9 re setting of msc process
9. Re-setting of MSC process
  • Frequency of reporting
  • Definition of domains to use
  • Who sorts SC stories into domains
  • Selection process design: participants & process used
  • Feedback and follow up

MSC @ Exchange

where to use msc
Where to use MSC?
  • Talk to your neighbour, and identify where you think MSC
    • Would be most useful, and why
    • Would be least useful, and why
  • Share this view with the whole group, when asked

MSC @ Exchange

where to use msc1
Where to use MSC
  • Not as a stand-alone method
  • Alongside indicator based systems
  • To identify unexpected changes
  • To engage people in analysis of change
  • To involve a wide range of people
  • To focus on outcomes rather than outputs

MSC @ Exchange

finding out more about msc
Finding out more about MSC
  • Original MSC paper (n’th version) is at

http://www.mande.co.uk/docs/ccdb.htm

  • MSC Mailing list is at

http://groups.yahoo.com/group/mostsignificantchanges

  • Rick Davies at rick@mande.co.uk

MSC @ Exchange