outcome mapping a method for measuring the results of social change interventions
Download
Skip this Video
Download Presentation
Outcome Mapping: A Method for Measuring the Results of Social Change Interventions

Loading in 2 Seconds...

play fullscreen
1 / 52

Outcome Mapping: A Method for Measuring the Results of Social Change Interventions - PowerPoint PPT Presentation


  • 160 Views
  • Uploaded on

Outcome Mapping: A Method for Measuring the Results of Social Change Interventions. Child Rights in Practice: ‘Measuring Our Impact’ May 10-15, 2009 Whistler, British Columbia, Canada. Terry Smutylo tsmutylo@magma.ca. Two approaches to Evaluation:. As inspection:

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about ' Outcome Mapping: A Method for Measuring the Results of Social Change Interventions' - mimir


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
outcome mapping a method for measuring the results of social change interventions
Outcome Mapping:A Method for Measuring the Results of Social Change Interventions

Child Rights in Practice:

‘Measuring Our Impact’

May 10-15, 2009

Whistler, British Columbia, Canada

Terry Smutylo

tsmutylo@magma.ca

two approaches to evaluation
Two approaches to Evaluation:
    • As inspection:
  • Judge success or failure
  • Measure against fixed goals
  • External for objectivity
  • Linear cause/effect models
  • Accountability to external
  • Accountability for control, blame
  • Evaluator controls evaluation
  • Engender fear of failure
  • As supporting innovation:
  • Provide feedback for improvement
  • Adapt measurements as goals evolve
  • Internal, integrated, interpretive
  • Seek to capture system dynamics
  • Accountability to values, commitments
  • Understand & respond strategically
  • Evaluator matches process to context
  • Feed hunger for learning

Adapted from: Patton, Michael Q., 2006, “Evaluation for the Way We Work”, The Nonprofit Quarterly, Spring.

an evaluation conundrum
An evaluation conundrum

the solution…

changed behavior

program influence decreases

community capacity & ownership increases

miracles occur

challenges in measuring results
Challenges in measuring « results »

Change is:

  • Complex (multiple actors and factors)
  • Continuous (not limited to the life of the project)
  • Non-linear (often unexpected, emergent)
  • Two-way (program may also change)
  • Beyond the control of the project
  • Incremental, cumulative (watersheds & tipping points)
focus on direct partners
...Focus on direct partners

The individuals, groups, and organizations you work with directly to support change

influence not impact
…Influence, not «impact»
  • The project contributes to a better world
  • …by influencing the behaviour (relationships, interactions and or actions) of its partners
pay attention to what happens along the way
Pay attention to what happens along the way…

The journey is as important as the destination.

slide10

a brief history

  • mid-1990s: need to demonstrate results
  • 1998: Barry Kibel and Outcome Engineering
  • 1999: methodological collaboration with FRAO & NEPED (IDRC funded projects)
  • 2000: publication of manual in English
  • training, facilitation & usage globally
  • 2006: www.outcomemapping.ca
  • 2008: CLAMA
what s new about om
What’s new about OM?
  • “Outcomes” = changes in behavior
  • Focus on actors who will drive & maintain the changes – “Boundary Partners”
  • Sets out the pathways of change with – “Progress Markers”
  • Recognizes the boundaries of the project’s influence
slide12

Beneficiaries

Partners

Project

Identifying system boundaries

sphere of interest

sphere of influence

sphere of ‘control’

Adapted from: Steff Deprez VVOB-CEGO, Nov 2006

slide13

What are we trying to accomplish and how?

What do we want to track & how?

What do we

want to learn & report?

slide14

Intentional Design

step 1: vision

slide15

Vision

improved human, social, & environmental wellbeing

slide16

I have a dream!

Martin Luther King, Jr.

August 28, 1963

slide17

vision guides…

…and energizes the team

slide18

Swayamsiddha Vision

Across rural India, women and girls utilize and benefit from appropriate health care, education, food and water security and freedom from violence. Women have access to the markets, credit, banking and municipal services they need to pursue their livelihood goals. They use drudgery-reducing technologies and agricultural inputs that contribute to personal well-being and to ecological sustainability. Villages are fully served by public transport, are well lit at night and police enforce all laws fully and equitably. Girls attend school full time and families have the information and resources to make informed decisions regarding their health, safety and social needs. Gender equity governs household labor and decision-making; and men in the community understand and support gender-responsive laws. As elected representatives and as pressure groups, women influence decisions and policies affecting the wellbeing of their families and communities.

slide20

Intentional Design

step 2: mission

mission statement
mission statement

Describes how the program intends to:

  • Apply its resources in support of the vision
  • Support the achievement of outcomes by its direct partners

(what it does, with who, how and why)

slide22

Example Mission Statement

The Swayamsiddha Project works with governments, NGOs & CBOs to improve women’s health and empowerment. It facilitates the formation of women’s self help groups by providing funding and training to help them influence community and government services to be more responsive to their health and livelihood needs. The project fosters mutual respect and joint action between women’s self-help groups and banks, police, health & social service providers and government agencies. It researches and promotes ecosystem and human health practices in agriculture and in community health and sanitation services. Swayamsiddha addresses equity issues in all its activities. It uses participatory methods to monitor progress, to learn how to become more effective in supporting its partners and to report on its results.

slide23

Summary

Vision

Mission

  • about the future
  • observable
  • idealistic
  • not about the program
  • feasible
  • identifies activities and relationships
  • about the program
slide24

Intentional Design

step 3: boundary partners

slide25

sphere of influence

The part of the world you are interested in

Program

= the project’s direct or ‘boundary’ partners

slide28

Intentional Design

step 4: outcome challenge

slide29

what indicators do you use for:

Greater awareness…

Empowered women…

Community ownership…

Reduced conflict…

Increased collaboration…

Governmental commitment…

Gender sensitivity…

Equal access…

Budgetary transparency…

Active participation…

Poverty alleviation…

Strengthened capacity…

?

outcome challenge statement
outcome challenge statement
  • Describes the behaviours…
  • of a single boundary partner…
  • contributing maximally to the vision
slide31

Intentional Design

step 5: progress markers

slide32

progress markers (ladder of change)

Love to see

Expanding influence, helping others, sharing expertise

Like to see

Actively engaged, learning, commitment

Expect to see

Early encouraging response to program, initial engagement

progress markers for local communities
Progress Markers for local communities
  • Participating in regular model forest meetings
  • Establishing a structure for cooperation
  • Acquiring new skills for managing model forests
  • Contributing resources to get the MF operational
  • Articulating a locally relevant vision for the MF
  • Promoting their MF nationally
  • Expanding the partnership
  • Calling upon external experts for advice
  • Requesting new opportunities for training
  • Publishing examples of benefits achieved through MF
  • Seeking out new partners for the MF
  • Obtaining funding from different national sources
  • Helping other communities establish MFs
  • Sharing lessons learned internationally
  • Influencing national policy debates on resource use
slide34

progress markers:

  • stimulate the program to try to contribute to profound transformations
  • provide basis for dialogue with partners
  • articulate the complexity of the change the program is engaged in
  • facilitate ongoing assessment of progress
  • signal need for mid-course corrections and improvements
slide35

mission

vision

Banks

Girls & Women

Women’s Self Help Groups

Police

BAIF

State NGOs

Families

Community Leaders

Public Health Clinics

Project’s Outcomes

Strategies

StrategicPartners

BoundaryPartners

BP’s outcomes

slide36

Intentional Design

step 6: strategy maps

slide39

?

what networks or relationships will be established or utilized?

how will sustained support, guidance or mentoring be provided?

E

aimed at boundary partner’s environment

what will be done to produce and “immediate output”?

what will be done to alter the physical or policy environment?

causal

persuasive

supportive

how will the media or publications be used?

I

aimed atindividual boundary partner

what will be done to build capacity?

strategies

facilitation questions

I-3

I-1

I-2

E-1

E-2

E-3

slide40

value of strategy maps

  • Clarify what the project is doing, with whom and why
  • Anticipate the program’s influence on the BP
  • Articulate the range, mix and logic of the strategies
  • Detect strategy gaps and over extension of resources
  • Encourage multiple interventions to increase effectiveness
  • Suggest appropriate evaluation methods
om principles of use
OM principles of use:

Flexible: modular to be adapted to use & context

Complementary: combine with other methods

om principles of use1
OM principles of use

Participatory: in dialogue and collaboration with partners

Evaluative: promote culture of reflection, results oriented thinking,

and social & organizational learning

slide43

Points to Remember:

  • Stay in touch with the complex reality.

2.Act out of passion, integrity & competence

3.Celebrate the contributions of your team and your partners.

4.Be “idealistic realists.”

5.Learn, then teach upwards.

slide44

for more information and to share experiences

www.outcomemapping.ca

www.idrc.ca/evaluation

www.mapeodealcances.net

what is monitoring
What is “monitoring”?

…routine, ongoing, collection of information on a program’s outputs, outcomes and indicators to measure, improve and report on the performance.

what is evaluation
What is “evaluation”?

…the systematic acquisition and assessment of information to provide useful feedback about a program, policy, technology or activity.

m e what s the difference
M&E – what’s the difference?

…What’s the relationship?

slide48

uses of M&E

  • Feeding learning into management cycle
  • Meeting accountability / reporting requirements
  • Reducing risk
  • Helping a partner learn
  • Decision-making
  • Inform publicity documents, communication activities, or case-study materials
  • What else…?
slide49

Program

Partner

situational data

Strategies

outcomes

(behaviour changes in the partners)

relevance & viability

(of the program)

implementation

(interventions by the program)

5 kinds of M&E information

c o n t e x t u a l i n f o r m a t i o n

slide50

Useful M&E Has 4 Dimensions

‘Rhythm & Spaces’

(Info to users/events/audiences)

Clarifying Intent

(Of interventions & outcomes)

M&E Implementation

(Data collection & interpretation)

M&E Planning

(Choosing what to track & how)

choices influencing m e design
Choices Influencing M&E Design

INFO

strategies implemented

morbidity & mortality

partner actions in community

community responses to proj.

partner collab. with ministry

community participation

funding flows

USES

Adjust Strategy

Improve Implementation

Report Performance

Risk Management

Build Capacity

Lobbying

EVENTS

Annual Project Mtg

Annual Report

Quarterly Reportto Donor

3 yr Prog Review

Quarterly Team Mtg

USERS

Program Staff

Board

Donor

Project Partner

Program Management

slide52

Exercise 12:

M&E Planning Worksheet

ad