Outcome mapping a method for measuring the results of social change interventions
This presentation is the property of its rightful owner.
Sponsored Links
1 / 52

Outcome Mapping: A Method for Measuring the Results of Social Change Interventions PowerPoint PPT Presentation


  • 117 Views
  • Uploaded on
  • Presentation posted in: General

Outcome Mapping: A Method for Measuring the Results of Social Change Interventions. Child Rights in Practice: ‘Measuring Our Impact’ May 10-15, 2009 Whistler, British Columbia, Canada. Terry Smutylo [email protected] Two approaches to Evaluation:. As inspection:

Download Presentation

Outcome Mapping: A Method for Measuring the Results of Social Change Interventions

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Outcome mapping a method for measuring the results of social change interventions

Outcome Mapping:A Method for Measuring the Results of Social Change Interventions

Child Rights in Practice:

‘Measuring Our Impact’

May 10-15, 2009

Whistler, British Columbia, Canada

Terry Smutylo

[email protected]


Two approaches to evaluation

Two approaches to Evaluation:

  • As inspection:

  • Judge success or failure

  • Measure against fixed goals

  • External for objectivity

  • Linear cause/effect models

  • Accountability to external

  • Accountability for control, blame

  • Evaluator controls evaluation

  • Engender fear of failure

    • As supporting innovation:

    • Provide feedback for improvement

    • Adapt measurements as goals evolve

    • Internal, integrated, interpretive

    • Seek to capture system dynamics

    • Accountability to values, commitments

    • Understand & respond strategically

    • Evaluator matches process to context

    • Feed hunger for learning

    Adapted from: Patton, Michael Q., 2006, “Evaluation for the Way We Work”, The Nonprofit Quarterly, Spring.


    An evaluation conundrum

    An evaluation conundrum

    the solution…

    changed behavior

    program influence decreases

    community capacity & ownership increases

    miracles occur


    Challenges in measuring results

    Challenges in measuring « results »

    Change is:

    • Complex (multiple actors and factors)

    • Continuous (not limited to the life of the project)

    • Non-linear (often unexpected, emergent)

    • Two-way (program may also change)

    • Beyond the control of the project

    • Incremental, cumulative (watersheds & tipping points)


    Outcome mapping a method for measuring the results of social change interventions

    key ideas in


    Focus on direct partners

    ...Focus on direct partners

    The individuals, groups, and organizations you work with directly to support change


    Influence not impact

    …Influence, not «impact»

    • The project contributes to a better world

    • …by influencing the behaviour (relationships, interactions and or actions) of its partners


    Pay attention to what happens along the way

    Pay attention to what happens along the way…

    The journey is as important as the destination.


    Outcome mapping a method for measuring the results of social change interventions

    a brief history

    • mid-1990s: need to demonstrate results

    • 1998: Barry Kibel and Outcome Engineering

    • 1999: methodological collaboration with FRAO & NEPED (IDRC funded projects)

    • 2000: publication of manual in English

    • training, facilitation & usage globally

    • 2006: www.outcomemapping.ca

    • 2008: CLAMA


    What s new about om

    What’s new about OM?

    • “Outcomes” = changes in behavior

    • Focus on actors who will drive & maintain the changes – “Boundary Partners”

    • Sets out the pathways of change with – “Progress Markers”

    • Recognizes the boundaries of the project’s influence


    Outcome mapping a method for measuring the results of social change interventions

    Beneficiaries

    Partners

    Project

    Identifying system boundaries

    sphere of interest

    sphere of influence

    sphere of ‘control’

    Adapted from: Steff Deprez VVOB-CEGO, Nov 2006


    Outcome mapping a method for measuring the results of social change interventions

    What are we trying to accomplish and how?

    What do we want to track & how?

    What do we

    want to learn & report?


    Outcome mapping a method for measuring the results of social change interventions

    Intentional Design

    step 1: vision


    Outcome mapping a method for measuring the results of social change interventions

    Vision

    improved human, social, & environmental wellbeing


    Outcome mapping a method for measuring the results of social change interventions

    I have a dream!

    Martin Luther King, Jr.

    August 28, 1963


    Outcome mapping a method for measuring the results of social change interventions

    vision guides…

    …and energizes the team


    Outcome mapping a method for measuring the results of social change interventions

    Swayamsiddha Vision

    Across rural India, women and girls utilize and benefit from appropriate health care, education, food and water security and freedom from violence. Women have access to the markets, credit, banking and municipal services they need to pursue their livelihood goals. They use drudgery-reducing technologies and agricultural inputs that contribute to personal well-being and to ecological sustainability. Villages are fully served by public transport, are well lit at night and police enforce all laws fully and equitably. Girls attend school full time and families have the information and resources to make informed decisions regarding their health, safety and social needs. Gender equity governs household labor and decision-making; and men in the community understand and support gender-responsive laws. As elected representatives and as pressure groups, women influence decisions and policies affecting the wellbeing of their families and communities.


    Outcome mapping a method for measuring the results of social change interventions

    vision in graphic form, Nagaland (India)


    Outcome mapping a method for measuring the results of social change interventions

    Intentional Design

    step 2: mission


    Mission statement

    mission statement

    Describes how the program intends to:

    • Apply its resources in support of the vision

    • Support the achievement of outcomes by its direct partners

      (what it does, with who, how and why)


    Outcome mapping a method for measuring the results of social change interventions

    Example Mission Statement

    The Swayamsiddha Project works with governments, NGOs & CBOs to improve women’s health and empowerment. It facilitates the formation of women’s self help groups by providing funding and training to help them influence community and government services to be more responsive to their health and livelihood needs. The project fosters mutual respect and joint action between women’s self-help groups and banks, police, health & social service providers and government agencies. It researches and promotes ecosystem and human health practices in agriculture and in community health and sanitation services. Swayamsiddha addresses equity issues in all its activities. It uses participatory methods to monitor progress, to learn how to become more effective in supporting its partners and to report on its results.


    Outcome mapping a method for measuring the results of social change interventions

    Summary

    Vision

    Mission

    • about the future

    • observable

    • idealistic

    • not about the program

    • feasible

    • identifies activities and relationships

    • about the program


    Outcome mapping a method for measuring the results of social change interventions

    Intentional Design

    step 3: boundary partners


    Outcome mapping a method for measuring the results of social change interventions

    sphere of influence

    The part of the world you are interested in

    Program

    = the project’s direct or ‘boundary’ partners


    Outcome mapping a method for measuring the results of social change interventions

    moving from stakeholders...


    Outcome mapping a method for measuring the results of social change interventions

    ...to boundary partners

    project


    Outcome mapping a method for measuring the results of social change interventions

    Intentional Design

    step 4: outcome challenge


    Outcome mapping a method for measuring the results of social change interventions

    what indicators do you use for:

    Greater awareness…

    Empowered women…

    Community ownership…

    Reduced conflict…

    Increased collaboration…

    Governmental commitment…

    Gender sensitivity…

    Equal access…

    Budgetary transparency…

    Active participation…

    Poverty alleviation…

    Strengthened capacity…

    ?


    Outcome challenge statement

    outcome challenge statement

    • Describes the behaviours…

    • of a single boundary partner…

    • contributing maximally to the vision


    Outcome mapping a method for measuring the results of social change interventions

    Intentional Design

    step 5: progress markers


    Outcome mapping a method for measuring the results of social change interventions

    progress markers (ladder of change)

    Love to see

    Expanding influence, helping others, sharing expertise

    Like to see

    Actively engaged, learning, commitment

    Expect to see

    Early encouraging response to program, initial engagement


    Progress markers for local communities

    Progress Markers for local communities

    • Participating in regular model forest meetings

    • Establishing a structure for cooperation

    • Acquiring new skills for managing model forests

    • Contributing resources to get the MF operational

    • Articulating a locally relevant vision for the MF

    • Promoting their MF nationally

    • Expanding the partnership

    • Calling upon external experts for advice

    • Requesting new opportunities for training

    • Publishing examples of benefits achieved through MF

    • Seeking out new partners for the MF

    • Obtaining funding from different national sources

    • Helping other communities establish MFs

    • Sharing lessons learned internationally

    • Influencing national policy debates on resource use


    Outcome mapping a method for measuring the results of social change interventions

    progress markers:

    • stimulate the program to try to contribute to profound transformations

    • provide basis for dialogue with partners

    • articulate the complexity of the change the program is engaged in

    • facilitate ongoing assessment of progress

    • signal need for mid-course corrections and improvements


    Outcome mapping a method for measuring the results of social change interventions

    mission

    vision

    Banks

    Girls & Women

    Women’s Self Help Groups

    Police

    BAIF

    State NGOs

    Families

    Community Leaders

    Public Health Clinics

    Project’s Outcomes

    Strategies

    StrategicPartners

    BoundaryPartners

    BP’s outcomes


    Outcome mapping a method for measuring the results of social change interventions

    Intentional Design

    step 6: strategy maps


    Outcome mapping a method for measuring the results of social change interventions

    6 kinds of strategies


    Outcome mapping a method for measuring the results of social change interventions

    ?

    what networks or relationships will be established or utilized?

    how will sustained support, guidance or mentoring be provided?

    E

    aimed at boundary partner’s environment

    what will be done to produce and “immediate output”?

    what will be done to alter the physical or policy environment?

    causal

    persuasive

    supportive

    how will the media or publications be used?

    I

    aimed atindividual boundary partner

    what will be done to build capacity?

    strategies

    facilitation questions

    I-3

    I-1

    I-2

    E-1

    E-2

    E-3


    Outcome mapping a method for measuring the results of social change interventions

    value of strategy maps

    • Clarify what the project is doing, with whom and why

    • Anticipate the program’s influence on the BP

    • Articulate the range, mix and logic of the strategies

    • Detect strategy gaps and over extension of resources

    • Encourage multiple interventions to increase effectiveness

    • Suggest appropriate evaluation methods


    Om principles of use

    OM principles of use:

    Flexible: modular to be adapted to use & context

    Complementary: combine with other methods


    Om principles of use1

    OM principles of use

    Participatory: in dialogue and collaboration with partners

    Evaluative: promote culture of reflection, results oriented thinking,

    and social & organizational learning


    Outcome mapping a method for measuring the results of social change interventions

    Points to Remember:

    • Stay in touch with the complex reality.

    2.Act out of passion, integrity & competence

    3.Celebrate the contributions of your team and your partners.

    4.Be “idealistic realists.”

    5.Learn, then teach upwards.


    Outcome mapping a method for measuring the results of social change interventions

    for more information and to share experiences

    www.outcomemapping.ca

    www.idrc.ca/evaluation

    www.mapeodealcances.net


    What is monitoring

    What is “monitoring”?

    …routine, ongoing, collection of information on a program’s outputs, outcomes and indicators to measure, improve and report on the performance.


    What is evaluation

    What is “evaluation”?

    …the systematic acquisition and assessment of information to provide useful feedback about a program, policy, technology or activity.


    M e what s the difference

    M&E – what’s the difference?

    …What’s the relationship?


    Outcome mapping a method for measuring the results of social change interventions

    uses of M&E

    • Feeding learning into management cycle

    • Meeting accountability / reporting requirements

    • Reducing risk

    • Helping a partner learn

    • Decision-making

    • Inform publicity documents, communication activities, or case-study materials

    • What else…?


    Outcome mapping a method for measuring the results of social change interventions

    Program

    Partner

    situational data

    Strategies

    outcomes

    (behaviour changes in the partners)

    relevance & viability

    (of the program)

    implementation

    (interventions by the program)

    5 kinds of M&E information

    c o n t e x t u a l i n f o r m a t i o n


    Outcome mapping a method for measuring the results of social change interventions

    Useful M&E Has 4 Dimensions

    ‘Rhythm & Spaces’

    (Info to users/events/audiences)

    Clarifying Intent

    (Of interventions & outcomes)

    M&E Implementation

    (Data collection & interpretation)

    M&E Planning

    (Choosing what to track & how)


    Choices influencing m e design

    Choices Influencing M&E Design

    INFO

    strategies implemented

    morbidity & mortality

    partner actions in community

    community responses to proj.

    partner collab. with ministry

    community participation

    funding flows

    USES

    Adjust Strategy

    Improve Implementation

    Report Performance

    Risk Management

    Build Capacity

    Lobbying

    EVENTS

    Annual Project Mtg

    Annual Report

    Quarterly Reportto Donor

    3 yr Prog Review

    Quarterly Team Mtg

    USERS

    Program Staff

    Board

    Donor

    Project Partner

    Program Management


    Outcome mapping a method for measuring the results of social change interventions

    Exercise 12:

    M&E Planning Worksheet


  • Login