Outcome mapping a method for measuring the results of social change interventions
1 / 52

Outcome Mapping: A Method for Measuring the Results of Social Change Interventions - PowerPoint PPT Presentation

  • Uploaded on

Outcome Mapping: A Method for Measuring the Results of Social Change Interventions. Child Rights in Practice: ‘Measuring Our Impact’ May 10-15, 2009 Whistler, British Columbia, Canada. Terry Smutylo [email protected] Two approaches to Evaluation:. As inspection:

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation

PowerPoint Slideshow about ' Outcome Mapping: A Method for Measuring the Results of Social Change Interventions' - mimir

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Outcome mapping a method for measuring the results of social change interventions
Outcome Mapping:A Method for Measuring the Results of Social Change Interventions

Child Rights in Practice:

‘Measuring Our Impact’

May 10-15, 2009

Whistler, British Columbia, Canada

Terry Smutylo

[email protected]

Two approaches to evaluation
Two approaches to Evaluation:

  • As inspection:

  • Judge success or failure

  • Measure against fixed goals

  • External for objectivity

  • Linear cause/effect models

  • Accountability to external

  • Accountability for control, blame

  • Evaluator controls evaluation

  • Engender fear of failure

    • As supporting innovation:

    • Provide feedback for improvement

    • Adapt measurements as goals evolve

    • Internal, integrated, interpretive

    • Seek to capture system dynamics

    • Accountability to values, commitments

    • Understand & respond strategically

    • Evaluator matches process to context

    • Feed hunger for learning

    Adapted from: Patton, Michael Q., 2006, “Evaluation for the Way We Work”, The Nonprofit Quarterly, Spring.

    An evaluation conundrum
    An evaluation conundrum

    the solution…

    changed behavior

    program influence decreases

    community capacity & ownership increases

    miracles occur

    Challenges in measuring results
    Challenges in measuring « results »

    Change is:

    • Complex (multiple actors and factors)

    • Continuous (not limited to the life of the project)

    • Non-linear (often unexpected, emergent)

    • Two-way (program may also change)

    • Beyond the control of the project

    • Incremental, cumulative (watersheds & tipping points)

    Focus on direct partners
    ...Focus on direct partners

    The individuals, groups, and organizations you work with directly to support change

    Influence not impact
    …Influence, not «impact»

    • The project contributes to a better world

    • …by influencing the behaviour (relationships, interactions and or actions) of its partners

    Pay attention to what happens along the way
    Pay attention to what happens along the way…

    The journey is as important as the destination.

    a brief history

    • mid-1990s: need to demonstrate results

    • 1998: Barry Kibel and Outcome Engineering

    • 1999: methodological collaboration with FRAO & NEPED (IDRC funded projects)

    • 2000: publication of manual in English

    • training, facilitation & usage globally

    • 2006: www.outcomemapping.ca

    • 2008: CLAMA

    What s new about om
    What’s new about OM?

    • “Outcomes” = changes in behavior

    • Focus on actors who will drive & maintain the changes – “Boundary Partners”

    • Sets out the pathways of change with – “Progress Markers”

    • Recognizes the boundaries of the project’s influence




    Identifying system boundaries

    sphere of interest

    sphere of influence

    sphere of ‘control’

    Adapted from: Steff Deprez VVOB-CEGO, Nov 2006

    What are we trying to accomplish and how?

    What do we want to track & how?

    What do we

    want to learn & report?

    Intentional Design

    step 1: vision


    improved human, social, & environmental wellbeing

    I have a dream!

    Martin Luther King, Jr.

    August 28, 1963

    vision guides…

    …and energizes the team

    Swayamsiddha Vision

    Across rural India, women and girls utilize and benefit from appropriate health care, education, food and water security and freedom from violence. Women have access to the markets, credit, banking and municipal services they need to pursue their livelihood goals. They use drudgery-reducing technologies and agricultural inputs that contribute to personal well-being and to ecological sustainability. Villages are fully served by public transport, are well lit at night and police enforce all laws fully and equitably. Girls attend school full time and families have the information and resources to make informed decisions regarding their health, safety and social needs. Gender equity governs household labor and decision-making; and men in the community understand and support gender-responsive laws. As elected representatives and as pressure groups, women influence decisions and policies affecting the wellbeing of their families and communities.

    Intentional Design

    step 2: mission

    Mission statement
    mission statement

    Describes how the program intends to:

    • Apply its resources in support of the vision

    • Support the achievement of outcomes by its direct partners

      (what it does, with who, how and why)

    Example Mission Statement

    The Swayamsiddha Project works with governments, NGOs & CBOs to improve women’s health and empowerment. It facilitates the formation of women’s self help groups by providing funding and training to help them influence community and government services to be more responsive to their health and livelihood needs. The project fosters mutual respect and joint action between women’s self-help groups and banks, police, health & social service providers and government agencies. It researches and promotes ecosystem and human health practices in agriculture and in community health and sanitation services. Swayamsiddha addresses equity issues in all its activities. It uses participatory methods to monitor progress, to learn how to become more effective in supporting its partners and to report on its results.




    • about the future

    • observable

    • idealistic

    • not about the program

    • feasible

    • identifies activities and relationships

    • about the program

    Intentional Design

    step 3: boundary partners

    sphere of influence

    The part of the world you are interested in


    = the project’s direct or ‘boundary’ partners

    Intentional Design

    step 4: outcome challenge

    what indicators do you use for:

    Greater awareness…

    Empowered women…

    Community ownership…

    Reduced conflict…

    Increased collaboration…

    Governmental commitment…

    Gender sensitivity…

    Equal access…

    Budgetary transparency…

    Active participation…

    Poverty alleviation…

    Strengthened capacity…


    Outcome challenge statement
    outcome challenge statement

    • Describes the behaviours…

    • of a single boundary partner…

    • contributing maximally to the vision

    Intentional Design

    step 5: progress markers

    progress markers (ladder of change)

    Love to see

    Expanding influence, helping others, sharing expertise

    Like to see

    Actively engaged, learning, commitment

    Expect to see

    Early encouraging response to program, initial engagement

    Progress markers for local communities
    Progress Markers for local communities

    • Participating in regular model forest meetings

    • Establishing a structure for cooperation

    • Acquiring new skills for managing model forests

    • Contributing resources to get the MF operational

    • Articulating a locally relevant vision for the MF

    • Promoting their MF nationally

    • Expanding the partnership

    • Calling upon external experts for advice

    • Requesting new opportunities for training

    • Publishing examples of benefits achieved through MF

    • Seeking out new partners for the MF

    • Obtaining funding from different national sources

    • Helping other communities establish MFs

    • Sharing lessons learned internationally

    • Influencing national policy debates on resource use

    progress markers:

    • stimulate the program to try to contribute to profound transformations

    • provide basis for dialogue with partners

    • articulate the complexity of the change the program is engaged in

    • facilitate ongoing assessment of progress

    • signal need for mid-course corrections and improvements




    Girls & Women

    Women’s Self Help Groups



    State NGOs


    Community Leaders

    Public Health Clinics

    Project’s Outcomes




    BP’s outcomes

    Intentional Design

    step 6: strategy maps


    what networks or relationships will be established or utilized?

    how will sustained support, guidance or mentoring be provided?


    aimed at boundary partner’s environment

    what will be done to produce and “immediate output”?

    what will be done to alter the physical or policy environment?




    how will the media or publications be used?


    aimed atindividual boundary partner

    what will be done to build capacity?


    facilitation questions







    value of strategy maps

    • Clarify what the project is doing, with whom and why

    • Anticipate the program’s influence on the BP

    • Articulate the range, mix and logic of the strategies

    • Detect strategy gaps and over extension of resources

    • Encourage multiple interventions to increase effectiveness

    • Suggest appropriate evaluation methods

    Om principles of use
    OM principles of use:

    Flexible: modular to be adapted to use & context

    Complementary: combine with other methods

    Om principles of use1
    OM principles of use

    Participatory: in dialogue and collaboration with partners

    Evaluative: promote culture of reflection, results oriented thinking,

    and social & organizational learning

    Points to Remember:

    • Stay in touch with the complex reality.

    2.Act out of passion, integrity & competence

    3.Celebrate the contributions of your team and your partners.

    4.Be “idealistic realists.”

    5.Learn, then teach upwards.

    for more information and to share experiences




    What is monitoring
    What is “monitoring”?

    …routine, ongoing, collection of information on a program’s outputs, outcomes and indicators to measure, improve and report on the performance.

    What is evaluation
    What is “evaluation”?

    …the systematic acquisition and assessment of information to provide useful feedback about a program, policy, technology or activity.

    M e what s the difference
    M&E – what’s the difference?

    …What’s the relationship?

    uses of M&E

    • Feeding learning into management cycle

    • Meeting accountability / reporting requirements

    • Reducing risk

    • Helping a partner learn

    • Decision-making

    • Inform publicity documents, communication activities, or case-study materials

    • What else…?



    situational data



    (behaviour changes in the partners)

    relevance & viability

    (of the program)


    (interventions by the program)

    5 kinds of M&E information

    c o n t e x t u a l i n f o r m a t i o n

    Useful M&E Has 4 Dimensions

    ‘Rhythm & Spaces’

    (Info to users/events/audiences)

    Clarifying Intent

    (Of interventions & outcomes)

    M&E Implementation

    (Data collection & interpretation)

    M&E Planning

    (Choosing what to track & how)

    Choices influencing m e design
    Choices Influencing M&E Design


    strategies implemented

    morbidity & mortality

    partner actions in community

    community responses to proj.

    partner collab. with ministry

    community participation

    funding flows


    Adjust Strategy

    Improve Implementation

    Report Performance

    Risk Management

    Build Capacity



    Annual Project Mtg

    Annual Report

    Quarterly Reportto Donor

    3 yr Prog Review

    Quarterly Team Mtg


    Program Staff



    Project Partner

    Program Management

    Exercise 12:

    M&E Planning Worksheet