slide1 l.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
Evaluating Experiences in Transitional Justice and Reconciliation Cape Town, South Africa April 2-4, 2007 PowerPoint Presentation
Download Presentation
Evaluating Experiences in Transitional Justice and Reconciliation Cape Town, South Africa April 2-4, 2007

Loading in 2 Seconds...

play fullscreen
1 / 56

Evaluating Experiences in Transitional Justice and Reconciliation Cape Town, South Africa April 2-4, 2007 - PowerPoint PPT Presentation


  • 182 Views
  • Uploaded on

Evaluating Experiences in Transitional Justice and Reconciliation Cape Town, South Africa April 2-4, 2007 . day 1. Objectives of Workshop. Discuss issues, challenges & opportunities for evaluating projects and programs that promote transitional justice and reconciliation.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Evaluating Experiences in Transitional Justice and Reconciliation Cape Town, South Africa April 2-4, 2007' - fauna


Download Now An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
slide1

Evaluating Experiences in Transitional Justice and Reconciliation

Cape Town, South Africa

April 2-4, 2007

objectives of workshop
Objectives of Workshop
  • Discuss issues, challenges & opportunities for evaluating projects and programs that promote transitional justice and reconciliation.
  • Introduce evaluation tools & methodologies that might assist in identifying and tracking outcomes.
  • Explore interest in further developing evaluation approaches for transitional justice and reconciliation.
why this workshop why now
Why this workshop, why now?
  • Stellenbosch conference on empirical research methodologies, Nov.2002
    • Prospects for using evaluation findings as a research tool.
  • Evaluation community: Growing methodological orthodoxy; shortcomings of methods for evaluating conflict prevention and post-conflict interventions.
  • IDRC`s approach to evaluation mirrors approach to development research (rigour/validity, action-oriented, focus on ownership, participation, prioritizes CB
  • `Evaluative thinking`- evaluation as an analytical way of thinking.
review of key concepts
Review of key concepts:
  • What do we understand by?
    • Monitoring
    • Evaluation
    • Results
    • Research
m e what s the difference
M&E – what’s the difference?

Monitoring

  • Ongoing, continuous
  • Internal activity
  • Responsibility of project staff and management
  • Continuous feedback to improve programme & report on performance

Evaluation

  • Periodic and time bound; `episodic assessment`
  • Can be internal, often external
  • Responsibility of evaluator with staff and management
  • Periodic feedback
what are results
What are “Results”

For the Evaluator:

  • Outputs, Outcomes, Impacts, Effects

For the Researcher:

  • When we talk about “research results, we actually mean “research findings”
how do evaluation and applied research differ
How do Evaluation and (Applied) Research Differ?

Many similarities:

  • Both rely on social science methods;
  • Examine multiple facets of a problem, often using multimethod approaches;
  • collect and analyse data;
  • utilize theory to inform work;
what s the difference between evaluation and applied research
What´s the Difference between Evaluation and (Applied) Research?

Distinctions:

Evaluation uses universally accepted standards (Utility, Propriety, Feasibility, Accuracy)

Always assesses the performance of the person or entity under investigation

Audience:Evaluation has a client who wants to know something.

slide10

overview of

outcome mapping

slide11

before we start, be aware...

  • OM is not a panacea
  • New vocabulary
  • OM depends on context, needs and realities
slide13

history of outcome mapping

    • mid-1990s: need to demonstrate results
    • 1998: met Barry Kibel and Outcome Engineering
    • methodological collaboration with FRAO & NEPED
    • 2000: publication of manual in English
    • presenting, training & using OM globally
    • 2006: www.outcomemapping.ca
    • …. towards the future
slide15

key evaluation challenges

    • measuring development results of research
    • establishing cause & effect in an open system
    • timing
    • encouraging iterative learning
    • clarifying values
connecting research to well being

POLITICS

  • New Knowledge/ Ideas
  • POLICIES
  • Motivations
  • NGOs
  • Environmental
  • Women Groups
  • Advocacy
  • ENVIRONMENT
  • National/ Regional
  • Money/ policy/ services
  • EXTENSION GROUPS
  • Government
  • NGOs,
  • Advisory services
  • OTHER RESEARCHERS
  • Universities
  • Technologies
  • Information
  • DISSEMINATION
  • New Knowledge
  • Extension
  • Inputs
  • FUNDERS
  • Policies
  • Motivations
  • Money + Inputs
  • PRIVATE SECTOR
  • Seed suppliers
  • Technical Assistance
  • MINISTRY
  • Agricultural
  • Facilitation
  • Regulations
  • NGOs
  • Facilitate Adoption of Technology
  • FARMERS
  • Farmer Orgs
  • Advocacy for research

NARO

Beneficiaries

  • RESEARCH INSTITUTES
  • Outside Community
  • GOV/ MINISTRIES
  • Incentives to Facilitate Adoption of Technology
  • NARO
  • Support Staff
  • Research Support
  • DONORS
  • Financial resources
  • Human resources
  • REG~L/ INTERN~L CENTRES
  • Information Technology
  • FARMER ORGANIZATIONS
  • Identify problems
  • Dissemination

RESEARCH MANAGERS

RESEARCHERS

  • FARMER ASSOCIATIONS
  • Extension Services
  • LOCAL ORGs
  • Leadership
  • Mobilization

USERS

(Farmers & Families)

  • FARMERS/ PRODUCERS
  • New Knowledge
  • Sharing
  • Motivation
  • NGOs
  • Farmer training
  • Transfer of Technology

PRIVATE SECTOR

National & International

RURAL SOCIOLOGISTS

Identification of opportunities & constraints

  • POST PRODUCTION
  • Marketing
  • Transportation/ Shipping
connecting research to well-being

8-15 Years

problem with impact
Impact Implies:

Cause & effect

Positive, intended results

Focus on ultimate effects

Credits a single contributor

Story ends when program obtains success

Development Implies:

Open system

Unexpected positive & negative results occur

Upstream effects are important

Multiple actors create results & need credit

Change process never ends

problem with « impact »
focus of outcome mapping
focus of outcome mapping

Behavioural Changes

slide19

where is the map?

  • OM is a guide to the journey we take with our partners. We co-create the map.
  • It focuses on the intention,
  • what happens - and the learning - along the way
  • The map is not the territory but the route taken
recommended reading
recommended reading

Liberia case study & OM manual foreward by

Michael Quinn Patton and

introduction (pages vii-ix and 1-15)

what is outcome mapping
what is outcome mapping?
  • A methodology for planning and assessing the social effects & internal performance of projects, programs, & organizations
a flexible multiple use tool
a flexible, multiple-use tool
  • Planning
  • Monitoring
  • Evaluation
slide25

What are we trying to accomplish and how?

What do we want to know?

What do we

want to learn?

looking at the bigger picture
looking at the bigger picture
  • Seeing yourself as a part of a interconnected web of relationships and systems
recognizing that change is
recognizing that change is…
  • Continuous
  • Complex
  • Non-linear
  • Multidirectional
  • Not controllable
embrace constant change
embrace constant change

“It’s not possible to see the same river twice.”

keeping your eyes wide open
keeping your eyes wide open
  • Being attentive along the journey is as important as the destination
focus on direct partners
focus on direct partners
  • Key concept is « boundary partners »
  • The individuals, groups, and organizations you work with directly and anticipate opportunities for influence
spheres of influence
spheres of influence

The rest of the world

project / program

= boundary partners

slide33

boundary partners have boundary partners

program

program’s bp

bp’s bp

slide34

Families

PHCs

Banks

Community

Leaders

SHG

Police

State

NGO

State

NGO

State

NGO

State

NGO

State

NGO

State

NGO

BAIF

IDRC

Swayamsiddha

CIDA

slide37

why behaviour changes?

  • To stress that development is done by and for people
  • To illustrate that although a program can influence the achievement of outcomes, it cannot control them because ultimate responsibility rests with the people affected
contribution not attribution
contribution not attribution
  • your influence on a better world
  • you can influence but not control change in your partners
principles of use
principles of use
  • Flexible: modular to be adapted to use & context
  • Complementary: use with other methodologies.
  • Participatory: seeks dialogue and collaboration with partners
  • Evaluative thinking: culture of reflection, results oriented thinking, andpromotes social & organizational learning
primary uses
primary uses
  • PLANNING: articulate goals & define activities
  • MONITORING:

assess program performance & partners’ outcomes

  • EVALUTION:

design & conduct a use-oriented evaluation

assessing internal performance
assessing internal performance

Behaviour Changes

Program

assessing influence
assessing influence

Behaviour Changes

Program

within their context
…within their context

Behaviour Changes

Program

slide46

progress markers

  • A graduated set of statements describing a progression of changed behaviours in the boundary partner
  • Describe changes in actions, activities and relationships leading to the ideal outcome; shows story of change
  • Articulate the complexity of the change process
  • Can be monitored & observed
  • Permit on-going assessment of partner’s progress (including unintended results)
slide47

progress markers are graduated

  • move from easier to more difficult to achieve changes in behaviour
  • describe the change process of a single boundary partner
  • are more complete than a single indicator
slide48

how many progress markers?

Suggestion: total of 15, with most occurring in the “like to see” range

Expect

4

Like

8

Love

3

Life of program

  • Remember, more PMs = more data points to monitor
slide49

progress markers = ladder of change

Outcome challenge

Love to see

Truly transformative

Set quite high

Like to see

More active learning,

engagement

Expect to see

Early response to

program’s basic activities

slide52
The program intends to see researchersinZambia that have gained the expertise required to carry out HIV/AIDS research in a multidisciplinary fashion using quantitative, qualitative, and/or participatory methodologies. They are conducting research on the legislative, economic, social, and health aspects of HIV/AIDS with special emphasis on vulnerable groups including youth and women. They are providing credible evidence that allows research findings to be translated into effective HIV/AIDS policies and programs. They are skillfully disseminating research results in the appropriate format for the audience (e.g., to the scientific community, the general public, NGOs, government officials). They are successfully obtaining funding from multiple sources to conduct HIV/AIDS research. More experienced researchers are mentoring those new to the field and HIV/AIDS researchers globally are engaged in productive communications using various means especially electronic networks.
progress marker checklist
progress marker checklist

Each Progress Marker:

  • Describes a changed behaviour by the boundary partner
  • Can be monitored & observed

As a set, Progress Markers:

  • Are graduated from easier to more difficult to achieve changes in behaviour
  • Describe the change process of a single boundary partner
uses of om
uses of OM
  • For strategic planning and monitoring of development and research projects and programs;
  • To frame the evaluation of a global forestry program and a knowledge network that had not used OM in its planning;
  • To conduct a self-assessment of a project to gather knowledge and lessons in preparation for an external evaluation by the donor agency;
  • To plan an organizational redesign for a regional NGO in West Africa;
  • To facilitate planning with women`s self-help groups in rural India in conjunction with other PRA tools;
  • As a framework for social research in technology interventions
slide56

Thank you!

For more information, examples of

use and to share your experienceswww.outcomemapping.ca

www.idrc.ca/evaluation