Monitoring and evaluation in cba the specificity of m e in the field of adaptation
Download
1 / 28

Monitoring and Evaluation in CBA - The specificity of M&E in the field of adaptation - PowerPoint PPT Presentation


  • 121 Views
  • Uploaded on

Monitoring and Evaluation in CBA - The specificity of M&E in the field of adaptation. UNDP M&E framework for adaptation, VRA and IAS C. O. Nyandiga and A.F Wittmann Presented at the UNDP CBA- UNV Workshop, August 2-10, Dakar-Senegal. What is Monitoring and Evaluation?.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about ' Monitoring and Evaluation in CBA - The specificity of M&E in the field of adaptation' - emory


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Monitoring and evaluation in cba the specificity of m e in the field of adaptation

Monitoring and Evaluation in CBA- The specificity of M&E in the field of adaptation

UNDP M&E framework for adaptation,

VRA and IAS

C. O. Nyandiga and A.F Wittmann

Presented at the UNDP CBA- UNV Workshop, August 2-10, Dakar-Senegal


What is monitoring and evaluation
What is Monitoring and Evaluation?

  • Monitoring = A continuous data collection and analysis process implemented to assess a project (or program or policy) and compare it with the expected performance

  • Evaluation = A systematic and objective measurement of the results achieved by a project, a program or a policy, in order to assess its relevance, its coherence, the efficiency of its implementation, its effectiveness and its impact, as well as its sustainability

    (source : World Bank)


Why do we do m e
Why do we do M&E ?

  • Monitoring and Evaluation = a systematic and on-going process, that is part of project inception, planning and implementation

  • M&E is necessary to ensure that the OBJECTIVES of the project are met

    M&E

  • Provides project management and stakeholders with on-going indications of progress or lack of progress in achievements of objectives

  • Helps identifying problems and successes of projects, during implementation

  • Provides a basis for corrective actions

  • Helps reinforcement of initial positive results

  • Helps in determining if a project is still relevant


Undp m e adaptation framework
UNDP M&E Adaptation framework

ADAPTATION :

What do we want to measure ?

We want to make sure our objectives are achieved : improving the adaptive capacity / reducing the vulnerability of the communities and the ecosystems on which they rely.

What can we measure ?

  • Coverage : extent to which the project reaches vulnerable stakeholders (individuals, households, businesses, government agencies, policymakers…)

  • Impact : extent to which the project reduces the vulnerability, through policymaking, capacity building…)

  • Sustainability : ability of stakeholders to continue the adaptation process beyond project lifetime

  • Replicability : extent to which projects generate and disseminate results and lessons of value for replication


Undp m e adaptation measurements at project level for all thematic areas
UNDP M&E Adaptation Measurements at Project Level for all Thematic Areas

Coverage

  • i. Number of households, businesses engaged in vulnerability reduction or adaptive capacity development activities, as a proportion of households in the community or region targeted by the project.

  • ii. Number of policies introduced or adjusted to incorporate climate change risks.

  • iii. Number of investment decisions revised or made to incorporate climate change risks.

  • iv. Number of stakeholders (individuals, households, communities, etc.) served by new or expanded climate information management systems (e.g. early warning systems, forecasting,etc.).

    Impact

  • i. Percent change in stakeholders’ behaviours utilizing adjusted practices or resources for managing climate change risks, assessed via QBS.

  • ii. Percent improvement in stakeholders’ capacities to manage climate change (e.g. communicate climate change risks, disseminate information, or make decisions based on high quality information), as relevant, assessed via QBS.

  • iii. Percent reduction in perceived vulnerability:

  • a. Percent improvement in stakeholder perceptions of vulnerability to a recurrence of primary climate change-related threat(s), assessed via QBS.


Undp m e adaptation measurements for all ta ctd
UNDP M+E Adaptation Measurements for all TA - Thematic Areasctd

Sustainability

  • i. Number of beneficiaries of project receiving training in implementation of specific adaptation measures or decision-support tools.

  • ii. Local (or spatially appropriate) availability of skills and resources necessary to continue adaptation after conclusion of project, assessed via QBS.

  • iii. Support for project activities among participating communities as assessed by QBS.

  • iv. Number of outside programmes, policies or projects incorporating project results into their processes.

    Replicability

  • i. Number of “lessons learned” from the project


Cba m e adaptation framework
CBA M&E Adaptation framework Thematic Areas

  • CBA falls within the UNDP Framework for M&E of Adaptation to Climate Change

  • The specificity of CBAdaptation = it is a challenge to monitor and evaluate it, because :

    • “Adaptation is not generally an outcome, but rather consists of a diverse suite of ongoing processes that enable the achievement of development objectives under climate change”

    • “Adaptation cuts across many development objectives”

    • The results of adaptation can be measured in the long-term, whereas projects have a short duration

    • It is difficult to decouple climate risks from other drivers

    • (UNDP Framework for Monitoring and Evaluation of ACC)


M e in the cba programme
M&E in the CBA programme Thematic Areas

  • M&E aims to ensure and show that the CBA objectives are met :

    • Building adaptive capacity

    • Reducing community vulnerability

    • Increasing the resilience of global environmental benefits to climate change (including variability) in the GEF focal areas

  • M&E applies to the 3 CBA outcomes

    • Outcome 1 – Local level (projects)

    • Outcome 2 – National level

    • Outcome 3 – Global level


Cba m e system
CBA M&E system Thematic Areas

  • 4 sets of indicators are monitored

    • The UNDP Adaptation Indicators

      • Adaptive capacities

      • Sustainable community management of natural resources

    • The SGP’s Impact Assessment System (IAS)

      • GEB

      • Livelihood / Empowerment

    • The Vulnerability Reduction Assessment

    • Volunteerism and Inclusive Participation indicator


Cba m e system1
CBA M&E system Thematic Areas

  • 4 sets of indicators are monitored

    • At the project level

    • Aggregated at the national level

    • Then again aggregated at the global level


UNDP adaptation indicators Thematic Areas

  • Adaptive capacity fostered among natural resource dependent communities through awareness building, enhanced access to climate change and scenario information, and improved access to alternative resources

    • 2.1 Number of stakeholders at community level (e.g. businesses, community representatives, CBOs, NGOs) engaged by project and provided with training in climate change risk management and scenario planning.

    • 2.2 Population covered by awareness building programmes to increase understanding of risks associated with climate change among general public and key stakeholder groups.

    • 2.3 Increase in awareness of climate change related risks to natural resources (QBS)

    • 2.4 Percentage change in natural resource dependent population with access to alternative or supplementary livelihood options (QBS)

  • Sustainable community management of natural resources in the face of climate change promoted

    • 4.1 Percentage of population in relevant areas engaged in sustainable community management activities

    • 4.2 Number of measures deployed as part of sustainable resource management activities.

    • 4.3 Percentage of area of concern in which sustainable resource management activities are implemented.

    • 4.4 Number of stakeholders (e.g. families/households) benefiting from sustainable resource management activities (e.g. in terms of increased income or food security).

    • 4.5 Success of sustainable resource management interventions in securing livelihoods and protecting resources (QBS).


Sgp s impact assessment system
SGP’s Impact Assessment System Thematic Areas

  • Why ?

    Measure the Global Environmental Benefits and the Livelihood / Empowermentbenefits generated by the project

  • What ?

    • GEB /

      • Biodiversity-(# of species, innovations/new technologies, local/national policies)

      • Land Degradation- (ha. of land restored, land sustainably managed, tons of soil erosion prevented, # of innovations/new technologies, local/national policies)

    • Livelihood / Empowerment /

      • Poverty Reduction –(# of households or individuals who benefited from the project, income generation achieved through the project…)

      • Capacity Building –(# of NGOs, community groups whose capacities were increased, #of women participating, support ensured from local / governmental institutions…)


Sgp s impact assessment system ctd
SGP’s Impact Assessment System Thematic Areasctd.

  • Who, How ? And When ?

    At project conception, chose one GEB indicator and one LIVELIHOOD / EMPOWERMENT indicator

    During project development, measure the baseline values of the indicators and prepare your monitoring plan (in project proposal) –

    During project implementation, measure the evolutions of the indicators (project reports) – RESPONSIBILITY OF THE GRANTEE

    At the end of the project, measure the final indicator (final participatory evaluation / final report) – RESPONSIBILITY OF THE GRANTEE

  • Monitoring Plan –

  • Whatyouwillmeasure (indicator)

  • Whatis the target value of yourindicator

  • Howyouwillmeasureit

  • Whenyouwillmeasureit

  • Whowillmeasureit

Don’tforget to report the indicators in the SGP database


Examples of the type of possible indicators for geb
Examples of the type of possible indicators for GEB Thematic Areas

Arid & semi-Arid zone ecosystems

  • Local population estimates of arid and semi-arid zone species

  • Changes in extent of undegraded arid and semi-arid habitat.

    Coastal, Marine and Freshwaters:

  • Local population estimates of target aquatic species

  • Changes in extent of healthy coral reef

  • Changes in extent and quality of mangroves (patch size, maximum tree size).

    Forest:

  • Local population estimates of forest species

  • Changes in natural forest extent

  • Number of concessionaries in the formulation of timber extraction

    Mountain:

  • Local population estimates of montane species

  • Changes in extent of undegraded montane habitat

  • Rate of deforestation on slopes >20% in project areas

    Agrobiodiversity:

  • Changes in number of local land races & domestic livestock breeds.

  • Changes in extent of agricultural area using low-input high diversity production methods

  • Incorporation of local land races and indigenous breeds in national breeding programmes

  • Certification standards for agricultural products


Ex post measurements
Ex-post measurements Thematic Areas

WHAT

  • Systematically capture and document off-site, indirect and longer-term impacts- “ex-post evaluation”. Generates GEB Measurements

    HOW

  • Using proxy and development indicators while project is ongoing

  • Using structured QBS during and after project closure

  • External independent consultants

    WHEN

  • Minimum 5 years after project completion-Done by UNDP EO


Vulnerability reduction assessment vra
Vulnerability Reduction Assessment (VRA) Thematic Areas

  • What ?

    The objective of a CBA project is to REDUCE the vulnerability of the community. VRA is a tool that contributes to measure achievement of this objective.

    VRA is a form of Participatory Impact Assessment, which focuses on the community’s own perception of its vulnerability and adaptation capacities.

  • Why ?

    • Measure the community’s perceptions of current and future climate change risks and adaptive capacities (now and in the long-run)

    • Measure the obstacles to adaptation, and the assets

    • Evaluate the sustainability of the project

    • Capture qualitative information / knowledge from the community, essential for project development/management (to make sure the project responds to community priorities) and for knowledge-management

    • Capture quantitative (thus comparable) information (=scores) that will allow to verify the achievement of CBA objectives (at project / national / global levels)


Vulnerability reduction assessment vra1
Vulnerability Reduction Assessment (VRA) Thematic Areas

  • How ? When ?

    VRA is composed of 4 indicators, that are transformed into 4 questions (tailored to the local context).

    These questions form the core of a “VRA participatory workshop”, that is organized at least 3 times in the course of a project, in the community (at the beginning, at mid-course of project implementation, at the end).


Vulnerability reduction assessment vra2
Vulnerability Reduction Assessment (VRA) Thematic Areas

  • The H-Form = the tool that is used for VRA facilitation and data collection


Organizing a vra session
Organizing a VRA session Thematic Areas

Where ?

Choose an appropriate location in the village, according to what you already know about the community.

The location has to be accepted by / accessible to everyone

When ?

Accommodate the community schedule, including women’s schedule

Who ?

Facilitating team :

1 or 2 lead facilitators (with leverage and experience ; who know the local language and culture)

1 person to take notes on the H-form

1 person to take extensive notes on the side

1 person to take pictures and coordinate the logistics

MOBILIZE the grantee as well as community-members.

BE GENDER-SENSITIVE

QUICK TIPS

-Make sure all the members of the community are invited and informed about the meeting

-Create a friendly atmosphere, where everyone is at ease to speak

-Promote open discussion (the participants are the “teachers”, because we want to learn from them and capture their perceptions)

-Use a language that is appropriate

-Prepare your H-forms in advance (as well as all the material you might need)

-Inform the authorities in advance


Sequencing a vra session example from morocco
Sequencing a VRA session (example from Morocco) Thematic Areas

  • 1-Introduction

    • Present the facilitating team

    • Explain the context of the workshop (CBA project) and the objectives of the workshop

    • Explain the “rules” : promote open participation (there is no wrong / right answer; everyone is allowed to speak…)

  • 2-Launch a general discussion about the local environment and climate

  • Use simple questions and tackle climate aspects one by one

    • what is the “usual” climate here ?

    • when does it rain ? when is it hot ? For how long…

    • Are there any storms, droughts ? When ?

    • you can draft a season calendar for example

  • 3-How does the community experience climate change?

  • Have you noticed any changes in the past years ?

  • Has it rained more ? less ? When did it rain last year ? for how long ?

  • Is it hotter than in the past ? is the dry season longer ?

  • Are there more storms etc.

  • QUICK TIPS

    • -Adapt to relevant climatic matters

    • -Be very specific : referring to “climate change” is too vague in most contexts

    • -Ask simple questions one by one, so people have time to discuss them


    Sequencing a vra session example from morocco1
    Sequencing a VRA session (example from Morocco) Thematic Areas

    4-What are the impacts of these changes on your livelihood

    = VRA QUESTION 1 / Fill out H-form

    5-What will be the impacts if these changes (more storms / more droughts) double in the future ?

    = VRA QUESTION 2 / Fill out H-form

    The impacts are noted in 2 columns (positive impacts / negative impacts)

    After the discussion, assessment : how serious are these impacts, in the view of the community

    What are the possible solutions to limit these impacts ?

    6-What are the obstacles / assets for adaptation ?

    = VRA QUESTION 3 / Fill out H-form

    What prevents you from implementing the solutions that you mention ?

    After the discussion, assessment : are the obstacles stronger than the assets

    What are the possible solutions to overcome the obstacles and build on the assets ?

    7-Do you think that with the project, you will be able to adapt further in the future ?

    = VRA QUESTION 4 / Fill out H-form

    (Assess the sustainability of the project, and how people will contribute)

    After the discussion, assessment : are the participants optimistic about the outcomes of the projects ? are they willing to actively participate ?

    8-Conclusion

    Thank people for their participation

    Present the next steps of the project


    Vulnerability reduction assessment vra3
    Vulnerability Reduction Assessment (VRA) Thematic Areas

    • What are the results of a VRA session

      • QUANTITATIVE DATA = SCORING

        Generate a score for each question (noted on 5; 1 shows that the vulnerability is high / 5 shows that the vulnerability is low

        Calculate the average from each question

        Multiply the total average by 2 to obtain a score noted on 10

      • QUALITATIVE DATA = all the information captured during the session, and classified in the different parts of the H-form

        • Negative impacts

        • Positive impacts

        • Ideas, Solutions

  • How to use the results of a VRA session

    • The initial VRA results

      • Are used for project development an incorporated in the project proposal (provides data for baseline assessment / local knowledge / and ideas for project outcomes/outputs)

      • Provide the baseline vulnerability measurement, reference for project monitoring

    • The 2nd and 3rd VRA results are to be compared with baseline, in order to monitor the project.

  • QUICK TIPS

    • Always make sure you thoroughly document the VRA sessions

      • Take detailed notes on your own H-form,

      • Note your personal side-comments –how many people ? Where ? When ? Who facilitated ? What went right / wrong ? What can be improved ? …

      • Take pictures

    Don’tforget to report the indicators in the SGP database


    Vulnerability reduction assessment vra4
    Vulnerability Reduction Assessment (VRA) Thematic Areas

    • Lessons learned from implementation

      • VRA is a powerful way to capture a lot of information from the community

      • It gives an opportunity for mobilization and discussion, and contributes to fostering community ownership of the project

      • Language and communication are critical

      • Gender appropriateness is essential (in the choice of the location, scheduling, language, facilitators) : in Morocco, for example, we do specific women’s workshops

      • VRA is a flexible tool : its organization can be tailored to the specific context (for ex. In Morocco, we don’t always use the big H-form, and we added some questions/discussions to prepare for the VRA questions) – PROVIDED ALL THE INFO IS COLLECTED FROM THE DISCUSSION

      • VRA can be incorporated in a larger workshop (information about CC, training, rural assessment…) so that the communities are prepared and so that the discussions are to the point.

      • The main challenge is the QUANTITATIVE DATA (for discussion)

        How can they be better understood (scale of value is not necessarily translated in numbers) ?

        How can they be better captured (voting/scoring might block participants; people don’t want to stand out)?

        How can they be more reliable for monitoring (composition of the group changes from one VRA to the next ; participants might over/underestimate the scores in order to ensure the funding… ?

        There is a potential for improvement, in order to be able to prove the impacts of our projects.


    Volunteerism inclusive participation
    Volunteerism / Inclusive participation Thematic Areas

    BASELINE indicators (project proposal)

    • What are the mechanisms for volunteerism that already exist in the community before the CBA project (for example, traditional mechanisms for mutual assistance, associations, etc.)?

    • Number of volunteers in the community already engaged in climate change adaptation activities before the CBA project

    • What are the opportunities or obstacles that could facilitate or impede people from engaging in voluntary activities?

      The CBA project aims at reinforcing Volunteerism : these indicators should be monitored over the course of the project

      Be sure to capture quantitative / qualitative data on community volunteers and participation during the project : in the activity reports / follow-up on the Volunteerism chart

      Inclusive participation : monitor the vulnerable group’s participation (esp. women, youth, people with a handicap, elders, property-less farmers…). In the activity reports / follow-up on the Volunteerism chart


    M e timeline
    M&E timeline Thematic Areas


    Conclusion and discussion
    Conclusion and discussion Thematic Areas

    • More and more, we are asked to prove our results.

    • M&E is ESSENTIAL to show that our projects achieve their objectives.

    • This is why we need to assess the challenges of our system, and try to improve it.


    Please share
    Please Thematic Areasshare …

    • How do you implement our M&E ?

    • How do you explain it to the proponents / communities, and support them in implementing ?

    • What are the challenges you are facing ?

    • What do you need to overcome these challenges ?


    Questions
    QUESTIONS Thematic Areas

    • Quantitative aspect of VRA ?

    • Consolidate our 4 set of indicators into 1, without losing any of the information ?

    • Incorporate project outcome indicators ?


    ad