Monitoring and evaluation in cba the specificity of m e in the field of adaptation
This presentation is the property of its rightful owner.
Sponsored Links
1 / 28

Monitoring and Evaluation in CBA - The specificity of M&E in the field of adaptation PowerPoint PPT Presentation


  • 90 Views
  • Uploaded on
  • Presentation posted in: General

Monitoring and Evaluation in CBA - The specificity of M&E in the field of adaptation. UNDP M&E framework for adaptation, VRA and IAS C. O. Nyandiga and A.F Wittmann Presented at the UNDP CBA- UNV Workshop, August 2-10, Dakar-Senegal. What is Monitoring and Evaluation?.

Download Presentation

Monitoring and Evaluation in CBA - The specificity of M&E in the field of adaptation

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Monitoring and evaluation in cba the specificity of m e in the field of adaptation

Monitoring and Evaluation in CBA- The specificity of M&E in the field of adaptation

UNDP M&E framework for adaptation,

VRA and IAS

C. O. Nyandiga and A.F Wittmann

Presented at the UNDP CBA- UNV Workshop, August 2-10, Dakar-Senegal


What is monitoring and evaluation

What is Monitoring and Evaluation?

  • Monitoring = A continuous data collection and analysis process implemented to assess a project (or program or policy) and compare it with the expected performance

  • Evaluation = A systematic and objective measurement of the results achieved by a project, a program or a policy, in order to assess its relevance, its coherence, the efficiency of its implementation, its effectiveness and its impact, as well as its sustainability

    (source : World Bank)


Why do we do m e

Why do we do M&E ?

  • Monitoring and Evaluation = a systematic and on-going process, that is part of project inception, planning and implementation

  • M&E is necessary to ensure that the OBJECTIVES of the project are met

    M&E

  • Provides project management and stakeholders with on-going indications of progress or lack of progress in achievements of objectives

  • Helps identifying problems and successes of projects, during implementation

  • Provides a basis for corrective actions

  • Helps reinforcement of initial positive results

  • Helps in determining if a project is still relevant


Undp m e adaptation framework

UNDP M&E Adaptation framework

ADAPTATION :

What do we want to measure ?

We want to make sure our objectives are achieved : improving the adaptive capacity / reducing the vulnerability of the communities and the ecosystems on which they rely.

What can we measure ?

  • Coverage : extent to which the project reaches vulnerable stakeholders (individuals, households, businesses, government agencies, policymakers…)

  • Impact : extent to which the project reduces the vulnerability, through policymaking, capacity building…)

  • Sustainability : ability of stakeholders to continue the adaptation process beyond project lifetime

  • Replicability : extent to which projects generate and disseminate results and lessons of value for replication


Undp m e adaptation measurements at project level for all thematic areas

UNDP M&E Adaptation Measurements at Project Level for all Thematic Areas

Coverage

  • i. Number of households, businesses engaged in vulnerability reduction or adaptive capacity development activities, as a proportion of households in the community or region targeted by the project.

  • ii. Number of policies introduced or adjusted to incorporate climate change risks.

  • iii. Number of investment decisions revised or made to incorporate climate change risks.

  • iv. Number of stakeholders (individuals, households, communities, etc.) served by new or expanded climate information management systems (e.g. early warning systems, forecasting,etc.).

    Impact

  • i. Percent change in stakeholders’ behaviours utilizing adjusted practices or resources for managing climate change risks, assessed via QBS.

  • ii. Percent improvement in stakeholders’ capacities to manage climate change (e.g. communicate climate change risks, disseminate information, or make decisions based on high quality information), as relevant, assessed via QBS.

  • iii. Percent reduction in perceived vulnerability:

  • a. Percent improvement in stakeholder perceptions of vulnerability to a recurrence of primary climate change-related threat(s), assessed via QBS.


Undp m e adaptation measurements for all ta ctd

UNDP M+E Adaptation Measurements for all TA -ctd

Sustainability

  • i. Number of beneficiaries of project receiving training in implementation of specific adaptation measures or decision-support tools.

  • ii. Local (or spatially appropriate) availability of skills and resources necessary to continue adaptation after conclusion of project, assessed via QBS.

  • iii. Support for project activities among participating communities as assessed by QBS.

  • iv. Number of outside programmes, policies or projects incorporating project results into their processes.

    Replicability

  • i. Number of “lessons learned” from the project


Cba m e adaptation framework

CBA M&E Adaptation framework

  • CBA falls within the UNDP Framework for M&E of Adaptation to Climate Change

  • The specificity of CBAdaptation = it is a challenge to monitor and evaluate it, because :

    • “Adaptation is not generally an outcome, but rather consists of a diverse suite of ongoing processes that enable the achievement of development objectives under climate change”

    • “Adaptation cuts across many development objectives”

    • The results of adaptation can be measured in the long-term, whereas projects have a short duration

    • It is difficult to decouple climate risks from other drivers

    • (UNDP Framework for Monitoring and Evaluation of ACC)


M e in the cba programme

M&E in the CBA programme

  • M&E aims to ensure and show that the CBA objectives are met :

    • Building adaptive capacity

    • Reducing community vulnerability

    • Increasing the resilience of global environmental benefits to climate change (including variability) in the GEF focal areas

  • M&E applies to the 3 CBA outcomes

    • Outcome 1 – Local level (projects)

    • Outcome 2 – National level

    • Outcome 3 – Global level


Cba m e system

CBA M&E system

  • 4 sets of indicators are monitored

    • The UNDP Adaptation Indicators

      • Adaptive capacities

      • Sustainable community management of natural resources

    • The SGP’s Impact Assessment System (IAS)

      • GEB

      • Livelihood / Empowerment

    • The Vulnerability Reduction Assessment

    • Volunteerism and Inclusive Participation indicator


Cba m e system1

CBA M&E system

  • 4 sets of indicators are monitored

    • At the project level

    • Aggregated at the national level

    • Then again aggregated at the global level


Monitoring and evaluation in cba the specificity of m e in the field of adaptation

UNDP adaptation indicators

  • Adaptive capacity fostered among natural resource dependent communities through awareness building, enhanced access to climate change and scenario information, and improved access to alternative resources

    • 2.1 Number of stakeholders at community level (e.g. businesses, community representatives, CBOs, NGOs) engaged by project and provided with training in climate change risk management and scenario planning.

    • 2.2 Population covered by awareness building programmes to increase understanding of risks associated with climate change among general public and key stakeholder groups.

    • 2.3 Increase in awareness of climate change related risks to natural resources (QBS)

    • 2.4 Percentage change in natural resource dependent population with access to alternative or supplementary livelihood options (QBS)

  • Sustainable community management of natural resources in the face of climate change promoted

    • 4.1 Percentage of population in relevant areas engaged in sustainable community management activities

    • 4.2 Number of measures deployed as part of sustainable resource management activities.

    • 4.3 Percentage of area of concern in which sustainable resource management activities are implemented.

    • 4.4 Number of stakeholders (e.g. families/households) benefiting from sustainable resource management activities (e.g. in terms of increased income or food security).

    • 4.5 Success of sustainable resource management interventions in securing livelihoods and protecting resources (QBS).


Sgp s impact assessment system

SGP’s Impact Assessment System

  • Why ?

    Measure the Global Environmental Benefits and the Livelihood / Empowermentbenefits generated by the project

  • What ?

    • GEB /

      • Biodiversity-(# of species, innovations/new technologies, local/national policies)

      • Land Degradation- (ha. of land restored, land sustainably managed, tons of soil erosion prevented, # of innovations/new technologies, local/national policies)

    • Livelihood / Empowerment /

      • Poverty Reduction –(# of households or individuals who benefited from the project, income generation achieved through the project…)

      • Capacity Building –(# of NGOs, community groups whose capacities were increased, #of women participating, support ensured from local / governmental institutions…)


Sgp s impact assessment system ctd

SGP’s Impact Assessment System ctd.

  • Who, How ? And When ?

    At project conception, chose one GEB indicator and one LIVELIHOOD / EMPOWERMENT indicator

    During project development, measure the baseline values of the indicators and prepare your monitoring plan (in project proposal) –

    During project implementation, measure the evolutions of the indicators (project reports) – RESPONSIBILITY OF THE GRANTEE

    At the end of the project, measure the final indicator (final participatory evaluation / final report) – RESPONSIBILITY OF THE GRANTEE

  • Monitoring Plan –

  • Whatyouwillmeasure (indicator)

  • Whatis the target value of yourindicator

  • Howyouwillmeasureit

  • Whenyouwillmeasureit

  • Whowillmeasureit

Don’tforget to report the indicators in the SGP database


Examples of the type of possible indicators for geb

Examples of the type of possible indicators for GEB

Arid & semi-Arid zone ecosystems

  • Local population estimates of arid and semi-arid zone species

  • Changes in extent of undegraded arid and semi-arid habitat.

    Coastal, Marine and Freshwaters:

  • Local population estimates of target aquatic species

  • Changes in extent of healthy coral reef

  • Changes in extent and quality of mangroves (patch size, maximum tree size).

    Forest:

  • Local population estimates of forest species

  • Changes in natural forest extent

  • Number of concessionaries in the formulation of timber extraction

    Mountain:

  • Local population estimates of montane species

  • Changes in extent of undegraded montane habitat

  • Rate of deforestation on slopes >20% in project areas

    Agrobiodiversity:

  • Changes in number of local land races & domestic livestock breeds.

  • Changes in extent of agricultural area using low-input high diversity production methods

  • Incorporation of local land races and indigenous breeds in national breeding programmes

  • Certification standards for agricultural products


Ex post measurements

Ex-post measurements

WHAT

  • Systematically capture and document off-site, indirect and longer-term impacts- “ex-post evaluation”. Generates GEB Measurements

    HOW

  • Using proxy and development indicators while project is ongoing

  • Using structured QBS during and after project closure

  • External independent consultants

    WHEN

  • Minimum 5 years after project completion-Done by UNDP EO


Vulnerability reduction assessment vra

Vulnerability Reduction Assessment (VRA)

  • What ?

    The objective of a CBA project is to REDUCE the vulnerability of the community. VRA is a tool that contributes to measure achievement of this objective.

    VRA is a form of Participatory Impact Assessment, which focuses on the community’s own perception of its vulnerability and adaptation capacities.

  • Why ?

    • Measure the community’s perceptions of current and future climate change risks and adaptive capacities (now and in the long-run)

    • Measure the obstacles to adaptation, and the assets

    • Evaluate the sustainability of the project

    • Capture qualitative information / knowledge from the community, essential for project development/management (to make sure the project responds to community priorities) and for knowledge-management

    • Capture quantitative (thus comparable) information (=scores) that will allow to verify the achievement of CBA objectives (at project / national / global levels)


Vulnerability reduction assessment vra1

Vulnerability Reduction Assessment (VRA)

  • How ? When ?

    VRA is composed of 4 indicators, that are transformed into 4 questions (tailored to the local context).

    These questions form the core of a “VRA participatory workshop”, that is organized at least 3 times in the course of a project, in the community (at the beginning, at mid-course of project implementation, at the end).


Vulnerability reduction assessment vra2

Vulnerability Reduction Assessment (VRA)

  • The H-Form = the tool that is used for VRA facilitation and data collection


Organizing a vra session

Organizing a VRA session

Where ?

Choose an appropriate location in the village, according to what you already know about the community.

The location has to be accepted by / accessible to everyone

When ?

Accommodate the community schedule, including women’s schedule

Who ?

Facilitating team :

1 or 2 lead facilitators (with leverage and experience ; who know the local language and culture)

1 person to take notes on the H-form

1 person to take extensive notes on the side

1 person to take pictures and coordinate the logistics

MOBILIZE the grantee as well as community-members.

BE GENDER-SENSITIVE

QUICK TIPS

-Make sure all the members of the community are invited and informed about the meeting

-Create a friendly atmosphere, where everyone is at ease to speak

-Promote open discussion (the participants are the “teachers”, because we want to learn from them and capture their perceptions)

-Use a language that is appropriate

-Prepare your H-forms in advance (as well as all the material you might need)

-Inform the authorities in advance


Sequencing a vra session example from morocco

Sequencing a VRA session (example from Morocco)

  • 1-Introduction

    • Present the facilitating team

    • Explain the context of the workshop (CBA project) and the objectives of the workshop

    • Explain the “rules” : promote open participation (there is no wrong / right answer; everyone is allowed to speak…)

  • 2-Launch a general discussion about the local environment and climate

  • Use simple questions and tackle climate aspects one by one

    • what is the “usual” climate here ?

    • when does it rain ? when is it hot ? For how long…

    • Are there any storms, droughts ? When ?

    • you can draft a season calendar for example

  • 3-How does the community experience climate change?

  • Have you noticed any changes in the past years ?

  • Has it rained more ? less ? When did it rain last year ? for how long ?

  • Is it hotter than in the past ? is the dry season longer ?

  • Are there more storms etc.

  • QUICK TIPS

    • -Adapt to relevant climatic matters

    • -Be very specific : referring to “climate change” is too vague in most contexts

    • -Ask simple questions one by one, so people have time to discuss them


    Sequencing a vra session example from morocco1

    Sequencing a VRA session (example from Morocco)

    4-What are the impacts of these changes on your livelihood

    = VRA QUESTION 1 / Fill out H-form

    5-What will be the impacts if these changes (more storms / more droughts) double in the future ?

    = VRA QUESTION 2 / Fill out H-form

    The impacts are noted in 2 columns (positive impacts / negative impacts)

    After the discussion, assessment : how serious are these impacts, in the view of the community

    What are the possible solutions to limit these impacts ?

    6-What are the obstacles / assets for adaptation ?

    = VRA QUESTION 3 / Fill out H-form

    What prevents you from implementing the solutions that you mention ?

    After the discussion, assessment : are the obstacles stronger than the assets

    What are the possible solutions to overcome the obstacles and build on the assets ?

    7-Do you think that with the project, you will be able to adapt further in the future ?

    = VRA QUESTION 4 / Fill out H-form

    (Assess the sustainability of the project, and how people will contribute)

    After the discussion, assessment : are the participants optimistic about the outcomes of the projects ? are they willing to actively participate ?

    8-Conclusion

    Thank people for their participation

    Present the next steps of the project


    Vulnerability reduction assessment vra3

    Vulnerability Reduction Assessment (VRA)

    • What are the results of a VRA session

      • QUANTITATIVE DATA = SCORING

        Generate a score for each question (noted on 5; 1 shows that the vulnerability is high / 5 shows that the vulnerability is low

        Calculate the average from each question

        Multiply the total average by 2 to obtain a score noted on 10

      • QUALITATIVE DATA = all the information captured during the session, and classified in the different parts of the H-form

        • Negative impacts

        • Positive impacts

        • Ideas, Solutions

  • How to use the results of a VRA session

    • The initial VRA results

      • Are used for project development an incorporated in the project proposal (provides data for baseline assessment / local knowledge / and ideas for project outcomes/outputs)

      • Provide the baseline vulnerability measurement, reference for project monitoring

    • The 2nd and 3rd VRA results are to be compared with baseline, in order to monitor the project.

  • QUICK TIPS

    • Always make sure you thoroughly document the VRA sessions

      • Take detailed notes on your own H-form,

      • Note your personal side-comments –how many people ? Where ? When ? Who facilitated ? What went right / wrong ? What can be improved ? …

      • Take pictures

    Don’tforget to report the indicators in the SGP database


    Vulnerability reduction assessment vra4

    Vulnerability Reduction Assessment (VRA)

    • Lessons learned from implementation

      • VRA is a powerful way to capture a lot of information from the community

      • It gives an opportunity for mobilization and discussion, and contributes to fostering community ownership of the project

      • Language and communication are critical

      • Gender appropriateness is essential (in the choice of the location, scheduling, language, facilitators) : in Morocco, for example, we do specific women’s workshops

      • VRA is a flexible tool : its organization can be tailored to the specific context (for ex. In Morocco, we don’t always use the big H-form, and we added some questions/discussions to prepare for the VRA questions) – PROVIDED ALL THE INFO IS COLLECTED FROM THE DISCUSSION

      • VRA can be incorporated in a larger workshop (information about CC, training, rural assessment…) so that the communities are prepared and so that the discussions are to the point.

      • The main challenge is the QUANTITATIVE DATA (for discussion)

        How can they be better understood (scale of value is not necessarily translated in numbers) ?

        How can they be better captured (voting/scoring might block participants; people don’t want to stand out)?

        How can they be more reliable for monitoring (composition of the group changes from one VRA to the next ; participants might over/underestimate the scores in order to ensure the funding… ?

        There is a potential for improvement, in order to be able to prove the impacts of our projects.


    Volunteerism inclusive participation

    Volunteerism / Inclusive participation

    BASELINE indicators (project proposal)

    • What are the mechanisms for volunteerism that already exist in the community before the CBA project (for example, traditional mechanisms for mutual assistance, associations, etc.)?

    • Number of volunteers in the community already engaged in climate change adaptation activities before the CBA project

    • What are the opportunities or obstacles that could facilitate or impede people from engaging in voluntary activities?

      The CBA project aims at reinforcing Volunteerism : these indicators should be monitored over the course of the project

      Be sure to capture quantitative / qualitative data on community volunteers and participation during the project : in the activity reports / follow-up on the Volunteerism chart

      Inclusive participation : monitor the vulnerable group’s participation (esp. women, youth, people with a handicap, elders, property-less farmers…). In the activity reports / follow-up on the Volunteerism chart


    M e timeline

    M&E timeline


    Conclusion and discussion

    Conclusion and discussion

    • More and more, we are asked to prove our results.

    • M&E is ESSENTIAL to show that our projects achieve their objectives.

    • This is why we need to assess the challenges of our system, and try to improve it.


    Please share

    Pleaseshare …

    • How do you implement our M&E ?

    • How do you explain it to the proponents / communities, and support them in implementing ?

    • What are the challenges you are facing ?

    • What do you need to overcome these challenges ?


    Questions

    QUESTIONS

    • Quantitative aspect of VRA ?

    • Consolidate our 4 set of indicators into 1, without losing any of the information ?

    • Incorporate project outcome indicators ?


  • Login