Evaluating adaptation

1. Evaluating adaptation Simon Anderson Contents Key challenges for the evaluation of adaptation Emerging results frameworks, incl. ICF, PPCR TAMD framework and indicators

2. Key challenges for the evaluation of adaptation- attribution or contribution? • Assessing attribution, establishing baselines and targets, contending with long time horizons, and addressing accountability • Attribution assessed through impact evaluation. • Setting counterfactuals through experimental and quasi-experimental design • More meaningful to consider contribution than attribution in complex contexts and a range of techniques can be used to infer counterfactuals when they are not available.

3. Key challenges for the evaluation of adaptation- baselines and targets • Data availability challenges and changing underlying contexts • Techniques to estimate or reconstruct baselines can be useful here, as well as normalisation of outcome data. • Utilising sequential targets can also help a programme adapt to changing contexts and remain relevant to adaptation objectives over time. • Adaptation investments often long term and results may only be judged when climate effects are fully experienced and efficacy of adaptation can be measured. • Limiting factor analysis and statistical modelling are techniques that may support long term evaluations. • There is the need to fund and support longer term evaluation, such as a longitudinal approach to a thematic area of adaptation work.

4. Key challenges for the evaluation of adaptation- addressing accountability • Public disclosure of information • Inclusion of beneficiary satisfaction scores and participatory techniques such as ‘most significant change’. • Downward and upward accountability can also be improved by integrating learning into programme design, so that a flexible, iterative approach allows periodic reflection and improvement.

5. Emerging results frameworks – challenges identified through a review of the EU GCCA programme • Formulating indicators that accurately measure effectiveness of adaptation can be problematic – theories of change and an adaptation pathway required. • Devising indicators to effectively track improvements in climate finance management systems, or changes in climate expenditure can also prove challenging. • Existing data systems may need supplementary climate change information to enable the characterization of climate challenges. • Lack of clarity regarding implementation and institutional arrangements for M+E. • Limited participation in the design and implementation of M+E frameworks. • Lack of specific focus and activities on M+E of adaptation. • Short term and programme/ project centredevaluation .

6. The TAMD approach - indicators, and use of ‘normalised’ development indicators

7. TAMD Indicators Indicators using vulnerability as proxy for developmental outcomes plus conventional development parameters – emphasis on using what’s available. • Examine: • Effectiveness of targeting the climate vulnerable poor; • Based on theories of change and an understanding the objectives of different stakeholders, etc. A set of indicators used to examine the performance of Climate Risk Management functions including adaptation delivery

8. TAMD Steps Scope 1 • 7 • 2 Lessons Learned Theory & Design • Climate risk management • 6 • 3 • Development performance Analysis Indicators • 5 • 4 Data Gathering Methodology

9. Purposes for applying the TAMD approach CLIMATE RISK MANAGEMENT CLIMATE IMPACTS ON DEVELOPMENT INTERVENTIONS What are the comparative advantages of approaches in different interventions How well do investments in CRMgt address the increases in vulnerability How much are investments in CRMgt leading to better adaptation How well does a single intervention enable adaptation How well is climate managed across on sector

10. TAMD framework