Progress Report to the Fifth Meeting of the DAC Evaluation Network, November 16, 2006

1. Developing Consensus Principles and Standards for Evaluating Global and Regional Partnership Programs (GRPPs) Progress Report to the Fifth Meeting of the DAC Evaluation Network, November 16, 2006 Chris Gerrard GRPP Coordinator, IEG

2. Outline of the Presentation • Background • Stakeholder consultative workshop held onSeptember 28-29, 2006 • Presentation of the revised Draft Principles and Standards document • Demand for a second, companion document • IEG’s continuing role moving forward

3. Background to the Presentation • IEG’s presentation to the Fourth Meeting onMarch 31, 2006 • Broad support expressed for collectively developing consensus principles and standard for evaluating GRPPs • Request that IEG play a leading role in developing them

4. What are GRPPs? • Programmatic partnerships in which: • The partners contribute and pool resources (financial, technical, staff, reputational, etc.) toward achieving agreed-upon objectives over time • The activities of the program are global, regional, or multi-country (not single country) in scope • The partners establish a new organization with a governance structure and management unit to deliver these activities. • The majority of GRPPs are housed in existing (host) organizations

5. Developing Consensus Evaluation Principles • Why? • To improve the independence and quality of program-level evaluations of GRPPs in order to enhance their relevance and effectiveness. • For Whom? • Governing bodies and management units of GRPPs • Evaluators engaged in evaluating these programs • High-level policy makers in aid agencies and developing countries

6. Workshop ParticipantsSeptember 28-29, 2006 17 Bilateral agencies and OECD secretariat 6 UN organizations 6 Multilateral development banks 3 Foundations 2 Developing country chairs of governing bodies 5 NGOs/private sector 6 GRPP management 3 GRPP evaluators 3 Evaluation associations

7. Objectives of the Workshop To review an initial set of draft principles and standards for GRPPs which IEG had prepared, based upon: • Extensive review of existing sets of related principles • DAC Principles and Standards • UNEG Norms and Standards • ECG, evaluation associations, etc. • Evaluative work that IEG done on GRPPs over the past few years: www.worldbank.org/ieg/grpp

8. A Self-Standing and Comprehensive Document • Evaluation Governance and Process Issues • Enabling conditions, including M&E framework • Independence and impartiality • Participation and inclusion • Transparency and disclosure • Planning for scope and methodology • Evaluation Content and Criteria Issues • Relevance • Effectiveness (or efficacy) • Efficiency or cost-effectiveness • Governance and management • Sustainability, risk, and strategy for devolution or exit

9. Major Improvements Between First and Second Drafts • Enabling conditions, including M&E framework • Independence and impartiality • Participation and inclusion • Planning for scope and methodology • Relevance • Effectiveness (or efficacy) • Efficiency • Governance and Management • Sustainability

10. Demand for a Second Companion Document • Workshop participants who represented the governing bodies and management of GRPPs liked the structure and content of the present document. • Evaluators working in bilateral agencies, UN organizations and MDBs as well as GRPP evaluators desired something more detailed and specific to GRPPs, including guidance notes and good practice examples on specific issues • A number of participants also offered to help contribute to the second document

11. Particular Requests for Guidance Notes and Good Practice Examples • Establishing good results-based M&E systems • Developing generic indicators for generic types of GRPP activities • Ensuring independence of evaluation and mitigating conflicts of interest in small programs • Preparing terms of references, including scope and methodology • Assessing relevance, effectiveness, and efficiency • Assessing legitimacy and effectiveness of governance and management • Assessing alternative strategies for devolution or exit

12. M&E Framework Often Lacking Among 36 Programs IEG Has Reviewed

13. Information on Evaluation Process Also Lacking in Evaluation Reports

14. Information on Evaluation Process inEvaluation Reports (cont.)

15. Difficult to Assess Degree of Independence from Evaluation Reports

16. Most Evaluations Assessed Relevance; Few Assessed Efficiency

17. Most Evaluations Used Interviews & Document Reviews; Half Used Surveys

18. Suggested Options for Moving Forward • Disseminate the draft Principles and Standards widely, without endorsing them at this time • Encourage GRPP governing bodies & management to be guided by these, and to provide feedback based on experience • Revise the document in 1-2 years in the light of experience and feedback received • Start preparing the second companion document, based on: • Reviews of existing GRPP evaluation policies, M&E frameworks, and completed evaluations • Consultations with partner representatives on governing bodies, GRPP management, GRPP evaluators, evaluation associations, and other evaluation experts