1 / 25

ADB ’ s Evaluation Architecture and Country Evaluations Guidelines

ADB ’ s Evaluation Architecture and Country Evaluations Guidelines. Mala Hettige Senior Advisor Independent Evaluation, ADB. Presentation at the meeting of Sub-Regional Development Institutions Johannesburg, 27 April 2014. Established in 1966 – in Manila, Philippines

aimon
Download Presentation

ADB ’ s Evaluation Architecture and Country Evaluations Guidelines

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. ADB’s Evaluation Architecture and Country Evaluations Guidelines Mala Hettige Senior Advisor Independent Evaluation, ADB Presentation at the meeting of Sub-Regional Development Institutions Johannesburg, 27 April 2014

  2. Established in 1966 – in Manila, Philippines • 67 shareholding member countries • Provides $ 21b in development assistance • About 760 projects • 5 regional department and PS department • Managed by MDG, 6 VPs • 3000 staff in HQ and 29 RMs + 3 rep. offices • Administrative Budget about $600 million Asian Development Bank (AsDB)

  3. Abides by OECD’s 4 dimensions of independence • DGE reports to the Board through the Development Effectiveness Committee (DEC) • Reports public upon DG-IE approval, discussed under Chair, DEC • DGE appointed/removed by DEC – 5 year non-renewable term. • Work Program and separate budget approved by Board • DGE has authority over staff recruitment • Staff subject to backward and forward conflict of interest policy Independent Evaluation @ AsDB

  4. Annual Reports Country Assistance Program Evaluations Corporate Evaluation Studies Impact Evaluation Study Thematic Evaluation Studies Project/Program Performance Evaluations Technical Assistance Program Evaluations Major Evaluation Products Real- time Evaluation Studies Project Validation Reports Other Evaluation Reports 4

  5. Accountability and Learning 5

  6. A database of evaluation lessons • Provides users easy access and search capability • Lessons by sector, theme, topic, country etc. • EVIS is the first system of its kind • Accessible through http://evis.adb.org Evaluation Information System (EVIS) 6

  7. Tracks ADB Management actions on recommendations • IED Validates actions taken by Management • Reports this validation to the Board • Computerized and available to staff and Board • Departments/offices update progress on actions Management Action Record System (MARS)

  8. Country Evaluations and ADB Operations 8

  9. History of Country Evaluations in ADB CAPEs (Country Assistance Program Evaluations) • 1998 first CAPE – Peoples Republic of China • 2002 Mongolia 2002 questioned choices made • 2005 Bhutan had recommendations + criteria ratings • 2006 Indonesia added ratings by sector • 2007 India had contribution to themes+ value added • 2007 China had Strategic and Institutional positioning • 2008 Sri Lanka considered the implications of conflict • 2008 Philippines considered political economy factors • 2009 Cambodia top-down + bottom up assessment • 2010 Lao existing guidelines applied • 2014 Tajikistan assessment of objectives- ongoing

  10. Evolving Guidelines for CAPEs in AsDB • From 1998 to 2006 no guidelines • First Country Evaluation Guidelines 2006 • Revised Guidelines issued 2010 • CAPE/ validation of a country self evaluation mandatory prior to approval of a CPS • Now doing a few revisions to the guidelines • Also preparing validation guidelines

  11. Good Practice Standards (GPS)for Country Evaluations: Adapting it to ADB

  12. Background: GPSon Country Evaluations • GPS for CSPE established in 2008 • Based on OECD–DAC evaluation principles • Draws on the 2007 ECG Review of CSPE • A core GPS criteria allows comparability • An optional GPS criteria helps accountability and learning

  13. Application of GPS: of Country Evaluation • Name of study differs and number differs • Coordinated with the upcoming country strategy • Substituted with validations or learning products • About 9-12 months, 25-75 pps, $ 250+, • Lending/non lending, about 10 years (2-3 cycles) • Application of principles, approach taken differs • Overall Rating and Evaluation Criteria followed by about half. But try not to emphasized much

  14. Key Aspects of CPSE • Objectives: • Accountability – Board on results over a period • Learning - engage with stakeholders on future CPS • Timing : to feed into the next strategy • Coverage: • Long enough to assess results and • Recent to ensure findings are relevant

  15. GPS forCountry Strategy and Program Evaluation • GPS for CSPE established in 2008 • Based on OECD–DAC evaluation principles • Draws on the 2007 ECG Review of CSPE • A core GPS helps allows comparability • An optional GPS helps accountability and learning

  16. Evaluation Criteria CoreCriteria Optional Criteria Positioning Coherence Institutional Development MDB Performance Borrower Performance Partner Coordination • Relevance • Effectiveness • Efficiency • Sustainability • Impact

  17. Relevance • GPS : Relevance refers to the degree to which MDB strategy and program were consistent to critical needs and long-term plans of the country

  18. Relevance –> Strategic Relevance • Can include optional criteria on positioning– how well did MDB respond or anticipate • Evolving development challenges • Priorities of the government • Build on MDB’s comparative advantage • Consider support from other development partners • Also can include coherence – extent to which there were measures to • Foster internal + external synergies within MDB • Foster external partnerships that promote cost-effective division of labor through complimentarities

  19. Efficiency • GPS: Efficiency refers to the extent to which the design and delivery of assistance were most cost effective.

  20. Effectiveness • GPS: Effectiveness refers to the extent to which the assistance instruments achieved the intentions and objectives set.

  21. Sustainability • GPS: Sustainability refers to the likelihood that actual and anticipated results will be resilient to risks beyond the program period.

  22. Impact • GPS: Impact refers to an MDB’s contribution to long-term changes in development conditions.

  23. Impact => Contribution to Development • Optional Criteria • Institutional Development • MDB Performance • Client Performance • Partnership Performance • Other concerns • By sector? • What about cross sector impacts? • By objectives? • How to attribute and measure? • How to account for unintended impacts?

  24. IED Website www.adb.org/evaluation

  25. evaluation@adb.org www.adb.org/evaluation https://www.facebook.com/adbevaluation Follow Us https://twitter.com/adbevaluation http://www.youtube.com/evaluationatadb Thank You! http://www.scribd.com/adbevaluation/ http://www.soundcloud.com/adbevaluation/

More Related