Bruce Murray Director General Operations Evaluation Department Asian Development Bank

1. Bangladesh Joint Country Assistance Evaluation: Assessing Total ODA at the Country LevelPresentation to OECD DAC November 2006 Bruce Murray Director General Operations Evaluation Department Asian Development Bank

2. Very Early Stages Everything said is subject to change.

3. Benefits of Total ODA Evaluation: Richard Manning’s Challenge • Consistent with Paris Declaration: (i) mutual accountability; (ii) partnering; (iii) harmonization; (iv) alignment. • Reduced transaction costs for Government • Objectivity and legitimacy • Broader scope • Ability to address issues that are beyond those addressed in evaluations undertaken by one donor • October 2005 Paper “The effectiveness of ODA – an evaluation proposal” focuses on the impact of total ODA on policy choice and implementation in a country.

4. Why Bangladesh as Pilot for Total ODA Evaluation? • Many donors active in country • 4 major donors have a joint country assistance program (WB; ADB; JBIC; DfID) • Some major donors require country evaluations by 2009 • Use of SWAps.

5. Phasing of the Evaluation • Phase 1 : joint country evaluation by the World Bank, ADB, JBIC/JICA and DFID (80% of ODA). • Phase 2: integrate the 28 donors delivering the other 20% of ODA. • Daunting coordination problems. Need for strong information sharing between donors involved in Phases 1 and 2. Mechanism not yet developed.

6. Issues Requiring Resolution: Involving the Government • Involving Government as partner in the evaluation work – Paris Declaration on mutual accountability. • Proposal to Government must be an open one, including on approach and methodology (listen to Government’s views) • Identify issues that the Government wants addressed (e.g., donor coordination; harmonization; use of country systems) • Work with Government evaluation units and local research organizations. • Involvement of NGOs that deliver much of the bilateral aid.

7. Issues Requiring Resolution: Approach and Methodology • Identify issues that cannot be addressed in the country assistance evaluations of individual donors. • What is the counterfactual? • Need for common approach and methodology for the separate building blocks of the evaluation. • Alignment with joint Country Strategy and PRSP. Note: Joint Strategy assigns lead donors by sectors and themes. • Lead donors evaluate all interventions in the sector/thematic area, regardless of funding (data base being assembled to map available evaluation reports and ongoing operations and strategy).

8. Issues Requiring Resolution: Study Management • Need a strong evaluation management mechanism • Composition and role of steering committee – Government, 4 major donors and 1 OECD DAC representative. • Coordination among task managers and teams, mission planning, etc. • How to balance internal needs of donors for evaluation of their programs and joint evaluation (attribution to total ODA, not individual donors) • Reporting – one report on total ODA or do donors also need separate reports on their operations for accountability? Design methodology to be flexible enough to produce single or multiple reports. • Logistics, timing and cost sharing of the evaluation

9. Issues Needing Resolution: Involving other Donors • How to involve the donors delivering the other 20% of ODA? • 1 representative from the OECD DAC Total ODA task force on the steering group • Involvement other donors up front in identifying the evaluation questions. • Sharing of information during the sector/thematic evaluations • Providing information during for the sector/ thematic evaluations • Role of country offices and Headquarters evaluation units.

10. Proposed Schedule

11. Challenges of Joint Evaluations (DAC Guidelines) • Donor need for accountability and attribution often takes precedence over joint efforts to assess interrelated impacts of multiple agencies • Finding subjects suitable for joint evaluations (program support, multilateral, multidonor funded projects) is difficult • Processes for coordinating joint evaluations can be complex and increase the cost and duration of the evaluation

12. DAC Lessons: Joint Evaluations • Joint evaluations only reduce transaction costs if they replace, not add to, individual evaluations • Identify key partners in evaluation; involvement of government agencies as evaluators is good practice • Agree on the management structure early (broad membership steering committee and smaller management group; mixed approaches, some areas of joint evaluation, others to be delivered separately)

13. DAC Lessons: Joint Evaluations • Key areas of agreement: • Plan for dissemination and follow up