IHP+ experience with

1. Assessing impact of health aid: how can public health sciences contribute to mutual accountability? Moving from evaluating the impact of programmes and interventions to health systems and capacity strengtheningPhyllida Travis, 6 February 2012

2. IHP+ experience with • Issues and challenges in communicating results of health aid • Some examples of linking aid effectiveness to 'results' • Of strengthening mutual accountability for results • Approaches to making health aid for system strengthening more effective / efficient

3. Who is IHP+? 2007 2012 Developing countries: 8 30 Bilateral donors: 8 13 Int'l agencies and foundations*: 11 12 TOTAL 27 55 *African Development Bank, Bill and Melinda Gates Foundation, European Commission, GAVI Alliance, Global Fund, International Labour Organization, UNAIDS, UNICEF, UNDP, WHO, UNFPA, World Bank

4. 1. Issues, challenges in communicating results Want results that are memorable, persuasive, credible, useful, but • Politicians / public v public health experts • Country policymakers/managers v global perspective • Stories v numbers • Progress v impact • Impact of aid per se v impact of way give aid • Joint v individual results • Plausible v causal links • 'Quick wins' v sustained gains • Improved messaging now v in 5 years, w new methods

5. Communicating benefits, results • IHP+ country teams meeting 2010: • Clear progress on putting aid effectiveness principles into practice • Need to better document results / outcomes, find ways of conveying better. • Need mix of good evidence, good stories, moderate claims • In 2011 • IHP+Results annual report on signatories' progress - visual; what not why • OECD TTHATS report for Busan - sound; not sexy • Accumulation of narrative country examples - what, why, how; distillation needed • Commission on Information and Accountability - new catalyst

6. May 2011 IHP+ Results report Individual scorecards

7. And comparisons across partners

8. 2. Some country specific examples linking aid effectiveness and results • Ethiopia, Nepal, Mali (all of which have positive trends in service delivery and health outcomes) plus DRC, Benin • Asking 3 questions • To what extent have aid effectiveness principles been put into practice? • Has this contributed to better results? • Has health aid actually become more effective? • Have health systems been strengthened? • Have health services improved? • What was critical in achieving these results? Where were the barriers and how have they been overcome?

9. Aid effectiveness and results…different perspectives • Can trace plausible links from 'putting Paris into practice' via stronger systems to improved services and then …more tenuously….health outcomes • All note 'two speed implementation' of Paris agenda • Is also an argument that goes • Outcomes are important • Proving link between AE and results (outcomes) is more of a global than country concern • Risks excluding other important, common sense benefits to countries of working to 'one plan, one budget, one report' Need frame the debate to reflect wider range of benefits/results

10. Links between effective aid and results: examples • Nepal: scale up of free maternal health care from a few districts to nation-wide over 2-3 years was possible because government and donors acted collectively – no one could have done it alone. Institutional deliveries risen from18% 2006 to 28% 2011. • Ethiopia: Health extension worker programme went nation-wide over 5 years, faster than many thought possible, because FMOH provided a clear vision; credible plan and indicated where support was needed from DPs • DRC: A new MOH single donor coordination unit led to threefold reduction in management costs for donor funds, from 28% - 9%. Liberated funds for other uses. Message: better coordination of resources can deliver greater value for money.

11. DRC: Reform of health financing – some results MOH wanted to see The reform benefits from the support, both technical and financial, of all partners even if its utilisation is gradual

12. AE principles in practice? Other types of results one MOH is interested in

13. 3. Strengthening mutual accountability • MA important route towards greater aid effectiveness, but slow progress • Mutual accountability efforts in health • Countries • MOH and EDPs: Compacts; some MOHs incorporating indicators of DP aid behaviour into JAR agendas eg Mozambique, Ethiopia, Nepal… • MOH and parliaments: politicians getting involved eg Nigeria • MOH and CSOs, research institutes – mixed; efforts to increase; Uganda • Global / regional level • 3rd round of monitoring by IHP+Results – 19 countries in 2012 (10 last time); 15 DPs • Upcoming meeting of southern health CSOs – will discuss scorecards as one tool for MA • Note: mutual v joint accountability / responsibility crops up a lot in-country

14. 4. Making health aid more effective in health system and capacity strengthening: Nepal • Existing instruments • 5 year health sector programme (NHSP2), results framework; partnership agreements; joint annual review (JAR) • 2012 JAR: systems problems prominent • E.g. only 15% of facilities had no drug stock outs (target 70%) • Getting TA better aligned with NHSP-2 priority problems a major theme • New actions to harmonize/align TA with priority problems • TA matrix against NHSP2 objectives identifies gaps / overlaps • Joint technical co-operation agreement agreed • To match TA to priorities in Annual Work Plan and Budget • Ie not just about systems knowledge/ research but practical instruments

15. Factors enabling aid to effectively contribute to health systems and capacity strengthening • Critical enabling factors seem to be • Government ownership • Leadership from MOH and at least some DPs • Strategic plans and review processes established and taken seriously • Joint financing arrangements that are adhered to (including more predictable funding), complemented by more joined up TA? • Some home-grown form of mutual accountability • Common barriers • DP own organizational incentives/adaptability • Willingness to synchronize work • Trust

17. spares

18. Moving from impact evaluation to health systems and capacity strengthening • An aid effectiveness perspective…joint responsibility • Context • post Busan; • Old chestnuts • One does not follow the other – move in parallel • what politicians and the public listen to – and when • Impact has to be more than health • Benefits of 'more effective aid' often on more proximal aspects of system performance • Existing approaches – 2 country examples • New developments • Increased emphasis • Utility of compacts – what they contain

19. Common monitoring framework of health progress and performance Inputs & processes Outputs Outcomes Impact Improved health outcomes & equity Social and financial risk protection Responsiveness Efficiency Intervention access & services readiness Intervention quality, safety Governance Financing Coverage of interventions Prevalence risk behaviours & factors Infrastructure; ICT Health workforce Supply chain Information Indicator domains Data collection Population-based surveys Coverage, health status, equity, risk protection, responsiveness Facility assessments Service readiness Administrative sources Financial tracking system; NHA Databases and records: HR, infrastructure, medicines etc. Policy data Clinical reporting systems quality, coverage, health status Civil registration Analysis & synthesis Communication & use Data quality assessment; Synthesis of progress and performance; Evaluation Regular country health sector review processes; Global reporting

20. Internationalfunding remains unpredictable

21. Growing numbers of players in health: Chad ONG nationales et internationales DED Nations Unies MSFs CCSI MDM EMEET Croix Rouge UNAD ACF OMS COOPI MENTHOR CSAI CICR ONUSIDA BASE BM Ordre de Malte UNFPA PAM BAD Instit. fin. int. UNICEF UNHCR Libye Egypte PNUD USAID Allemagne Chine OCHA Suisse FAO Cuba France GAVI ECHO Initiatives mondiales Fonds mondial contre le Sida, la tuberculose et le paludisme UE Membres de l'UE

22. issues • Focus on (quick) results does not need to compromise HSS / CB efforts, but needs to be managed • Results need to be expressed in different ways • Traditional approaches to HS and capacity strengthening mixed record • AE p's apply to this as

23. Links back to impact evaluations / public health research • One of the 9 standard JAR background documents is a research report, summarising major health related studies in Nepal in last year. • In 2011 research report covered findings from • Major periodic surveys eg DHS, NLSS, service tracking survey • Financing studies commissioned as part of strategy development • Studies on effectiveness of new policies eg reducing barriers to maternal health services • Studies on aid effectiveness and results • Improving background documents is an iterative process • Some research topics identified in JAR eg on procurement

24. Snapshot of Mali • Progress with putting AE principles into practice • Coherent planning framework (PRODESS+, MTEF); stronger MOH ownership; more alignment with strategies; less with systems; greater management for results; some greater donor harmonization, MA • Results 1: more effective aid • Mixed: more predictable pledges; disbursement worse; some transaction costs seen as investment (but many still negative); TA more focused on systemic priorities • Result 2: health system strengthening

25. Important progress in implementing aid effectiveness principles practice and behavior changes  highly probable contribution to HSS and sector results BUT “two-speed implementation” of the principles of the Paris Declaration and IHP+: Only 13/50 donors signed the Compact and 3 do SBS Sharp decrease in executed ext. financing since 2008 Still many earmarked funding (thematic./geograph.) Specific procedures and PIUs Aid not predictable Individual missions / audits in addition to joint ones  Expectations must be realistic Mali: Conclusion and perspectives 27 October 2011 Mali Case Study 25