Managing for Results: World Bank Perspectives Public Service M&E Learning Network

1. Managing for Results: World Bank Perspectives Public Service M&E Learning Network National Treasury, 14-15 May 2007 Pretoria Andrew O. Asibey, Senior M&E Specialist, World Bank

2. Achieving Results • Each year the World Bank provides substantial resources for development – approximately $20-25 billion a year in grants and loans • The disbursement of huge resources requires that results are embedded in all bank operations and programs

3. What are Results? • The Development Assistance Committee of the OECD defines results as the “output, outcome or impact (intended or unintended, positive and/or negative) of development interventions” • Key instruments for managing for results are the Results Chain and the Results Framework at program and project level • Development of Country-wide M&E Systems

4. Overview of Results-Based Management Concepts Outcomes (Short-term) Outcomes (Medium-term) Activities Outputs Inputs The Results Chain Model PLANING FOR RESULTS Long-term Goal (Impact) Utilization of resources to generate products and services Resources committed to program activities Products and services to be used to simulate the achievement of results Long-term, widespread improvement in society Effects or behavior changes resulting from program/project outputs TIME Results Implementation Implementation M&E Results-based M&E

5. Overview of Results-Based Management Concepts Example. Agriculture Results Chain Long-Term Impact Intermediate Outcomes Outcomes Outputs What are the long-term benefits for beneficiaries? Example: Decreased poverty among farmers What are the benefits for beneficiaries? Example: Increased productivity How do the beneficiaries make use of the goods and services provided? Example: use of new technologies goods and services provided through the intervention? Example: Extension services provided

6. Overview of Results-Based Management Concepts Examples of Results Chains Intermediate Outcomes Long-Term Goal (Impact) Outcomes Outputs Education Increased student/ text books ratio Increase literacy rates • Teachers trained • Text Books provided Increased student completion rates Economic Growth New business registration system developed Business administrative barriers reduced Increased private investment Increased GDP Growth

7. Overview of Results-Based Management Concepts Example. Identify the sequence of inputs, activities, outputs, outcomes, and impacts Example of Results Chain (Health Sector) • Information is available for parents about the importance of breast feeding • Children in Local Community healthier • Fewer children are having diarrhea diseases • Mothers Parents breast feeding rather than using formula • New funds available to implement a health project to reduce babies mortality rates • Implement information campaigns on the importance of breast feeding

8. Overview of Results-Based Management Concepts What are the critical elements of an effective M&E System? • Activeinvolvement of customersand partners (planning, conducting, reviewing, & interpreting performance information) • Fullyintegrated into the government’s existingmanagement systems at the central and local level • Responsive to information management and policy making of government institutions at the central and local level • Built aroundreporting requirements • M&Ecapacity in placefor collecting, analyzing, and reporting performance information

9. Overview of Results-Based Management Concepts • Managing for results.Use the information to improve decision making and steer development interventions towards clearly defined goals. Results-Based Management Approach Focus on the results obtained rather than just on the inputs used or the activities conducted. Traditional Management Approach Focused mainly on inputs and activities.

10. The Results Framework

11. Module 3: The Results Framework Results Framework • Program’sstrategy for achieving specific objective • Identifies the Program Development Objective (PDO) and Intermediate Outcomes which are critical to achieve it • Illustrates the development hypothesis and the cause & effect relationships and underlying assumptions, linking all levels of a program’s objectives • (IF a and b happen THEN cshould happen)

12. Module 3: The Results Framework Results Framework • An effective tool for: • Strategic clarity and planning & managing for results • Building consensus and ownership • Effective communication

13. Module 3: The Results Framework • Causal Chain Hypothesis.Results are caused by some interventionsIF… THEN • IFwe build health centersTHENlocal population will use them to for health services • IFwe train teachers on new pedagogic skills • THENteachers will use these skills to improve the quality of education • IFwe increase the salary of the staff • THEN staff will be motivated to work harder

14. Module 3: The Results Framework • Design with clear logic 1. Determine the key development hypothesis & its contribution to higher order impacts Reduction in the child mortality THENincidence of preventable disease will be reduced IFweexpand child immunizations

15. Module 3: The Results Framework • Design with clear logic (cont.) 2. Develop a hierarchy of objectives showing the causal linkages 3.Determine what is possible in the operation (program or project) Reduction in Infant Mortality Expanded health care system Improved nutrition Incidence of preventable disease reduced Increased immunizations Increased family health practices (infrastructure) Vaccines availability Public awareness • DESIGN. Project success depends on the development of the right activities & assumptions • IMPLEMENTATION.Each part of the results chain has a role to play in achieving the PDO Enhance regulations Expand facilities Provide Funds

16. Module 3: The Results Framework • Program Outcomes • The outcomes describe the effect that the program’s outputs will have on the beneficiaries in terms of changed behavior or improved performance. It defines the program’s success. • The outcomes are one step beyond the products of the activities Examples • Increased households access to potable water

17. Module 3: The Results Framework Program Outcomes, cont. • Keep the outcome statement clear, precise, and stated as realistic results— not overly broad or too ambitious • Outcome Statement.Capturesingle outcomeby outcome statement

18. Module 3: The Results Framework Key Features for Developing Good Indicators • “What gets measured gets done” • “If you don’t measure results, you can’t tell success from failure” Indicator: A variable that provides accurate and reliable evidence about the achievement of a specific result. • Work with stakeholders and technical expertsto determine whichindicators are most appropriate.Use the existing indicators • Avoid selection of too many indicatorsLess is Better! • Information about indicators shouldbe easy to gather and useful for management decisions • Performance indicators should be identified at all levels of results chain (show attribution and causal linkages)

19. Module 3: The Results Framework Criteria for Selecting Good Indicators Performance indicators should be SMART: Specific; Measurable; Attributable; Realistic; Targeted • Specific:measure as closely as possible what is intended to measure Indicators Outcome Competitive recruitment process 1. % of public posts filled 2. % of vacancies filled through competitive procedures Increased trade in the region • % reduction of agricultural tariffs 2. Volume of export and import in the region

20. Module 3: The Results Framework Criteria for Selecting Good Indicators Performance indicators should be SMART: Specific; Measurable; Attributable; Realistic; Targeted • Measurable: clear and unambiguous about what is measured • # of Public Institutions successfully using the new financial management system 2. # of Public Institutions reporting 100 % of budget execution through the financial management system report

21. Module 3: The Results Framework Criteria for Selecting Good Indicators Performance indicators should be SMART: Specific; Measurable; Attributable; Realistic; Targeted • Attributable: attributable to the project’s or program’s efforts 1. Poverty incidence rate 2. Annual revenues generated

22. Module 3: The Results Framework Criteria for Selecting Good Indicators Performance indicators should be SMART: Specific; Measurable; Attributable; Realistic; Targeted • Realistic:Data obtained at reasonable cost with enough frequency 1. HIV prevalence among 15-24 year old pregnant women 2. HIV prevalence among the total population

23. Module 3: The Results Framework Criteria for Selecting Good Indicators Performance indicators should be SMART: Specific; Measurable; Attributable; Realistic; Targeted • Targeted: It should be specific about the targeted population/area 1. Percent increase in employment 2. Percent increase in employment of graduate of business school X in the first year after completion of training 8

24. Module 3: The Results Framework Types of Indicators • Quantitative • Qualitative

25. Module 3: The Results Framework Types of Indicators • Quantitative.Measure of dimension or amounts in specific number, percentage, or ratio. The most commonly used indicators. Example Infant mortality rate.

26. Module 3: The Results Framework Types of Indicators (Cont.) • Qualitative.Qualitative assessment of progress. • Used when quantitative data is not available or the indicator must convey a more complex process that is difficult to quantify. They are hard to verify and involve subjective judgment. • Example • Participation of communities in the elaboration of the District plan

27. Next Step • Setting up a detailed M&E Plan for data collection and reporting.

28. Development of Country-Wide M&E Systems • Support to development of PRSP M&E systems • Key instrument is using Institutional Development Fund (IDF) • Development of robust monitoring systems at national/sectoral, regional and district levels • Impact evaluations • Capacity building through hands on training (i.e. demystifying the results agenda) • The Case of Ghana Poverty Reduction Strategy II (GPRS II)