Monitoring and Evaluation in MCH Programs and Projects

1. Monitoring and Evaluation in MCH Programs and Projects MCH in Developing Countries Feb 10, 2011

2. Session objectives: • Explain similarities and differences between monitoring and evaluation • Describe the major purposes, types and tools for MCH program evaluation • Contrast traditional evaluation approaches with participatory evaluation • Develop a monitoring plan for one program activity

3. Objectives of both monitoring & evaluation: • Improve program planning process • Improve program management • Improve program performance • Assess program effects on beneficiaries • Strengthen links with participant groups

4. What is monitoring?Why do we do it?

5. Monitoring: • The periodic, regular collection and analysis of selected indicators • Conducted to determine whether key activities are being carried out as planned, and identify corrections needed • The MIS (management information system) provides early indication of progress, or a lack of progress

6. Monitoring: • Can be carried out by any staff, not a specific evaluation staff person • Informs a project about strengths & weaknesses of the strategies: do we need to refine them? • Important: use it to provide regular feedback to staff

7. Is Monitoring Supervision?

8. Is Monitoring Supervision? • We monitor implementation of an intervention – typically output and process indicators • We supervise an individual

9. Monitoring: the regular collection and analysis of selected indicators conducted to determine whether key activities are being carried out as planned, and identify corrections needed Evaluation: done at longer intervals than monitoring conducted to provide an indication of success of a program, and identify problem areas Monitoring and Evaluation:how are they related?

10. Characteristics of Evaluation • Assesses the value of something • Systematic and ‘objective’ review of evidence related to specific questions about a program • Carried out selectively – costly and time consuming • Assess performance and effects of program efforts -- requires analysis • Process: outputs, organization, management • Outcome or impact: effects on beneficiaries

11. Evaluation ProvidesInformation on: • Strategy – Are the right things being done? • Operations – Are things being done right? • Learning – Is it having the desired effects? Are there better ways?

12. Types of evaluation: • Formative • Summative • Of relative need for a program • Of the feasibility of a program design • Of program performance or process • Of direct effects or outcomes of a program • Of long-term impact • Internal or external

13. Internal Evaluations • Staff know more history, organization, culture, problems, successes • Can interpret evaluation findings more accurately • May be too close, hard to be ‘objective’ • Part of power structure, may have personal agendas • Likely not highly trained in evaluation • Known to the program so less likely to be threatening, disruptive

14. External evaluators • Can take a ‘fresh’ look at the program • Not personally involved; more ‘objective’ • Not a part of the normal power structure • Less chance of personal gain • Well trained in evaluation methods, seen as an ‘expert’ by program • Outsiders may not understand the program or people involved • May cause anxiety if not known and trusted

15. The best of both: Participatory Evaluation • Participation by a range of stakeholders • Focus on program participants, not donors • Focus is on learning, not just accountability • Flexible design, not predetermined • Use rapid appraisal methods, less formal • Consultants are facilitators, not the actual ‘evaluator’ • Evaluation team is key to approach

16. What does an evaluation team do? Preparation: • Establish purpose, methods (‘scope of work’) including the evaluation questions • Develop data gathering instruments • Develop team plan and itinerary Main activities: • Visit main offices, field sites • Gather data (document review; observations, group discussions, interviews; review surveys)

17. Report preparation: • Team discussion of findings, come to tentative conclusions and recommendations • Draft main findings of the evaluation report • Discuss draft of findings with program staff, other stakeholders • Revise findings, recommendations as needed • Finalize and submit report • Share with staff, community, MOH, others

18. When in doubt, observe and ask questions.When certain, observe at length and ask many more questions! From Halcolm’s Evaluation Laws

19. Monitoring exercise: Choose one objective for your group project For that objective, select one important activity that the project will monitor Discuss and fill in the information for the six exercise questions