Introduction to Monitoring and Evaluation

1. Introduction to Monitoring and Evaluation

2. Objectives of session • Definitions: Monitoring and Evaluation • Rationale for M&E • Review elements of an M&E plan • Steps toward constructing the M&E plan • Implementing the M&E plan

3. Monitoring is the routine process of data collection and measurement of progress toward program objectives. Evaluation is the use of specific study designs and special studies to measure the extent to which changes in desired health outcomes are attributable to a program’s interventions. Monitoring - Evaluation Interventions Outcome

4. Rationale for M&E • M&E help to make informed decisions regarding ongoing programs • They facilitate effective and efficient use of resources • They determine whether a program is right on track and where changes need to be considered • M&E help stakeholders conclude whether the program is a success • M&E preserve institutional memory

5. Three Primary Uses of Evaluation Findings • Rendering judgments • Summative evaluations of program’s overall effectiveness e.g., audit, renewal, quality control, accreditation • Facilitating improvements • Formative evaluation to improve program e.g., program’s strengths/weaknesses, progress • Generating knowledge • Conceptual use of findings e.g., generalization, theory building

6. What is an M&E Plan? • An M&E Plan is a document that describes a system which links strategic information obtained from various data collection systems to decisions that will improve health programs. • Fundamental document to ensure: • Accountability • Measure of success

7. Functions of an M&E Plan • State how the program is going to measure what it has achieved (ensure accountability) • Document consensus (encourage transparency and responsibility) • Guide M&E implementation (standardization and coordination) • Preserve institutional memory M&E plan is a living document and needs to be adjusted when a program is modified.

8. Functions of an M&E Plan: by level of the health system • Facility (primary data collection): • Manage patients/clients • Supervise providers • Manage logistics • Report to district HIS and vertical programs and community • District: • Allocate resources • Report to provincial/national • National (Comprehensive vs. Vertical Programs) • Allocate resources • Collect national data • Report to international donors and other stakeholders

9. Elements of M&E Plan • Brief project description • Purpose(s) of M&E plan • Brief history of M&E plan development • Evaluation framework • Indicator system • Information system (data sources) • Impact evaluation design • Dissemination and utilization plan • Adjustments to M&E plan

10. 1. Brief project description Include: • objectives/results • implementation responsibility • dates/duration • geographic focus • resources

11. 2. Purpose(s) of M&E plan Specify purpose for: • performance (process) monitoring • impact evaluation

12. 3. Brief history of M&E plan development Helps to define: • shared vision (internal/external) • source and extent of interest/commitment

13. 4. Develop Evaluation Framework(Log Frame, Results, Conceptual) Show: • Primary components • Levels and units of action • Causal sequence • Component-related indicator(s)

14. 5. Develop Indicator system • Organized by framework component • Attributes • Name of indicator • Description/definition • Unit of measurement • Data source (primary/secondary) • Baseline/target value(s) by year(s) • Frequency of data collection • Responsibility • Reporting plan and frequency

15. 6. Information Systems(Data sources) Develop: • Information systems for collecting data for indicators • Diagram data collection, processing, analysis, and reporting system • Data collection tools (routine vs. non-routine) • patient records • registers • survey instruments

16. 7. Select impact evaluation design Decide on design: • Experimental vs. observational • Quasi-experimental (post-test, pre- and post-test with no control) • Implementation plan • Execution responsibility (internal/external)

17. 8. Dissemination and utilization plan Decide on: • Databases for information storage • Clearly defined users • Schedule and timeliness • Dissemination methods • Reports (schedule and audience) • Media • Speaking events • Information-utilization models

18. 9. Adjustments to M&E plan • Program changes can occur, affecting the original M&E plan for both performance monitoring and impact evaluation M&E plan is a living document and needs to be adjusted when a program is modified.

19. Steps toward constructing the M&E plan • Preparing to build the plan • Constructing the evaluation framework • Identifying potential data sources • Selecting and defining indicators and their values • Designing the impact evaluation • Developing a reporting schedule and system • Planning for mid-course adjustments

20. In implementing the M&E plan • Utility - serve practical information needs of intended users • Feasibility - be realistic, prudent, diplomatic and economical • Propriety - conducted legally, ethically, and with regard to those involved in and affected by the evaluation • Accuracy - reveal and convey technically accurate information