Monitoring and evaluation
1 / 32

Monitoring and evaluation - PowerPoint PPT Presentation

  • Uploaded on
  • Presentation posted in: General

Monitoring and evaluation. Our Approach. What is monitoring and evaluation? Conceptual differences and terminologies. Approaches to Monitoring and Evaluation. Establishing M&E System. How to do Monitoring and Evaluation. What is monitoring?.

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.

Download Presentation

Monitoring and evaluation

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript

Monitoring and evaluation

Monitoring and evaluation

Our approach

Our Approach

  • What is monitoring and evaluation? Conceptual differences and terminologies.

  • Approaches to Monitoring and Evaluation.

  • Establishing M&E System.

  • How to do Monitoring and Evaluation.

What is monitoring

What is monitoring?

  • Day-to-day follow up of activities during implementation to measure progress and identify deviations

  • Routine follow up to ensure activities are proceeding as planned and are on schedule

  • Routine assessment of activities and results

  • Answers the question, “what are we doing?”

Why to monitor activities

Why to monitor activities?

  • Tracks inputs and outputs and compares them to plan

  • Identifies and addresses problems

  • Ensures effective use of resources

  • Ensures quality and learning to improve activities and services

  • Strengthens accountability

  • Program management tool

What is evaluation

What is evaluation?

  • It is a time-bound exercise that attempts to assess systematically and objectively the relevance, performance and success of ongoing and completed programmes and projects.

  • Designed specifically with intention to attribute changes to intervention itself

  • Answers the question, “what have we achieved and what impact have we made”

  • Evaluation commonly aims to determine the relevance, efficiency, effectiveness, impact and sustainability of a programme or project.

Monitoring and evaluation

  • Relevance: The degree to which the outputs, outcomes or goals of a programme remain valid and pertinent as originally planned or as subsequently modified owing to changing circumstances within the immediate context and external environment of that programme.

  • Efficiency: A measure of how economically or optimally inputs (financial, human, technical and material resources) are used to produce outputs.

  • Effectiveness: A measure of the extent to which a programme achieves its planned results (outputs, outcomes and goals).

Monitoring and evaluation

  • Impact: Positive and negative long term effects on identifiable population groups produced by a development intervention, directly or indirectly, intended or unintended. These effects can be economic, socio-cultural, institutional, environmental, technological or of other types.

  • Sustainability: Durability of programme results after the termination of the technical cooperation channelled through the programme.

  • Static sustainability – the continuous flow of the same benefits, set in motion by the completed programme, to the same target groups;

  • Dynamic sustainability – the use or adaptation of programme results to a different context or changing environment by the original target groups and/or other groups.

Why evaluate activities

Why evaluate activities

  • Determines program effectiveness

  • Shows impact

  • Strengthens financial responsibility and accountability

  • Promotes a learning culture focused on service improvement

  • Promotes replication of successful interventions.

Types of evaluation

Types of Evaluation

  • Ex-ante Evaluation: An evaluation that is performed before implementation of a development. intervention. Related term: appraisal.

  • Ex-post Evaluation: A type of summative evaluation of an intervention usually conducted after it has been completed.

  • External Evaluation: An evaluation conducted by individuals or entities free of control by those responsible for the design and implementation of the development intervention to be evaluated (synonym: independent evaluation).

  • Internal Evaluation: Evaluation of a development intervention conducted by a unit and /or individual/s reporting to the donor, partner, or implementing organization for the intervention.

Monitoring and evaluation

  • Formative Evaluation: A type of process evaluation undertaken during programme implementation to furnish information that will guide programme improvement.

  • Impact Evaluation: A type of outcome evaluation that focuses on the broad, longer-term impact or results of a programme.

  • Joint Evaluation: An evaluation conducted with other partners, bilateral donors or international development banks.

  • Meta-evaluation: A type of evaluation that aggregates findings from a series of evaluations.

Monitoring and evaluation

  • Process Evaluation: A type of evaluation that examines the extent to which a programme is operating as intended by assessing ongoing programme operations. A process evaluation helps programme managers identify what changes are needed in design, strategies and operations to improve performance.

  • Qualitative Evaluation: A type of evaluation that is primarily descriptive and interpretative, and may or may not lend itself to quantification.

  • Quantitative Evaluation: A type of evaluation involving the use of numerical measurement and data analysis based on statistical methods.

Monitoring and evaluation

  • Summative Evaluation: A type of outcome and impact evaluation that assesses the overall effectiveness of a programme.

  • Thematic Evaluation: Evaluation of selected aspects or cross-cutting issues in different types of interventions.

Confusing terms

Confusing terms

  • Audit

  • Appraisal

  • Inspection

M e tools

M&E Tools

  • Evaluating programme strategy and direction:Log-frames, Stakeholder Analysis

  • Evaluating programme management:Horizontal Evaluation; Appreciative Inquiry

  • Evaluating programme outputs: Evaluating academic articles and research reports; Evaluating websites; After Action Reviews

  • Evaluating outcomes and impacts:Outcome Mapping, Most Significant Change; Episode Studies.

M e tools1

M&E Tools

Following is a non – exhaustive list of M&E Tools:

  • Performance indicators

  • Formal surveys

  • Rapid appraisal methods

  • Participatory methods

  • Cost-benefit and cost-effectiveness analysis

  • Impact evaluation

1 performance indicators

1. Performance indicators

  • Performance indicators are measures of inputs, processes, outputs, outcomes, and impacts for development projects, programs, or strategies.

  • Uses:

    • Setting performance targets and assessing progress toward achieving them.

    • Identifying problems via an early warning system to allow corrective action to be taken.

  • Problems:

    • Poorly defined indicators are not good measures of success.

    • Tendency to define too many indicators, or those without accessible data sources,

    • Often a trade-off between picking the optimal or desired indicators and having to accept the indicators which can be measured using existing data.

How to make indicators

How To Make Indicators

  • Identify the problem situation you are trying to address.

  • Develop a vision for how you would like the problem areas to be/look. This will give you impact indicators.

  • Develop a process vision for how you want things to be achieved. This will give you process indicators.

  • Develop indicators for effectiveness.

  • Develop indicators for efficiency .

2 formal surveys

2- Formal Surveys

  • Formal surveys can be used to collect standardized information from a carefully selected sample of people or households.

  • Uses:

    • Providing baseline data against which the performance of the strategy, program, or project can be compared.

    • Comparing different groups at a given point in time.

    • Comparing changes over time in the same group.

    • Comparing actual conditions with the targets established in a program or project design.

2 formal surveys1

2- Formal Surveys

  • Problems:

    • With the exception of CWIQ, results are often not available for a long period of time.

    • The processing and analysis of data can be a major bottleneck for the larger surveys even where computers are available.

    • LSMS & household surveys are expensive & time-consuming.

    • Many kinds of information are difficult to obtain through formal interviews.

Different types of survey

Different types of survey

  • Multi-Topic Household Survey (also known as Living Standards Measurement Survey—LSMS).

  • Core Welfare Indicators Questionnaire (CWIQ).

  • Client Satisfaction (or Service Delivery) Survey.

  • Citizen Report Card.

3 rapid appraisal methods

3- Rapid appraisal methods

  • Rapid appraisal methods are quick, low-cost ways to gather the views and feedback of beneficiaries and other stakeholders, in order to respond to decision-makers’ needs for information.

  • Uses:

    • Providing rapid information for management decision-making, especially at the project or program level.

    • Providing qualitative understanding of complex socioeconomic changes, highly interactive social situations, or people’s values, motivations, and reactions.

    • Providing context and interpretation for quantitative data collected by more formal methods.

3 rapid appraisal methods1

3- Rapid appraisal methods

  • Problems:

    • Findings usually relate to specific communities or localities—thus difficult to generalize from findings.

    • Less valid, reliable, and credible than formal surveys.

4 rapid appraisal methods

4- Rapid appraisal methods

  • Key informant interview

  • Community group interview

  • Focus group discussion

  • Direct Observation

  • Mini surveys

4 participatory methods

4- Participatory methods

  • Participatory methods provide active involvement in decision-making for those with a stake in a project, program, or strategy and generate a sense of ownership in the M&E results and recommendations.

  • Uses:

    • Learning about local conditions and local people’s perspectives and priorities to design more responsive and sustainable interventions.

    • Evaluating a project, program, or policy.

    • Providing knowledge and skills to empower poor people.

4 participatory methods1

4- Participatory methods

  • Problems:

    • Sometimes regarded as less objective.

    • Time-consuming if key stakeholders are involved in a meaningful way.

    • Potential for domination and misuse by some stakeholders to further their own interests.

4 participatory methods2

4- Participatory methods

  • Participatory rural appraisal

  • Participatory monitoring and evaluation

5 cost benefit cost effectiveness analysis

5- Cost-benefit & cost-effectiveness analysis

  • Cost-benefit and cost-effectiveness analysis are tools for assessing whether or not the costs of an activity can be justified by the outcomes and impacts. Cost-benefit analysis measures both inputs and outputs in monetary terms. Cost-effectiveness analysis estimates inputs in monetary terms and outcomes in non-monetary quantitative terms.

  • Uses:

    • Informing decisions about the most efficient allocation of resources.

    • Identifying projects that offer the highest rate of return on investment.

5 cost benefit cost effectiveness analysis1

5- Cost-benefit & cost-effectiveness analysis

  • Problems:

    • Fairly technical, requiring adequate financial and human resources available.

    • Requisite data for cost-benefit calculations may not be available, and projected results may be highly dependent on assumptions made.

    • Results must be interpreted with care, particularly in projects where benefits are difficult to quantify.

6 impact evaluation

6 – Impact Evaluation

  • Impact evaluation is the systematic identification of the effects – positive or negative, intended or not – on individual households, institutions, and the environment caused by a given development activity such as a program or project.

  • Uses:

    • Measuring outcomes and impacts of an activity and distinguishing these from the influence of other, external factors.

    • Helping to clarify whether costs for an activity are justified.

    • Informing decisions on whether to expand, modify or eliminate projects, programs or policies.

Monitoring and evaluation

  • Problems:

  • Some approaches are very expensive and time-consuming

  • Reduced utility when decision-makers need information quickly.

  • Difficulties in identifying an appropriate counter-factual.

  • Login