MONITORING AND EVALUATION TOOLS Monitoring and Evaluation

1. MONITORING AND EVALUATION TOOLSMonitoring and Evaluation The National Democratic Institute

2. Introductions/Ground rules • Introductions • Ground Rules • Ice Breaker Exercise Photo: Sanja Gjenero, RGBstock.com

3. Monitoring and Evaluation (M&E) Tools Objectives • To become familiar with and practice using a range of M&E tools • To consider when and how the tools can be employed during the project life cycle

4. M&E Tools Topics • What is M&E? • Gender analysis • Project lifecycle • Data collection • Program design tools • Evaluations • Other tools Image: www.pixabay.com

5. Key Terms • Goal • Target • Baseline • Impact • Gender • Stakeholder • Indicator • Input • Output • Intermediate result • Outcome • Objective

6. M&E Defined • Monitoring • Continuous • Ensures project on track • Identifies problems • Evaluation • Assesses whether project is achieving objectives • Periodic • Focuses on outcomes and impacts

7. Gender and M&E • Gendered priorities, constraints and impact • Failure to address gender leads to inefficient and unsustainable results and exacerbates inequities

8. Project life cycle

9. Data collection Tools • Quantitative methods • Household surveys • Public opinion polls/surveys • Qualitative methods • In-depth interviews • Focus groups • Direct observation • Document review • Participatory assessments

10. Exercise: Which tools? When? • Multi-workshop training program for women political activists • Objective: Improve the capacity of women to run for elected office • Which data collection tools? When? Photo: Amy Hamelin, NDI

11. Example: Which tools? When? • Workshops • Pre-and post questionnaires • Evaluation forms • In between events • Direct observation • End of project • Focus groups • In-depth interviews • Election results Photo: Amy Hamelin, NDI

12. Program design tools • Gender, context and stakeholder analysis • Problem and objective analysis • Activity plan • Indicators • Critical assumptions • Risk analysis and management • Resource plan • Results framework Image: www.pixabay.com

13. Gender analysis • Can be undertaken at any stage but most effective if included in design • Systematic way of analyzing different roles and impacts • Asks the “who” questions • How this will affect women and men?

14. Context analysis • Institutions or vehicles that have positive or negative impact? • Key actors or factors that help or hinder? • Socio-cultural, political and economic environment?

15. Context analysis Example

16. stakeholder analysis • All people who have a stake • Participants • Implementers • Decision-makers • Donors • Who are they? • What is their interest? Influence? Power? • Relations with other stakeholders

17. stakeholder analysis High • Survey all people who have a stake INFLUENCE Low High Low POWER

18. Exercise: stakeholder analysis • Program to enhance the capacity of the Liberian legislature to represent citizen’s interests • Identify stakeholders • Plot them on chart Photo: NDI

19. problem analysis

20. Exercise: problem Tree • Problem: Low level of women in elected bodies • Identify root causes • Identify effects • Link root causes to their effects Photo: www.pixabay.com

21. objective analysis • Links problem analysis to program design • Provides visual map • Defines scope of program • Maps out pathway of change • Identifies what will be monitored and evaluated • Facilitates planning and management

22. objective analysis • Core problem reframed as objective • Root causes reframed as intermediate results

23. Indicators • Qualitative versus quantitative • SMART • Specific • Measurable • Achievable • Relevant • Time-bound • Negative effects reframed become indicators Photo: www.pixabay.com

24. Critical assumptions • Factors outside our control • Conditions under which program logic will hold true • Conditions likely to affect achievement of results • Identify at each level of framework

25. Risk analysis and management • Every program entails risk • Identify risks and contingency plans/mitigation measures for each level of framework Image: www.pixabay.com

26. RESOURCE PLAN • What resources are needed for: • Staff time and materials for setting up an M&E system • Evaluation activities

27. Results framework • Links evaluation strategy to problem analysis • Helps identify indicators • Aids in M&E • Clarifies logic of project • Provides roadmap for planning and management

28. Example: framework

29. Exercise: Results framework

30. Program Evaluation • Types of evaluations: • Baseline • Mid-term • Final • Impact • Internal versus external

31. Example: assessments Win with Women Political Party Assessment Tool

32. Program Evaluation • Steps • Determine evaluability • Select research questions • Identify methods for data collection • Develop and test data collection instruments and protocols Photo: Sanja Gjenero, RGBstock.com

33. Event Evaluations • Verbal feedback • Evaluation forms • Pre- and post-workshop questionnaires

34. Public opinion research • Qualitative • Focus groups/in-depth interviews • Quantitative • Surveys/polls • Combination of both

35. In-depth Interviews • Key informant/one-on-one interviews • Explore thoughts on deeper level • More flexible • Avoid influence of group dynamics Photo: NDI

36. Focus groups • Why and how strongly people hold opinions • Can’t project results to wider population • Groups formed around common characteristics • Six to ten people and moderator • Open-ended questions

37. Surveys • Identical set of close-ended questions • Representative sample of population • Results projected to larger population • Predict future behaviors and trends

38. Example: Public opinion research in Ukraine • Nearly 50% thought women underrepresented • More likely to vote for party with outreach to women • Perceive women as better managers Photo: NDI

39. M&E TOOLS REVIEW • Monitoring: continuous activity to ensure project is on track • Evaluation: assesses impact • M&E must address gender • Wide range of qualitative and quantitative data collection tools that can be used throughout project life cycle