1 / 45

Monitoring and Evaluation in Development

Monitoring and Evaluation in Development. Know Before You Go EWB-UK 2014 Thalia Konaris. Think of a project:. Identify: The aim of the project Geographical context The activities The donor The client / organisation The beneficiary. What is M&E?.

tekla
Download Presentation

Monitoring and Evaluation in Development

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Monitoring and Evaluation in Development Know Before You Go EWB-UK 2014 Thalia Konaris

  2. Think of a project: • Identify: • The aim of the project • Geographical context • The activities • The donor • The client / organisation • The beneficiary

  3. What is M&E? • Systems that create a feedback loop in your project which tells you about its progress, risk, efficiency, effectiveness, impact Quality * Learning*Accountability*

  4. What is M&E? • Without it: • Successful methods lack visibility • Mistakes repeated • Potentially harmful or ineffective projects receive further funding to continue • Donors & communities increasingly frustrated, more laborious reporting requirements

  5. What is M&E?.. expensive! • Who needs to know? • What do they need to know? • How frequently? • In what format?

  6. Monitoring - ‘How are we doing?’ 2 3 4 5 1 Middle Project Start Project End/ Milestone Evaluation – ‘Are we doing the right thing?’

  7. Monitoring • An Instrument of internal management • Functions: • Progress tracking • Quality • Risk management • Financial management • Tools: • Work Plan • MIS /FMIS /GIS • Risk Management Matrix • Logical Framework Matrix

  8. Evaluation • Many different types, depending on the questions • Measures Efficiency, Effectiveness, Relevance, Sustainability, Impact (5 OECD DAC criteria) • Can involve use of qualitative, quantitative or mixed-methods • Expensive, labour intensive • By nature, independent

  9. Evaluation • Sometimes it can be controversial…

  10. The bigger M&E picture.. • Development sector is under increasing pressure to prove development effectiveness • International Aid Effectiveness High Level Fora (Rome 2003, Paris 2005, Accra 2008, Busan 2011)

  11. Logical Framework Approach

  12. Underlying theory.. ACTIVITIES e.g. Running WatSan training courses Change OUTPUTS e.g. 3 courses conducted in three villages OUTCOMES e.g. 30 entrepreneurs with good WatSan technical knowledge IMPACT e.g. 3 communities with improved WatSan and quality of life

  13. Logical Framework Matrix Change Activity Log

  14. Understanding your project: • Participatory Market Mapping • Project Logic Model / Theory of Change

  15. Project Example: Donor: World Bank – IFC Client: Government of Nepal Project: Lighting Asia Aim: Build private sector capacity in Nepal for affordable, off-grid, solar-powered lighting solutions Source: HEDON, 2010. Lighting Africa: Innovative marketing and business models for the rapid development of off-grid lighting markets in Africa. Boiling Point 58: Marketing

  16. Participatory Market Mapping

  17. Tool: Participatory Market Mapping (Developed by Practical Action, 2005) Enabling Environment Market Actors Supporting Services Source: IFC, 2012. Lighting Asia: Solar Off-grid Lighting.

  18. Tool: Participatory Market Mapping (Developed by Practical Action, 2005) Enabling Environment Company branches Foreign product suppliers $ Off-grid consumer Solar product SMEs Market Actors $ Dealers/ Agents No local manufacturing capacity Supporting Services Source: IFC, 2012. Lighting Asia: Solar Off-grid Lighting.

  19. Tool: Participatory Market Mapping (Developed by Practical Action, 2005) Enabling Environment Company branches Foreign product suppliers $ Off-grid consumer Solar product SMEs Market Actors $ Dealers/Agents No local manufacturing capacity UNDP Small Grants programme Business Support Centres Supporting Services Technicians Commercial Banks Source: IFC, 2012. Lighting Asia: Solar Off-grid Lighting.

  20. Little awareness of solar products Dominance & cost of Kerosene 3 million off-grid homes Enabling Environment Government subsidies for solar Poor rural infrastructure Company branches Foreign product suppliers $ Off-grid consumer Solar product SMEs Market Actors $ Dealers/Agents No local manufacturing capacity UNDP Small Grants programme Business Support Centres Supporting Services Technicians Commercial Banks Source: IFC, 2012. Lighting Asia: Solar Off-grid Lighting.

  21. Project Logic (aka. Results Chain/ Theory of Change)

  22. Developing the Theory of Change.. • Step 1: Write at the top of your page, the project’s vision • Hint: Write as if it has already happened.. “If my project is successful, there is…”

  23. Step 2: Underneath your vision, create statements of what is needed, in order to bring about the vision For this to happen, there needs to be…

  24. For this to happen, there needs to be… • Step 3: Repeat the process… For this to happen, there needs to be…

  25. Impact Outcomes Activities & Outputs

  26. Exercise 1: • Identify a project for which to develop a Theory of Change • Starting at the top, write down your Impact Statement • Broaden your mind – don’t be restricted by the activities you know your project is organising, and try and build a comprehensive TOC • Continue down the chain and write as ‘outcome statements’ conditions that are needed to satisfy the Impact statement • Go as far down as you need to, to start talking about activities • Hint: all your statements should be written ‘as if they’ve already happened’

  27. Assumptions • Indicators • Means of Verification

  28. 1 4 3 2 Exercise 2: Create your Log Frame Matrix & identify your Assumptions

  29. Assumptions ‘Elements influencing the success of your project, that are not under your control’

  30. Goal Assumptions Outcomes Activities & Outputs

  31. Exercise 3: • In your Logframe Matrix write down potential output, outcome and impact indicators that could be used to prove that these have been achieved • In your ‘Means of Verification’ column, write down potential sources of data to inform your indicators

  32. Indicators S.M.A.R.T • Specific: qualify exactly what should be achieved, where and by whom; • Measurable: quantify the achievement (where possible); • Attributable: strongly link the achievement to the expected performance; • Realistic: can be reliably, cost-effectively and timely collected; • Time-bound: state when the achievement must be reached • Qualitative or Quantitative

  33. Quality – Quantity – Target group – Place -(Time ) • Examples.. • Output indicators: • No. of training courses carried out • No. of course participants • Outcome indicators: • No. of enterprises developed as a result of the training • (Increase in) eco-san construction skills of participants • Impact indicators: • Proportion of population of village X with access to sanitation • Proportion of population of village X with (improved) hygiene and health

  34. Means of Verification ‘Sources of information that will inform your indicators, including who will collect and how frequently’ • Examples: • Technical surveys & studies • Participant questionnaires • Competency tests • Field visits by trained staff • Meeting minutes

  35. Is your LogFrame complete? • Add milestones and targets • Carry out a baseline assessment of indicator values • Prepare an Activities Schedule/Work Plan • Use your assumptions to develop a Risk Management Matrix • Determine your reporting schedule (who needs to know what, in what format, how often) • Determine your data collection schedule

  36. Alternative Approaches to M&E

  37. Outcome Mapping • There is no use looking beyond the Outcome level • All change, is behaviour change • Tracks the behaviour change of boundary partners • Expect to see, Like to see, Love to see • Useful for M&E of capacity building

  38. Most Significant Change Stories • Impact is gauged through collection of stories of MSC in people’s lives • ‘Looking back over the last month, what do you think was the most significant change that happened in [a certain domain of change]?’

  39. I bet we’ve run out of time.. • Thank you  • Any questions? Contact me: thalia.konaris@gmail.com

  40. Additional resources -I • Outcome Mapping: • Outcome Mapping Learning Community http://www.outcomemapping.ca/ Includes OM Manual, OM case studies, discussion forums • Logical Framework Approach: • http://ec.europa.eu/europeaid/multimedia/publications/publications/manuals-tools/t101_en.htm • Theory of Change: • ActKnowledge (2003). Theory of Change - Guided example: Project Superwomen. http://www.theoryofchange.org/pdf/Superwomen_Example.pdf • TOCO: Theory Of Change Online. Online, open source software for constructing a Theory of Change Diagram. http://www.theoryofchange.org/toco-software/ • Outcome Harvesting: • Wilson-Grau R., Grit H. (2012). Outcome Harvesting. Ford Foundationhttp://www.outcomemapping.ca/resource/resource.php?id=374 • Presentation summarising the Outcome Harvesting methodologyhttp://www.googlesyndicatedsearch.com/u/OM?q=outcome%20harvesting&sa=Google+Search • Most Significant Change: • Davies R., Dart J. (2005). The ‘Most Significant Change’ Technique: a Guide to its Use’.http://www.mande.co.uk/docs/MSCGuide.pdf

  41. Additional resources - II • Excellent introduction to Evaluation: [book] • Bamberger J. M. et al. (2012). RealWorldEvaluation: Working Under Budget, Time, Data, and Political Constraints. • Results-based Monitoring & Evaluation: • Kusek, J. Z., & Rist, R. C. (2004). Ten Steps to a Results-based Monitoring & Evaluation System. World Bank https://openknowledge.worldbank.org/bitstream/handle/10986/14926/296720PAPER0100steps.pdf?sequence=1 • The Donor Committee for Enterprise Development (DCED) Standard: • DCED, (2010). The DCED Standard for Measuring Results – Walk Through. [Brief Overview] http://www.enterprise-development.org/page/implementing-standard • Data collection methods: • Bamberger J. M. et al. (2012). RealWorld Evaluation: Working Under Budget, Time, Data, and Political Constraints. [Book] • Kumar S. (2002). Methods for Community Participation: A complete guide for practitioners [Book]. ITDG Publishing • M&E Glossary: • OECD Glossary of key M&E terms http://www.oecd.org/development/peer-reviews/2754804.pdf

  42. Albu and Griffith, 2005; Griffith and Osorio, 2008

More Related