Monitoring, evaluation, and learning (MEL) framework for the Millennium Water Program, Kenya

1. Monitoring, evaluation, and learning (MEL) framework for the Millennium Water Program, Kenya May 8, 2012 MWP-K Learning Event

2. Background • Collaboration between: • Millennium Water Alliance • MWP-Kenya partners (CARE, CRS, FH, Living Water, Lifewater, World Vision) • Center for Global Safe Water, Emory University • Objectives: • Improve effectiveness and increase sustainability through more data-driven decision making • Build foundation of learning and collaboration through MEL to support future continuation of MWP-Kenya

3. Challenges Addressed by MEL Framework • Monitoring usually limited to project activities and outputs (e.g. # latrines built, # trainings completed) • Final evaluation is only time seized to assess achievement of outcomes (e.g. behavior change) Want to use monitoring to track progress toward intended behavior change outcomes to enable a change of course if needed

4. Challenges Addressed by MEL Framework • Minimal information sharing & reflection on monitoring data or other lessons throughout project life cycle. Want to intentionally budget time and persons responsible for guiding data sharing and reflection to maximize lessons learned to improve practice

5. Challenges Addressed by MEL Framework • Sustainability seldom evaluated after project life cycle due to close of project, budget limitations, etc. Want to incorporate M&E indicators into tools used during project life cycle to assess likelihood of future sustainability (are factors in place to improve longevity of benefits after project is finished?)

6. Components of MEL for MWP Kenya • Key program indicators • Evaluation: baseline and follow-up • Monitoring: systematic, regular data collection • Includes process and tools to guide “Data for decision-making”: Sharing, feedback, and action; • Monitoring tools to assess progress towards outcomes • Focused topical learning • Includes partners’ leadership in special learning topics; • Emory support for some sub-studies (e.g. sanitation assessment)

7. 1. Key Program Indicators • Emory assisted partners in developing logical framework in May 2010 • Indicators include: • Outputs (activities completed by partners) • Outcomes = central focus (changes that beneficiaries adopt in response to program) • Includes indicators that suggest future sustainability • E.g. School has specific WASH budget; school management committee is involved in WASH program

8. 2. Evaluation • Emory facilitated baseline assessment planning and implementation: • Led development of evaluation tools and protocols • Household, school, and health facility surveys • Analyzed data • Each organization collected data prior to implementation

9. 3. Monitoring • Output-level indicators • Responsibility of each partner • Tracked through internal monitoring system • Outcome-level indicators • Measured throughout course of project • To engage implementers to critically reflect on and address programmatic challenges in real-time • Monitoring activities conducted quarterly or once per school term

10. Data for Decision-Making • Sharing of experiences and lessons learned between partners • Data Reflection Tool (DaRT) • Summarizes findings from each monitoring and learning activity • Project expectations • Progress toward achieving indicators • Plans to address lack of progress • Potential threats to sustainability • Shared and discussed within partnership on a quarterly basis before Program Management Group meeting

11. Data for Decision-Making Quarterly Work Flow = Responsibility of each organization = Responsibility of Coordinator = Responsibility of all partners together Decide: Needed course changes Additional learning needs Meeting: Internal Learning/ Reflection Meeting: Pre-PMG Learning/ Reflection Meeting: PMG Complete monitoring activities quarterly Summarize findings into Data Reflection Tool (DART) MWP Coordinator reviews DARTs Send Discussion of: What’s going well/ not as well as expected Possible course changes Additional info needs Flag issues for follow-up Use to guide agenda for Pre-PMG meeting Distribute copies at pre-PMG meeting Discussion of: Highlights from DARTs Lessons learned, best practices Group feedback Issues to raise at PMG Discussion of: Issues flagged at Pre-PMG meeting Lessons learned & best practices

12. Summary of Monitoring Tools

13. 4. Focused Topical Learning • Partners expressed interest in specific topics for further investigation • As learning leader, each org. should organize one meeting with partners to discuss lessons on topic

14. Summary • Thoughtful, timely collection of relevant data • Data used to drive decision-making: • Guide targeting, technology choice, and program approaches to improve effectiveness and sustainability • Feed back findings into continued program activities in a timely and meaningful way • Engage donors, beneficiaries, and government stakeholders in the water, sanitation, and hygiene (WASH) sector • Embrace benefits of working in partnership • Collective learning and reflection on best practices & lessons learned