1 / 30

Evaluation Planning and Management John Wooten ~ Lynn Keeys

WELCOME. Evaluation Planning and Management John Wooten ~ Lynn Keeys. June 2012. Session 9: Evaluation Follow-up. Evaluation Road Map : Stage 6. Objective: Review some priority evaluation follow-up requirements. Evaluation Follow-up. EPM on Easy Street?. Not just yet…

starr
Download Presentation

Evaluation Planning and Management John Wooten ~ Lynn Keeys

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. WELCOME Evaluation Planning and Management John Wooten ~ Lynn Keeys June 2012

  2. Session 9: Evaluation Follow-up Evaluation Road Map: Stage 6 Objective: • Review some priority evaluation follow-up requirements

  3. Evaluation Follow-up EPM on Easy Street? Not just yet… “Strike while the evaluation trail is hot~”

  4. Evaluation Follow-up Activate Evaluation Follow-up Plan… Disseminate results Prioritize recommendations Negotiate, formalize decision/implementation assignments Agree on administration framework/approach Identify/share key lessons Assess the evaluation report/experience Identify immediate project actions Store evaluation data * Develop/distribute success stories, vignettes Organize formal/informal appreciations * Peruse selective bibliography content…

  5. Disseminate Results Activate Evaluation Follow-up Plan… • IPs and sub-IPs(1) • DEC/Washington(1) • All or key stakeholders (1) • Select beneficiary reps (2) • Select donors in sector (3) • Select NGOs (3) • Select private sector reps (2) • Select replication reps (1) • Mission website (4) • Project website (4) • Select missions in region (1) • Official files (1,2) • Communications officers (1) • US embassy counterpart (2) • Washington reps (1) • Key: 1 = Full report • 2 = Summary or excerpts • 3 = Summary noting full report available from Mission • 4 = Summary noting full report available from DEC

  6. Prioritize Recommendations • Organize by: • Decision-making urgency • Likely costs • Potential stakeholder impacts • Potential beneficiary impacts • Mission internal/external focus • Political/cultural sensitivity

  7. Negotiate Assignments • Assign by: NB: Ensure assignees have authority… • Agency • Section • Team • Person (w/supervisory ok)

  8. Select Monitoring Framework • Monitor by: • Implementation schedule • Benchmarks • Means (staff/special meetings, memo/e-mail; frequency…)

  9. Identify/Share Key Lessons • Share with: • Other donors with tangential (non-direct) interests • Potentially replication/extension groups • Local NGOs, NGO associations • Colleges/universities • Community-based organizations • Donor group meetings (plenary, sector) • Journalists (newsprint exposés) • Scholarly journals • USAID/Washington knowledge management systems Requires e-x-p-a-n-s-i-v-e thinking Get help~

  10. Assess the Evaluation • Assess the: • Share selectively, with sensitivity and tact • Team, team members • Report • Overall experience • Contractor (A&A office assessment form) • Participatory approach experience • Evaluation team augmentation experience • Who needs to know; who would benefit?

  11. Id Immediate Project Actions • Explore implications for… • NB: USAID auditors study evaluation reports~ • On-going and future project implementation (what) needs to and can change immediately?) • Reporting (reports content and scope) • Management and performance indicators, baselines, targets, interim targets (design structure) • Audit requirements (performance, financial, internal, external, concurrent)

  12. Store Evaluation Data • Quantitative Data… NB: Washington guidance pending • Check for completeness • Back-up • Audit-proof • Upload to new central evaluation data warehouse

  13. Develop Success Stories/Vignettes • Be creative… Format content for: • Local ~ Regional • US ~ Global • TV ~ Radio • Newsprint • Newsletters • Websites (Mission/Project) • Performance reports Use in: • Social events • Project ceremonies • Embassy events • Host country events • Community events

  14. Express Appreciations • Make your list...check it thrice • ! • ~Recognize the value ~o~ Be culturally sensitive~ Evaluation planning/management team Evaluation team IP reps Host country reps Other donor / project reps Logistic facilitators USAID/Washington peer contributors Secretaries/administrative assistants Schedulers

  15. Peruse the Bibliography • You, a continuing learner…licensed to learn Schedule time for continued learning Experts you hired conducted research Learn from what they learned NB: USAID is a ‘learning organization’

  16. Implications for Performance Monitoring? Ponder: Now that you have a better understanding of planning and managing an evaluation, including using evaluation information… …how can this insight help you better perform your regular monitoring responsibilities?

  17. Implications for Performance Monitoring? Consider: New strategy, evaluation and project design guidance targets program development and evaluation elements of the program cycle Comprehensive M&E contracts outsource in-house monitoring responsibilities, potentially leaving staff less interested in/less capable of performing basic monitoring functions Project officers are often too busy to perform quality monitoring, read performance reports and reflect on performance data until portfolio reviews Monitoring is often tightly focused on PMP performance indicators

  18. Implications for Performance Monitoring? Possible Results: Site visits may become “funded escapes” from heavy office work loads and schedules Postponing all “Why?” questions until evaluation time may trigger important and expensive missed opportunities More evaluations may be tasked to “Tell me what I missed as a result of weak, ineffectual monitoring”

  19. Implications for Performance Monitoring? RegularMonitoring “Evaluative Monitoring” Possible Response: Increase the “evaluative content” of monitoring

  20. Implications for Performance Monitoring? Potential Benefits: Officers empowered/enabled to pose some “Why?” questions using locally available/affordable tools to help get timely answers Officers prompted to take more time to read and think reflectively on performance data Less potential need for expensive evaluations Evaluation resources focused on bigger, higher value “WHY?”questions Inherently governmental functions brought back into the Agency

  21. Session 10: Common Pitfalls in Evaluation Planning and Management Objectives: • Conduct a content review by exploring common pitfalls • Identify plausible approaches, and commit to avoid identified pitfalls • Re-tune our thinking to be alert for other potential pitfalls

  22. Common Pitfalls • Nothing new under the sun? …but there are a lot of old things we don't know to do~ Let’s consider some of your repeater problems… EPM pit

  23. Common Pitfalls • Exercise… EPM Based on your experiences, suggest: • Why these common evaluation pitfalls occur? • What could result from falling in these pitfalls? • How you will avoid these pitfalls yourself? • How you could help others to avoid them? pit

  24. Common Pitfalls • Exercise… EPM Identify pits this EPM fell into: • What went wrong? • Who dropped the ball? • How will you avoid these pits? pit

  25. Session 11: Course Closure and Evaluation Objectives: • Overview reference materials • Conduct a post-test • Conduct course evaluation

  26. Continuing Learning “In today's challenging and complex world, being highly effective is the price of entry to the playing field. To thrive, innovate, excel, and lead in this new reality, we must reach beyond effectiveness toward fulfillment, contribution, and greatness. Research is showing, however, that the majority of people are not thriving. They are neither fulfilled nor excited. Tapping into the higher reaches of human motivation requires a new mindset, a new skill-set…” Stephen Covey, Author

  27. Continuing Learning • A few good habits… • Systematically continuepersonallearning • Peruse reference materials (Tabs 6 and 7) “A fire hose sip”… to whet your appetite; share with colleagues • Review evaluation report bibliographic materials • Browse recent evaluations (summaries; bibliographies) • Browse donor websites (relevant evaluations) • Seek/give coaching - mentoring (Give it away~)

  28. Post-Test • Closed “book & mouth”, please  • Questions: • Multiple choice • Fill in the blanks • Cross references

  29. Course Evaluation • Name and organization (Optional & appreciated~) • Rating per session (Rating scale 1 – 4) • Rating the: • Overall course • Course materials • Facilitation • Exercises • Venue • Breaks • Improvement opportunities

  30. Evaluation Planning and Management John Wooten ~ Lynn Keeys  • Thank you~ (jwootenjr@yahoo.com) (lynnkeeys50@yahoo.com)

More Related