Increasing Impact Understanding Measurements of Impact in Evaluation June 2013 Jennifer Farley
Overview • Why evaluate? • Evaluation: • Types, Continuous Improvement and Neglect • Outputs vs. Impact • Measures of Impact • Is impact good enough? • When it’s not good enough • Maximizing Evaluation • Questions
Why Evaluate? • Ensures Quality • Increases Knowledge • Prioritize Resources • Accountability • Demonstrate need and effectiveness • Marketable (Russ-Eft & Preskill)
Types of Evaluation • Developmental • Formative • Summative • Monitoring and Auditing • Outcome Evaluation • Impact Evaluation • Performance Measurement (Russ-Eft & Preskill)
Continuous Improvement Evaluation
Top 10 Reasons Evaluation is Neglected • Misunderstanding the purpose and role of evaluation • Fear • Lack of Skills (Real or Perceived) • No one has asked for it • It is not used • Time-consuming • Perceived costs vs. Perceived benefit • Leaders already know what works • Prior experience • Lack of Value (Russ-Eft & Preskill)
Measures of Impact • Measures of Change • Attitude, Knowledge and Behavior • Comparison • Pre- and Post-program • Calibration • Is the primary impact what is measured? • Is the program driving the measure, or is the measure driving the program?
Is the Impact Good Enough? • Goals • Program established goals • Funder established goals • Comparison • Population level data • Are the target and general population the same? • The continuum of evaluation and research
When Impact isn’t “Good Enough” • Re-calibrate Measures • What is impact? • Participant focus groups • What best measures impact? • Refine measurement tools • Staff Training • Re-Pilot surveys • Incorporate Measures of Program Quality • Look at HOW programs and services are offered • Explore field-specific measures of quality • Youth Program Quality Assessment • Quality Rating System
Results: Maximizing Evaluation • What is the purpose of the evaluation? • Funder Required? • Is there a requirement to “Act” on the results? • How are the results communicated to all stakeholders? • How are results used in planning?
Contact Information Jennifer Farley Censeo Solutions firstname.lastname@example.org 515.371.1754 www.censeosolutions.com
Works Cited Russ-Eft, D., & Preskill, H. (2009). Evaluation In Organizations: A systematic approach to enhancing learning, performance, and change.New York: Basic Books.