“ If you don’t know where you are going, how are you gonna’ know when you get there?” Yogi Berra

1. “If you don’t know where you are going, how are you gonna’ know when you get there?” Yogi Berra Common Elements of Strong Program Evaluation THEORY OF CHANGE: Cause/effect relationship of program LOGIC MODEL: Identifies key program components and their relationship to program outcomes PERFORMANCE MEASUREMENT & IMPACT EVALUATION: Differences between the two

2. If you deliver the intervention as planned, it will bring about a measurable change/outcome in the community/participants in relation to your identified community need. Theory of Change Main Elements

3. LOGIC MODEL INPUTS: What we invest ACTIVITIES: What we do OUTPUTS: Direct products from program activities EVIDENCE OF CHANGE/OUTCOMES Short-Term Outcomes – one year • Changes in knowledge, skills, attitudes Medium -Term Outcomes – three years • Changes in behavior or action resulting from new knowledge Long-Term Outcomes – ten years • Meaningful changes in condition

4. Birth & Beyond Evaluation Practices A home visitation program at 9 Family Resource Centers serving parents at risk of child abuse and neglect to prevent entry/re-entry into Child Protective Services • Common Outcomes • Sacramento County Commitment to CPS Evaluation • External Evaluator from the onset • Shared Database across all 9 sites • Monthly Data Review distributed program wide • Quarterly and Annual Reports • Program Improvement Based on Data & Outcomes

5. Birth & Beyond Lessons Learned • Integrated evidence-based practices into the model • Intervention must align with assessments • Analyze attrition of those served and ways to improve • Member training aligns with intervention • Since 2008, serving higher % of at-risk risk families • Data facilitates peer to peer learning • Track/monitor short-term outcomes to ensure meeting medium-term outcomes • Expanded services by adding one FRC (based on data)

6. Birth & Beyond Evaluation Challenge Impact Evaluation: strives for high rigor with scientifically-based research design comparing people exposed to an intervention (“experimental” group) to people not exposed (“control” or “comparison” group) How do you design an impact evaluation that does not deny services to parents at risk of child abuse & neglect? • Technical assistance from NORC • Work in partnership with Sacramento County Child Protective Services to enhance B&B evaluation conducting a quasi-experimental study • Select a matched sample of families who have been referred to CPS during the same time as Birth & Beyond families and did NOT receive Birth & Beyond services (“control” or “comparison” group)

7. 2010 -2013 Child Protective Services Pre-, During, and Post-Program

8. 20 Year Child Death Review Team Report

11. PERFORMANCE MEASUREMENT & IMPACT EVALUATION Performance Measurement • Provides a view of how program is functioning • Does not “prove” that an intervention caused an outcome • Strives for high quality data, striking a balance between rigor & staff time • Focuses on shorter term changes, observed within a year Impact Evaluation • Strives for high rigor; with scientifically-based research designs • Seeks to prove causal relationship; program is the specific cause of improvements within the target population • Evidence that intervention causes the outcomes • Focuses on longer term changes as well as short term outcomes