Evaluating Intensive Family Preservation Services Determining program success and trajectory using administrative, client-centered and cost-centered data. Presented at the Intensive Family Services 5th National Practice Symposium Sydney, Australia April 2, 2004 Presented by: Raymond S. Kirk, PhD Research Professor School of Social Work University of North Carolina at Chapel Hill email@example.com
Features of the North Carolina Intensive Family Preservation Services (IFPS) Program • IFPS is more effective than traditional child welfare services for high-risk families; • Measured improvements in family functioning occur during IFPS and are associated with positive treatment outcomes; • Placement prevention rates range from 87-93% of families, and 89-94% of children each year since the program began; • IFPS is a cost effective program, and yields a very favorable cost/benefit ratio; • Generalized benefits accrue to families that have received the service (as measured by living arrangements of families, service utilization by families, and families’ abilities to handle future challenges and stress)
Was the IFPS program effective ? Did it: Stabilize families in crisis ? Keep children safe ? Improve family functioning ? Prevent unnecessary placement ? Was IFPS program model followed (QA) ? Targeting the right families ? Home-based, time-limited services ? 24/7 availability, caseloads < 4 ? Focus on strengths and building competencies? How do we know?What are the basic evaluation questions? A comprehensive evaluation answers these questions directly.
Comprehensive evaluation requires Administrative data Program Quality Assurance (QA) data Family Assessment/Clinical data Treatment Outcome data Cost data Comprehensive evaluation of IFPS
Regardless of data type/source, datamust be Available (minimal “missing” data) Accessible (easily retrievable, transportable, automated) Reliable (consistent use of terms, service units, variable definitions) Longitudinal (minimize “snap shots” or cross-sectional studies) Comprehensive evaluation of IFPS
NC / IFPS / MISAn automated case record system • Completely replaced paper records • Standardizes data entry • Assures complete data (no missing data) • Integrates administrative and clinical data • Eliminated the need for perfunctory QA visits • Provides case-level, longitudinal records for both administrative needs and program evaluation
Imminent Risk Criteria for Imminent Risk Children by Referral Source Similar tables available for Mental Health and Juvenile Justice referrals.
Cross-tabulation and testing of relationships between variables • For all domains on the NCFAS, a positive, statistically significant relationship is observed between strengths and the absence of placement, and between problems and out-of-home placement on all domains. For the 1924 families served during SFY 2000, 2001 and 2002, the results are: • for Environment: Chi Square = 85.821, df = 5, p=<.001; • for Parental Capabilities: Chi Square = 127.944, df = 5, p<.001; • for Family Interactions: Chi Square = 137.667, df = 5, p<.001; • for Family Safety: Chi Square = 159.372, df = 5, p<.001; and • for Child Well-Being: Chi Square = 165.706, df = 5, p<.001.
Longitudinal data permit dynamic analysis of placements Risk of Placement After CPS Report/Referral to IFPS
…and to control for various risk levels or other variables during analysis Risk of Placement After CPS Report/Referral to IFPS for Children with 1+ Prior Placements
Cost analysis is essentialBoth cost-effectiveness and cost/benefit Cost Table 1. Children at Risk of Out-Of-Home Placement at Intake
Cost analysis, continued:Modeling costs and cost-savings Cost Table 2. Estimating potential and actual costs of placements * This number is less than 70 because 6 children were homeless, “on runaway” or hospitalized for health
Cost-effectiveness and Cost/Benefit Estimates • 1,232 children at imminent risk, a total potential cost of $11,071,532; • 64 children were placed in, known placements at an cost of $767,033; • IFPS diverted an estimated $10,304,499 from placement costs; a cost savings of 93.07%; • if the cost of operating the IFPS program ($3,683,403) is subtracted from the gross savings ($10,304,499), a net savings of $6,621,096 results; • cost/benefit ratio of IFPS for is 1:1.8; that is, for every $1.00 spent providing IFPS, $1.80 is not being spent on placement services for imminent risk children; • the cost of delivering IFPS was $2,990 per imminent risk child, and $5,598 per family; • had all 1,232 children been placed as originally indicated, the placement cost per imminent risk child would have been $8,987, and the families would not have received any services as part of those expenditures
Cost analysis, continued:Determining the fiscal “break-even” point
Summary of Major Findings (1) • Intensive Family Preservation Services are able to improve family functioning in all areas measured by the NCFAS. • Some areas of family functioning (e.g., Parental Capabilities, Family Interactions, Family Safety, Child Well-Being) are more amenable to change during a brief intervention than other areas (e.g., Environment). • Family functioning scores on all domains, as measured on the NCFAS, are statistically significantly associated with placement and non-placement at the end of IFPS. • Overall, placement prevention rates have been very stable, ranging between 88% and 94% each year since SFY 1994
Summary of Major Findings (2) • IFPS program cost analysis indicates that IFPS is very cost-effective. It also reveals a very favorable cost/benefit ratio. • Long-term client tracking reveals durability of IFPS services one year after service: living arrangements, service utilization and abilities to handle family stress. But also reveals continuing significant family stressors beyond the means of IFPS programs to impact over the long term: poverty, housing, and chronic emotional/mental health needs. • The retrospective study demonstrates the clear superiority of IFPS over traditional services when risk factors are controlled in the analysis. • Retrospective study survival curves indicate a predictable attrition phenomenon among IFPS families that occurs by 6 months after IFPS. This finding suggests that follow-up family contact and opportunity for additional services should be offered 3 to 5 months after IFPS.
THANK YOU! • Evaluation of Intensive Family Services • For the complete SFY 2002 annual report on the North Carolina IFPS program, on which this presentation is based, please feel free to contact me, Ray Kirk, directly at firstname.lastname@example.org