Performance and Outcomes that Demonstrate Effectiveness of Criminal Justice

1. Performance and Outcomes that Demonstrate Effectiveness of Criminal Justice FADAA/FCCMH Annual Conference August 7 & 8, 2013 Neil Gaer, MPA VP, Senior Director Phoenix Houses of New England Jack M. Feinberg, LMHC, CAP VP, Clinical Director Phoenix Houses of Florida

2. Outline of workshop • Introduction • Project overview – what we set out to do • Demonstration of the integrated CEST assessment and EHR • Reports, data, and how to use it • Discussion and Q&A

3. I. Introductions

4. II. Project Overview

5. What we set out to do Context: Organization was implementing an EHR nationally Aim: Standardize a process where every client in the NE region * receives a CEST assessment at intake * and gets re-assessed at periodic intervals (depending on length of stay) during treatment

6. Why do we want to do this? NIATx promising practice – Enhance Motivation • Tailor treatment to each client’s circumstances and needs • Offer motivation groups for clients to enhance treatment engagement • Identify clients at risk for dropping out and intervene • Map ongoing care at regular intervals (treatment plan reviews using assessment results)

7. Why CEST and what is it? • Client Evaluation of Self and Treatment • Developed by the Institute of Behavioral Research at TCU • decades of research, federal outcome studies • studies of treatment processes (what happens inside treatment, what are the elements of effective treatment)

8. Tailoring treatment TCU Treatment Process Model Readiness Interventions what’s inside the “black box”? Aftercare Services & Support Networks what’s inside the “black box”? Recovery Process Social Skills Training • Early Engagement • Participation • Therapeutic Relationship Client Factors Problem Severity & Treatment Readiness RecoverySkills Training Retention Threshold • Follow-up • Outcomes • Drug use • Crime • Social • Functioning • Early Recovery • Changes in • Thinking • Behavior Cognitive Interventions Behavioral Interventions

9. CEST assessments • Client self-report • 4 modules, about 30 questions each • Measures client changes in thinking and behavior • which may lead to more long term outcomes in housing, employment, legal involvement

10. CEST Scales measuring...

11. sample questions PSYCHOLOGICAL FUNCTIONING SCALES E. Self Esteem (SE) 17. You have much to be proud of. 59. You feel like a failure. ® 71. You wish you had more respect for yourself. ® 88. You feel you are basically no good. ® 100. In general, you are satisfied with yourself. 109. You feel you are unimportant to others. ® F. Depression (DP) 57. You feel interested in life. ® 74. You feel sad or depressed. 76. You feel extra tired or run down. 90. You worry or brood a lot. 97. You feel hopeless about the future. 123. You feel lonely.

12. Adaptive Interventions

13. Recap of TCU’s Adaptive Treatment Model • manualized curricula for identified problem areas • evidence-based (listed on NREPP) • it’s FREE • download CEST assessments, curriculum manuals at www.ibr.tcu.edu

14. How did Phoenix House New England come upon and utilize the CEST • Searching for evidenced based practices • NREPP • Need for outcome studies for funding sources • Monitor Quality Performance Improvement (QPI) as part of management reporting • And it sounded like a good idea at the time

15. Meeting CARF definitions of • Effectiveness – measures the quality of care through measuring change over time. • Efficiency – measures how well resources are used to accomplish agency program goals. • Satisfaction – measures oriented toward the person served, family members, personnel, and funding sources.

16. III. The integrated CEST and EHR

17. What was the plan? • Build the assessments into EHR. Have automatic date triggers. Automate real-time scoring. • Build reports to look at change over time on aggregate level. • Create a Quality Performance Improvement (QPI) system to check how we are doing (efficiency & effectiveness).

18. The old way...

21. Example of On Screen

23. Close up of Scoring Summary

25. CEST in New England • Residential • Adult • Adolescent • Corrections • Outpatient • Community Corrections

26. Welligent and CEST • Integration • Transparency • Individualized • Facility Averages • National Norms • Graph All Client Scores Over Time • REAL TIME

27. So how did we do

28. How many clients were receiving comprehensive assessments? • At admission/intake for initial treatment planning • many programs were using the ASI, some were using other tools • During treatment for ongoing monitoring of progress • none

29. Setting due date triggers example: Adolescent Residential Programs

30. Efficiency Benchmarks

31. Let’s focus on the New England Adult Residential programs

33. III. Reports, Data, and How to Use it

34. So what does this all mean?

35. Individual case example“John” • 44 y.o. male • crack (age 23), marijuana/alcohol (age 11) • referred by parole – was incarcerated 1 year • 5 prior treatment episodes • H.S. diploma, unemployed • depression, on medication • parole officer and client felt he needed treatment to maintain sobriety while transitioning back to community

36. Assessing client motivation levels

37. Assessing client risksand needs

38. Monitoring progress in treatment

39. Real time feedback monitoring progress and defining treatment goals • Reports can be reviewed with clients in clear, simple language “I see that your scores on feelings of anger have increased quite a bit (peer support decreased, treatment satisfaction decreased) since last month. Can you tell me what’s been going on?” • Opportunities for therapeutic intervention “So you’re having problems with your roommate. Have you and [roommate] talked about it? What skills might you use from your “Ideas for Better Communication” group? • Collaborative treatment planning “Last time we met you talked about how your feelings of anger sometimes get in the way of your relationships with others. What would you think about participating in the “Emotional Cartography” (or Anger Management) group?

40. Aggregate data

41. Monitoring program deliverables(and adjusting evidence-base practices) • Our goal within the course of treatment is to • increase psycho-social strengths scores in • Self Esteem • Decision Making • Social Support • Expectancy for Recovery • and to decreasepsycho-social problems scores in • - Depression • - Anxiety • - Anger/Hostility • - Risk Taking

42. Increase in scores on Self Esteem, Decision Making

43. Increases in Social Support, Recovery Expectations

44. Decrease in scores on Depression, Anxiety scales

45. Decrease in scores for Hostility/Anger, Risk taking

46. What was the plan? Build the assessments into EHR. Have automatic date triggers. Automate real-time scoring. Build reports to look at change over time on aggregate level. Create a QPI system to check how we are doing (efficiency & effectiveness).

47. Interactive reports

48. Efficiency Benchmarks

49. Interactive reports on SharePoint

50. Report filters