OYA Recidivism Risk Assessment (ORRA) OYA Recidivism Risk Assessment – Violent Crime (ORRA-V)

1. OYA Recidivism Risk Assessment (ORRA)OYA Recidivism Risk Assessment – Violent Crime(ORRA-V) Research & Evaluation

2. Defining Recidivism • Felony adjudication (conviction) within 3 years of release from closed custody or commitment to probation.

3. Defining the Assessments • ORRA: OYA Recidivism Risk Assessment. Identifies the likelihood a youth will recidivate after release from closed custody or commitment to probation. • ORRA-V: OYA Recidivism Risk Assessment–Violent. Identifies the likelihood a youth will VIOLENTLY recidivate after release from closed custody or commitment to probation. Recognizes the propensity for violence or threatening crimes that may result in physical harm.

4. Why Develop a Risk Assessment • Program evaluation • Placement and treatment decisions • Parole Decisions • Sentencing practices • RNA fails to differentiate risk populations • Poor predictive accuracy

5. The Data • 15,968 total youth • Date range of population January 2005 to May 2007 • Four Youth Populations • County Probation • Committed to OYA Probation • Released from OYA Close Custody Facility • Released from OYA Close Custody Facility to Supervision in the adult system

6. Dozens of Variables were Considered • Age at first referral • Total prior sex offense referrals • Total prior felony referrals • Total prior theft referrals • Total prior runaway referrals • Total prior property referrals

7. Dozens of Variables were Considered (cont.) • Total burglary referrals • Total prior misdemeanor referrals • Total prior robbery referrals • Total prior violation referrals • Total prior dependency referrals • Total prior harassment referrals

8. Variables contributing to the ORRA Scores and their effects Prior felony AOD referral (Y/N) Prior weapon referral (Y/N) Age Criminal mischief referral (Y/N) No. prior misdemeanor referrals No. prior theft referrals Adjudicated delinquent (Y/N) No. prior AOD referrals Current sex offense (Y/N) No. prior runaway referrals Gender (male higher risk) Interactions – mischief referral by No. prior misdemeanors No. prior theft referrals No runaway referrals +25.5% +21.2 + 4.6 +83.2 +10.3 + 5.2 +21.6 +11.1 - 39.5 +11.4 +20.4 Flattens out Increases Flattens out

9. So… What exactly does this mean?

10. Meet the Twins… Age 15 Male 1 Prior Misdemeanor Referral 3 Runaways 1 Prior Felony Drug Referral

11. Interpreting Scores • Each youth has a score between 0% and 100% • The score approximates the probability that the youth will recidivate • For example, a youth with a score of 40% has a 40% probability they will recidivate • This also means the youth has a 60% approximate score that the youth will NOT recidivate.

12. Model Accuracy • Overall Accuracy for ORRA = 73% • Accurate for all subpopulations • Accuracy of 50% suggests poor predictive accuracy • Accuracy of 100% suggests perfect predictive accuracy

13. Comparison of the Four Populations

14. Differences in Youth Populations

15. Predictive Accuracy

16. Interpreting Scores • ORRA and ORRA-V scores can also be evaluated for a specific population • The average score for a group of youth estimates the expected recidivism rate for the group

17. Program EvaluationActual vs. Expected Recidivism • Calculate risk scores • Expected (based on average risk of youth served) • Actual (based on recidivism of youth served) • Determine Relative increase or decrease • Facilitates meaningful comparisons across providers

18. Things Done and Things Still to Do • Done -- Test for all OYA youth groups • Males/Females • Minorities • Crime Type • Still to Do -- Make the ORRA dynamic • Incidence • Revocations • Programming

19. ORRA-V Used the same dataset Used “violent recidivism” – a subset of recidivism Violent recidivism includes murder, arson…robbery, assault, and burglary

20. Variables contributing to the ORRA-V Scores and their effects Male Prior weapon referral (Y/N) No. prior misdemeanor referrals No. prior felony referrals Prior felony assault referral (Y/N) Prior felony theft referrals (Y/N) Misdemeanor theft referrals (Y/N) Prior curfew violation (Y/N) No. prior runaway referrals Interactions: Weapons X felony theft Misdemeanor Referrals X felony referrals +178.1% + 62.0 + 13.5 + 31.3 + 32.1 + 36.1 + 20.1 + 22.1 + 8.6 Flattens out Flattens out

21. Differences between the ORRA and the ORRA-VVariable ORRA ORRA-V

22. OVIRA and ONIRA OVIRA measures the likelihood a youth will engage in a violent act in the first six months of closed custody ONIRA measures the likelihood a youth will engage in numerous non-violent incidents in the first six months of closed custody

23. Data for OVIRA and ONIRA Youth admitted to OYA between November 2007 and December 2009 N = 1,258 90% male and 10% female 27% property crime, 25% sexual offenses, and 9% robbery 64% YCF, 11% DOC, and 11% revoked

24. Variables considered for OVIRA and ONIRA ORRA and ORRA-V RNA data – aggression, drugs/alcohol, mental health, employment, relationships, attitudes, etc. Gender Age Sexual offender Special education and learning disability Other variables

25. OVIRA – OYA Violent Incident Risk Assessment Violence considered an assault or peer fight resulting in isolation/segregation Considered “immediately threatening to life, health, or facility safety, security, or good order.”

26. ONIRA – OYA Nuisance Incident Risk Assessment Considered four or more non-violent incidents in the first months of closed custody

27. Variables contributing to OVIRA and ONIRA scoresVariable OVIRA ONIRA

28. Typologies

29. Typologies C A E F B D

30. Type A Description Highest need population AOD use is high both current and historical Poor relationships and likely lack relationship skills Highest on aggression and attitude issues History of Mental Health = ADD/ADHD or mental health diagnosis – recommend analysis of RNA items 15.5 and 15.6 to differentiate ADD/ADHD versus Formal MH Diagnosis Education issues are prominent – recommend analysis of RNA item 3.1 for potential responsivity issues 3.1 = Special Education or Formal Diagnosis of Special Education Need (LD, SED, MRDD Indicators)

31. Treatment Recommendation Estimated to require longest dosage of treatment (e.g., 12-18 months) Group may require more stabilization than other groups due to co-occurring mental health and learning concerns AOD Treatment (longer in duration due to persistency) MH treatment with QMHP Educational intervention Social Skills/Relationship Skills development (intensive) Engagement in prosocial activities that can foster protective factors Potential family therapy component Aggression Replacement Training (intensive) Cognitive Behavioral program to address thinking

32. Type E Description 66% of this cluster is SO Highest on protective factors Low need for MH = ADD/ADHD or mental health diagnosis – recommend analysis of RNA items 15.5 and 15.6 to differentiate ADD/ADHD versus Formal MH Diagnosis Education issues are low – recommend analysis of RNA item 3.1 for potential responsivity issues 3.1 = Special Education or Formal Diagnosis of Special Education Need (LD, SED, MRDD Indicators)

33. Treatment Recommendations (Type E) Sex Offender Treatment when appropriate (Abbreviated Kaufman or general cognitive behavioral treatment) Capitalize on whatever activities youth engaged in prior to coming as leverage for treatment engagement Seek opportunities for continued engagement

34. Optimal Length of Stay Calculated length of stay in months Plotted LOS against recidivism for the overall sample On average, providers reduce recidivism by approximately 3% per month of supervision But, there may be a window of time where providers are most effective

35. Program Evaluation Continuum

36. Summary ORRA ORRA-V OVIRA ONIRA Typology (being completed) Optimum dose (next project) Program continuum (being developed) LOS report Recidivism report Timing study for JJPOs Revocation (being completed) Culture climate survey (data collection completed) Staff-management/leadership survey (data collection now) PREA – identifying vulnerable youth (surveyed thru October)

38. Close Custody Populations Making comparisons while controlling for risk

39. Why • Problems with the RNA • Not valid for OYA females • Approximately 85% of the youth in Close Custody were High Risk – little practical information • The Area Under the Curve (AUC) was .56 • DOC had the solution • Methodology for developing risk tool based on local data • The AUC for their risk tool was .78

40. How • Methodology • Subjects • N = 28,431 dispositions (19,309 unique youth) • Qualifying events occurred between 1/1/2005 and 5/14/2010 • Youth qualified if they were: • Placed on county probation • Committed to OYA probation • Released from an OYA close custody facility • Release from OYA close custody to supervision in the adult system

41. What’s Next • ONIRA: OYA Nuisance Incident Risk Assessment • OVIRA: OYA Violent Incident Risk Assessment

42. How • Methodology continued • Omitted disposition records of youth: • Supervised under interstate compact • Returned to DOC to complete their sentences in adult institutions • Committed to OYA or county probation who were subsequently committed to an OYA YCF without recidivating • Randomly selected one disposition per youth • Final dataset: N = 15,986

43. How • Methodology continued • Dependent (Outcome) Variable • Recidivism event: OYA official recidivism measure • Felony Adjudication or Conviction • Disposition of formal supervision • Groups • Tracking Dates • Tracking Periods: 12-, 24-, 36-, 48-Month

44. How • Methodology continued • Independent Variables • Over 50 starting variables • Bootstrap Re-sampling • Run 1000 randomly sampled logistic regressions for each tracking period • Lists the proportion of time each variable is significantly related to the outcome variable • Selected the top 30% of the variables to develop the model • Run stepwise Logistic Regression for each tracking period

45. How • Methodology continued • Developing the Model • Run stepwise Logistic Regression for each tracking period • Determine the concordance rate for each model • Test for interactions • Run stepwise Logistic Regression for each tracking period including significant interaction variables

46. How • Methodology continued • Selecting and refining the final model • 36-Month tracking period had the highest concordance rate (.73) and included • 12 predictor variables • 3 interaction terms

47. How • Results • Model Accuracy: • AUC = .72 • Estimates Actual Recidivism

48. How The Model

49. What for • Interpreting ORRA Scores • Each youth get a score between 0 and 1 • The score represents the probability that the youth will recidivate • For example, a youth with a score of .42 has a 42% probability they will recidivate • The average score for a group of youth estimates the expected recidivism rate for the group • For example, the average ORRA score for females on OYA probation was 13.1 and the actual recidivism rate was 13.0.

50. What for • ORRA has multiple uses • Placement and treatment decisions • Parole decisions • Program evaluations • Sentencing practices • Foundation for future improvement in risk assessment