Elimination of Familial Sex Offenders Inflate the Estimated Efficiency of the MnSOST-R

1. Elimination of Familial Sex Offenders Inflate the Estimated Efficiency of the MnSOST-R Jacqueline Waggoner, Ed.D. Richard Wollert Ph.D. Assistant Professor 1220 SW Morrison St. #930 University of Portland Portland, OR 97205 503.943.8012 360.737.7712 waggoner@up.edu rwwollert@aol.com www.richardwollert.com 2005 Western Psych. Assn. Convention April 14-17 Portland, OR

2. Purpose • This study is a continuation of an ongoing evaluation of the validity of the Minnesota Sex Offender Screening Tool-Revised (MnSOST-R) • MnSOST-R is an actuarial test for the prediction of sexual recidivism (ATSR) of convicted sex offenders 2005 Western Psych. Assn. Convention April 14-17 Portland, OR

3. Major Components of Actuarial Tests • Test items • Examples: Age, number of sex offenses • Items added to obtain a total test score • Experience table • Consulted to find out what percentage of those with the offender’s test score have re-offended in the past. This is the offender’s estimated recidivism risk. 2005 Western Psych. Assn. Convention April 14-17 Portland, OR

4. Development of MnSOST-R • Funds provided by the Minnesota Department of Corrections (DOC), 1991 • Has never been published in a peer- reviewed journal 2005 Western Psych. Assn. Convention April 14-17 Portland, OR

5. Measurement Theory and Research Methodology and the MnSOST-R • Does not conform to a number of criteria for evaluation of tests, testing practices and effects of test use* • (Revised Standards for Educational and Psychological Testing)* *American Educational Research Association (AERA); American Psychological Association (APA); National Council on Measurement in Education (NCME) 2005 Western Psych. Assn. Convention April 14-17 Portland, OR

6. Effect of Unpublished Status of MnSOST-R • Limits outlets for scholarly review and criticism of the MnSOST 2005 Western Psych. Assn. Convention April 14-17 Portland, OR

7. The MnSOST is Widely used in Sexually Violent Predator (SVP) Cases • Overview of SVP Laws • Involve post-prison civil commitment of “high risk” sex offenders • SVP laws have been passed by legislatures in 16 states and the District of Columbia 2005 Western Psych. Assn. Convention April 14-17 Portland, OR

8. States with SVP Civil Commitment Laws • Arizona • California • Florida • Illinois • Iowa • Kansas • Massachusetts • Minnesota • Missouri • New Jersey • North Dakota • South Carolina • Texas (outpatient commitment) • Virginia • Washington • Wisconsin • District of Columbia 2005 Western Psych. Assn. Convention April 14-17 Portland, OR

9. Why Continued Scrutiny of MnSOST-R is Important • Poor test development procedures • Serious implications of the test for justice and public safety in civil commitment cases 2005 Western Psych. Assn. Convention April 14-17 Portland, OR

10. Development of MnSOST-R • Small original sample (256 incarcerated sex offenders) for a 16-item test • Enrichment: included 16 recidivists who did not belong to the sample • De-selection: took out 113 familial offenders who belonged in the sample • Recidivism Rate = 35% - 41% vs. usual 20% – 25% • “Cut Score” > 13 used for recommendation for commitment 2005 Western Psych. Assn. Convention April 14-17 Portland, OR

11. Validity Indicators • Sensitivity at cut score of 13 = .15 • 1-Specificity = .02 • Correlation with recidivism = .45 • Area under the ROC Curve = .77 • Comparative test efficiency (calculated by Wollert, 2005, using Bayes’s Theorem) 2005 Western Psych. Assn. Convention April 14-17 Portland, OR

12. Efficiency of MnSOST vs. Other ATSRs For High Risk Offenders of Different Ages 2005 Western Psych. Assn. Convention April 14-17 Portland, OR

13. Most Thoroughly Documented Shortcoming of the MnSOST • Serious flaws have been found in all proposed experience tables • 1995 SOST (Base Rate = 41%) • Replaced by MnSOST-R in 1999 • Base Rate for MnSOST-R was 35% • Same offenders were in both samples • No explanation for the drop in base rate 2005 Western Psych. Assn. Convention April 14-17 Portland, OR

14. Flaws in 1999 MnSOST-R • Test Contents • Original Experience Table (Base Rate = 35%) • Estimated Table for samples with 21% Base Rate • Estimated Table for samples with 15% Base Rate • On cross-validation the recidivism rate for those with scores >13 decreased from 88% to 44% (Wollert, 2002). 2005 Western Psych. Assn. Convention April 14-17 Portland, OR

15. 12/03: A Third Version of MnSOST Was Formulated • Includes almost all of the original sample of offenders and a new sample of 220 offenders • Classification was much less accurate for the new sample than it was for the original sample • Who knows when the next MnSOST will appear, or why? 2005 Western Psych. Assn. Convention April 14-17 Portland, OR

16. Another Possible Problem That Has Not Been Studied Involves the Elimination of Familial Offenders • The reliability of this procedure has never been determined • Raises possibility that some or all of the MnSOST validity estimates are inaccurate (Wollert, 2003) 2005 Western Psych. Assn. Convention April 14-17 Portland, OR

17. Method for Studying This Issue • Obtain data from a representative sample • Re-calculate validity indicators 2005 Western Psych. Assn. Convention April 14-17 Portland, OR

18. Bartosh et al. (2003) Reported Data for a Representative Sample • N = 186 Sex offenders released from prison • r with sexual recidivism = .096 (ns) • ROC = .58 (ns) • This means the test does not differentiate recidivists from non-recidivists 2005 Western Psych. Assn. Convention April 14-17 Portland, OR

19. Bartosh Shared Her Frequency Data with Waggoner and Wollert • Results of Waggoner’s and Wollert’s analysis of Bartosh data • Sensitivity at cut score of 13 = .14 • 1- Specificity = .067 • Much larger than that for the non-representative sample (i.e., .02) • Much less efficient than other ATSRs when used to predict recidivism for those with high scores 2005 Western Psych. Assn. Convention April 14-17 Portland, OR

20. 2005 Western Psych. Assn. Convention April 14-17 Portland, OR

21. Discussion • Validity indicators reported by MnSOST test developers are inaccurate and idiosyncratic as a result of non-representative sampling • A second study on a large representative sample (N = 468) is currently underway with Barbaree and Langton to explore this problem further 2005 Western Psych. Assn. Convention April 14-17 Portland, OR

22. Conclusion • Rationale for SEPT was to identify procedures that enhance test validity • Test developers need to be aware of and follow SEPT procedures • Clinicians need to be aware of the extent to which the tests they use are based on SEPT procedures 2005 Western Psych. Assn. Convention April 14-17 Portland, OR

23. References • American Educational Research Association, American Psychological Association, & National Council on Measurement in Education. (1999). Standards for educational and psychological testing. Washington, DC. • Bartosh, D. L., Garby, T., Lewis, D., Gray, S. (2003). Differences in the predictive validity of actuarial risk assessment in relation to sex offender type. International Journal of Offender Therapy and Comparative Criminology, 47(4), 422-438. • Doren, D. M., Dow, E. A. (2003). What “shrinkage” of the MnSOST-R? A response to Wollert (2002b). Journal of Threat Assessment, 2(4), 49-64. • Epperson, D. L., Kaul, J. D. and Huot, S. J. (1995) Predicting risk of recidivism for incarcerated sex offenders: Updated development on the sex offender screening tool (SOST). Paper presented at the 14th Annual Research and Treatment Conference of the Association for the Treatment of Sexual Abusers, New Orleans, Louisiana. • Epperson, D. L., Kaul, J. D., & Hesselton, D. (1997) Final report on the development of the Minnesota Sex Offender Screening Tool-Revised (MnSOST-R). St. Paul, MN: Minnesota Department of Corrections. • Epperson, D. L., Kaul, J. D., Huot, S., Goldman, R., & Alexander, W. (2003, December). Minnesota Sex Offender Screening Tool-Revised (MnSOST-R): Development, validation, and recommended risk level cut scores. St. Paul, MN: Minnesota Department of Corrections. • Wollert, R. W. (2002). The importance of cross-validation in actuarial test construction: Shrinkage in the risk estimates for the Minnesota sex offender screening tool-revised. Journal of Threat Assessment, 2(1), 87-102. • Wollert, R. W. (2003). Additional flaws in the Minnesota Sex Offender Screening Tool-Revised: A response to Doren and Dow. Journal of Threat Assessment, 2(4), 65-78. • Wollert, R. W. (2005). An application of Bayes’s Theorem to age-wise sexual recidivism rates. Paper presented at the meeting of the Western Psychological Association, Portland, OR. 2005 Western Psych. Assn. Convention April 14-17 Portland, OR