Assessing Evaluator Bias in Cases of Alleged Child Sexual Abuse

1. Assessing Evaluator Bias in Cases of Alleged Child Sexual Abuse Mark D. Everson, PhD University of North Carolina at Chapel Hill May 14, 2011 Email: mark_everson@med.unc.edu

2. CSA evaluators and trial jurors share two ethical responsibilities: • Approach cases with impartiality • Make objective judgments based on the available evidence Objectivity

3. The existence of juror bias is well documented in 100+ studies across several decades of research. Juror bias

4. A preference or inclination, especially one that inhibits impartial judgment (Online Dictionary) • A predisposition to decide a cause or an issue in a certain way (Black’s Law Dictionary) “Bias” defined

5. Evidence usually trumps bias • Structural safeguards reduce the impact of bias • Effective safeguards include: • Voir dire examinations of potential jurors • Structured deliberation process • Unanimous verdict requirements Research findings on juror bias

6. Examine evidence that evaluator bias may be common in our field • Conduct self-assessment of attitudes using the Child Forensic Attitude Scale (CFAS) • Review research using the CFAS to assess evaluator bias • Discuss implications for the field Our focus today

7. Unexplained differences in professional judgments (e.g., unrelated to training or experience) • Differences in professional judgments related to variations in preexisting attitudes Evidence of possible bias

8. Our personal experience of disagreements between evaluators assessing the same cases Anecdotal evidence

9. Seven studies suggest substantial unreliability in professional judgments about abuse allegations, Findings of prior research

10. The generalizability of these seven studies is limited by: • Sample issues • Ecological validity Limitations of prior research

11. Summary of Reliability Studies

12. Summary of Reliability Studies, continued

13. Summary of Reliability Studies, continued

14. Decision Exercise & Survey

15. Study Design: • 209 professionals: law enforcement, child protective services, child forensic interviewers • Detailed, 8 single-spaced page case summary, including partial transcript of child interview • Rating of probability of “true” allegation on 10 point scale Everson & Sandoval Study

16. (n = 209)

17. The most common explanation: Differences in training, experience or job title between professionals. Often, the more likely explanation: The intrusion of subjectivity and bias into the decision making process. Why the disagreement?

18. Evaluators should approach each case impartially and objectively, while weighing a range of hypotheses to explain the allegations. The prevailing view

19. How do subjectivity and bias enter a process that is supposed to be objective in nature? Even with well-trained, well-meaning evaluators? The critical question

20. “Evaluator decisions in cases of unconfirmed abuse allegations are based on the individual clinician’s review and analysis of the available data in light of the clinician’s own personal experiences, knowledge of the research, and implicit or explicit beliefs and values.” Herman (2005) Law and Human Behavior Vol. 29, No. 1, p. 91 One answer

21. “Evaluator decisions in cases of unconfirmed abuse allegations are based on the individual clinician’s review and analysis of the available data….” The process begins objectively:

22. “… in light of the clinician’s own personal experiences, knowledge of the research, and implicit or explicit beliefs and values.” But subjective factors quickly intrude:

23. Subjective factors can substantially increase the error rate in case decisions. • Subjective factors can undermine fairness and equity in how different cases are handled. So what? Why the worry?

24. Individual differences in 3 specific attitudes. Premises • Discrepant opinions in professional judgments about CSA can be explained in part by: • These 3 attitudes serve as predispositions or biases in how cases are approached.

25. Hypothetical Scenario • Observation: Many CSA victims fail to disclose, or falsely deny anal penetration • Development of new interview technique to address this issue • Field-testing in pediatric clinic

26. Field-Testing Design 72 5-to-7 year old girls Normal physical exam 50% receive additional anal exam 50% receive additional scoliosis exam Interviewed one week later

27. Field-Testing Results

28. Field-Testing Results

29. Field-Testing Results

30. Question: Assuming additional field tests with similar results- Should this new interview technique be implemented?

31. Group Poll

32. The answer depends upon a value judgment about which is worse: False negative errors (False denials) OR False positive errors (False disclosures)

33. CSA Decision Making Model Reality Abused Not Abused

34. CSA Decision Making Model Reality Abused Not Abused Evaluation findings

35. CSA Decision Making Model Reality Abused Not Abused Abused Not abused Evaluation findings Sensitivity = Concerns for False Negatives (or False Denials) Specificity = Concerns for False Positives (or False Disclosures)

36. Sensitivity and Specificity Both are indicators of diagnostic accuracy. Both are desirable. Improving one usually involves a trade-off with the other.

37. The Trade-Off The best strategy to minimize false negatives is to substantiate every case. However, this will maximize false positives.

38. In unclear cases: Emphasizing sensitivity = erring on the side of the alleged child victim Emphasizing specificity = erring on the side of the alleged abuser

39. Brief survey to assess three forensic attitudes: • Sensitivity emphasis • Specificity emphasis • Skepticism Self-scored Normed on 1600+ professionals Child Forensic Attitude Scale (CFAS)

40. Determining your sensitivity/ specificity predispositionfrom Parts 1 & 2

41. Part 3 of CFAS: Add scores from all 6 items for a total score • Rank your total score using the following table Scoring your CFAS skepticism subscale

42. Skepticism LevelTotal Score Low 6-10 Moderate 11-15 High 16-30 Part 3: Skepticism of Child Reports

43. Research Summary

44. Child Forensic Attitude Survey • Administered to 1,613 professionals • Six decision making exercises on subsets of participants • Data collection: 2004-2010 Everson & Sandoval (2012)

45. Research sample

46. Attitude Predisposition by Professional Group Sensitivity = black, Specificity = white, Balanced = hatched

47. Investigation CPS 1.5 LE 1.6 CFE 1.5 Treatment/ Support MH .89 VA .23 Assessment CFI .47 MED .23 Adjudication ADA 1.3 ATT 4.9 JUD 27.6 Specificity/ Sensitivity Ratio by Professional Group

48. Case TypeDescriptionN Highly Probable 2 ½ page vignette 269 Probable 3 hour exercise 150 Ambiguous #1 8 page case review 387 Ambiguous #2 1 page vignette 430 Improbable 1 page vignette 436 Highly Improbable 2 page vignette 269 Decision-Making Exercises

49. Sen/Spec Emphasis Subgroups: % rating case as “likely true”

50. Sen/Spec Emphasis Subgroups: % rating case as “likely true”