Ethical Considerations in Online Testing for Pre-Employment Psychological Evaluations

1. Police and Public Safety Ethical Considerations: Internet Based Testing & Access to and Use of Supplemental Information in Post-Offer Psychological Pre-Employment Evaluations  Scott Stubenrauch, Psy.D., Shaun Wehle, Psy.D., Heather McElroy, Ph.D., ABPP & E.E. Griffith, Ph.D.

2. Presentation/Discussion Objectives Attendees will be able to... 1) Identify common ethical dilemmas related to online testing. 2) Utilize best practices to mitigate and address challenges related to online testing. 3) Identify various sources of supplemental information beyond test and interview data when conducting psychological preemployment evaluations. 4) Apply relevant laws and best practices related to use of supplemental information when conducting psychological preemployment evaluations.

3. Online Testing

4. Online Testing Vignette Ethical Considerations A licensed clinical psychologist colleague of yours has been providing post-conditional offer preemployment psychological evaluations for Awesome County Sheriff’s Department for the past several years. Their typical assessment battery includes instruments that measure cognitive ability, personality and psychopathology, followed by a 30 to 45-minute structured clinical interview. As a small county department, they have a limited demographic pool of applicants and a 40% failure rate.   The Sheriff expressed a history of concerns related to a decreasing number of suitable candidates over the past few years from an already limited candidate pool to begin with. The department has made changes to address this issue including an increase in marketing efforts to expand their applicant pool to include applicants from surrounding counties and even out-of-state applicants. While the number of applications from individuals living in surrounding counties and out-of-state has increased, the rate of those who drop out of the process before reaching the psychological evaluation has also increased.  In completing a survey of these applicants, the department learned that barriers related to travel was the main driver of this attrition.   After attending a national conference, the Sheriff learned about how other departments are using online testing to combat this issue.  They have raised a concern that many of these applicants will not be able to complete their psychological assessments in person and would like your colleague to offer an online model of assessing potential applicants. As a competent psychologist they understand that there are legal, ethical and competency considerations, so they are reaching out to you for consultation.

5. Online Testing Vignette Potential Concerns… Ethical Considerations Things to Consider… Professional Competence New Assessments New Vendor Platform Understanding of Telepsychology Telepsychology-Specific Laws Collecting, Storing, Accessing and Using of PHI (& PII) HIPAA (Confidentiality/Privacy/BAA) HITECH (CyberSecurity) Informed Consent Jurisdictional Licensing Laws (PSYPACT) Client Satisfaction Applicant/Candidate Satisfaction • Including a cognitive assessment in testing battery • Adverse Impact • Proctored vs non-proctored online assessment administration • Using a cognitive assessment online (cheating - right and wrong answers) • Multiple Forms/Adaptive Testing (CAT) Engine • Writing non-Internet searchable items • Using personality & psychopathology assessment online (accuracy of self-reporting traits and tendencies) • Confirming examinee identity • Examinee access to technology

6. Advantages and Disadvantages to UIT Pros Cons Test standardization issues to be considered Environmental factors Lighting Temperature The presence of others Consistency and reliability of: Computer hardware Software Internet connectivity Potential cheating http://www.siop.org/tip/jan11/05UIT.aspx • Decreases cost, increases speed and efficiency • Allows applicants to access at their convenience • Can conserve organizational resources • Equipment • Staffing • Expands recruitment options • Location flexibility • Increases diversity of the applicant pool • Can ensure precise instructions, timing, and scoring (i.e., structured computerized system in place)

7. Cheating Detecting Cheating http://www.siop.org/tip/jan11/05UIT.aspx Deter Cheating Adaptive testing and sampling items randomly from a large bank of items Unique passwords or unique single-use web link for each applicant Issuance of a warning Decreases the individual’s belief in the ability to successfully cheat or fake the assessment, resulting in decreased intention to fake (e.g., informed consent, testing instructions) Inform applicant that methods are being used to detect cheating or faking and if falsification is detected, there will be consequences (e.g., criminal prosecution) – research shows this deters cheating in personality assessments Reasoning with the test taker Normalizing that no human is perfect Appeal to the test takers morality (e.g., ensuring the process is structured and fair, and hitting on their belief that they are a good and honest person • Response Style Indices • Monitor/restrict computer functions: print screen, copy and paste, or access to the Internet browser • Use a warning and or notify test administrator when these functions are attempted • Verification of a successful candidate’s score with the later interview data and data obtained from records review • Identify applicant through webcam, audio monitoring, or biometric verifications (where legal) • Suspicious or abnormal response tendencies • Quick response times - Time Stamping

8. Online Testing Vignette Efficacy/Effectiveness of Online Testing Articles: Jones, J.W. and Newhouse, N. (2005). Unproctored online testing programs: Managing risks to benefit from a flexible and low cost assessment model. Institute for Personality and Ability Testing (IPAT). Joubert, T. and Kriek, H.J. (2009) Psychometric comparison of paper-and-pencil and online personality assessments in a selection setting. Kobak, K. A., Greist, J. H., Jefferson, J. W., & Katzelnick, D. J. (1996). Computer-administered clinical rating scales: A review. Psychopharmacology, 127(4), 291—301 Lievens, F. and Burke, E. (2011), Dealing with the threats inherent in unproctored Internet testing of cognitive ability: Results from a large-scale operational test program, UK Journal of Occupational and Organizational Psychology, 84, 817–824 Mead, A. D., & Drasgow, F. (1993). Equivalence of Computerized and Paper-and-Pencil Cognitive Ability Tests: A Meta-Analysis. Psychological bulletin, 114(3), 449-458. Vallejo, M. A., Maňanes, G., Comeche, M. I., & Díaz, M. I. (2008). Comparison between administration via Internet and paper-and-pencil administration of two clinical instruments: SCL-90-R and GHQ-28. Journal of Behavior Therapy and Experimental Psychiatry, 39, 201-208. • Arthur, W., Glaze, R.M., Villado, A.J. and Taylor, J.E. (2010) The magnitude and extent of cheating and response distortion effects on unproctored Internet-based tests of cognitive ability and personality. International Journal of Selection and Assessment, 18, 1, 1-16. Gupta, B.A. (2007). • Erlanger, D. M., Kaushik, T., Broshek, D., Feldman, D., & Festa, J. (2002). Development and validation of a web-based screening tool for monitoring cognitive status. Journal of Head Trauma Rehabilitation, 17(5), 458-476.
 • French, C. C., & Beaumont, J. G. (1989). A computerized form of the Eysenck Personality Questionnaire: A clinical study. Personality and Individual Differences, 10(10), 1027-1032. • Gnambs, T., amdKaspar, K. (2015). Disclosure of sensitive behaviors across self-administered survey modes: a meta-analysis. Behavior Research Methods, 47, 4, 1237. • Hart, R. R., & Goldstein, M. A. (1985). Computer-assisted psychological assessment. Computers in Human Services, 1(3), 69--75. • Herrero, J., & Meneses, J. (2006). Short Web-based versions of the perceived stress (PSS) and Center for Epidemiological Studies-Depression (CESD) Scales: A comparison to pencil and paper responses among Internet users. Computers in Human Behavior, 22, 830-846.

9. Online Testing Vignette Resources: Telepsychology Best Practices Resources Gamble, N., Boyle, C. & Morris, Z.A. (2015). Ethical practice in telepsychology. Australian Psychologist, 50, 292-298. Luxton, D. D., McCann, R. A., Bush, N. E., Mishkind, M. C., & Reger, G. M. (2011).  mHealth for Mental Health: Integrating Smartphone Technology in Behavioral Healthcare. Professional Psychology: Research & Practice, 42, 505-512. Luxton, D. D., Pruitt, L. D. & Osenbach, J. E. (2014). Best Practices for Remote Psychological Assessment via Telehealth Technologies. Professional Psychology: Research & Practice. 27-35. Naglieri, J.A., Drasgow, F., Schmit, M., Handler, L., Prifitera, A., Margolis, A. & Velasquez, R. (2004). Psychological Testing on the Internet: New Problems, Old Issues. American Psychologist, 59(3), 150-162. Shore, J. H., Mishkind, M. C., Bernard, J., Doarn, C. R., Bell Jr, I., Bhatla, R., Brooks, E., Caudill, R., Cohn, E., Barthold, J., Eppolito, A., Fortney, J., Friedl, K., Hirsch, P., Jordan, P., Kim, T., Luxton, D. D., Lynch, M., Maheu, M., McVeigh, F., Nels. (2014). A Lexicon of Assessment and Outcome Measures for Telemental Health. Telemedicine and e-Health, 20(3) 282 - 292. Telepsychology Standards & Guidelines • American Telemedicine Association • https://www.americantelemed.org/resource/learning-development/http://www.apa.org/practice/guidelines/telepsychology.aspx • American Psychiatric Association • APA-ATA-Best-Practices-in-Videoconferencing-Based-Telemental-Health.pdf • American Psychological Association • https://www.apa.org/pubs/journals/features/amp-a0035001.pdf • International Test Commission • https://www.intestcom.org/files/guideline_computer_based_testing.pdf

10. Online Testing Vignette Resources: Telepsychology Training Programs: TeleMental Health Institute (TMHI) ZurInstitiute Person-Centered Tech Center for Credentialing & Education • HIPAA & HITECH LAW (PHI) • https://www.hhs.gov/hipaa/for-professionals/index.html • The European General Data Protection Regulation GDPR (PII) • Telepsychology 50-State Law Review • https://www.apaservices.org/practice/advocacy/state/telehealth-slides.pdf (2013) • https://www.ebglaw.com/content/uploads/2017/10/EPSTEIN-BECKER-GREEN-2017-APPENDIX-50-STATE-TELEMENTAL-HEALTH-SURVEY1.pdf (2017) • ASPPB (PSYPACT) – Operational • https://www.asppb.net/page/PSYPACT • https://www.asppb.net/mpage/legislative

11. Contact Information: Scott Stubenrauch, Psy.D. swspsych@gmail.com 708-401-7625 Shaun Wehle, Psy.D. shaunwehle@gmail.com 219-213-4931

13. Supplemental Data/Resources

14. Types of Supplemental Data/Resources Q: As a psychologist, in occupational evaluation contexts, can you and should you use? • Previous Psychological Evaluation Report (Pre-Employment & FFDE) • Background Investigation • Previous Mental Health Treatment Records • VA Records • Collateral Contact Interviews/Reference Checks • Social Media Checks • Sexual Behavior Inventory (Grif’s presentation) A: It depends (e.g., job-relevancy, invasiveness, legality)

15. Obtaining Collateral Information - FFDE • Vignette Information: • Firefighter with the department for 8 years with numerous complaints against him for his attitude, off duty behavior including driving too fast through the neighborhood, and two off duty domestic violence incidents. • Firefighter’s lawyer has successfully sealed the arrest record • Department believes, based on information available that alcohol was a factor, that the firefighter may have an alcohol use disorder • Department refers firefighter for FFDE with provider A, a board certified PPSP. • Provider A sends a letter to the department and says that her findings are indeterminable.

16. Next steps…. • What could the department do from here? • Handle the behavior through disciplinary channels • Obtain a 2nd Opinion FFDE • Other options?

17. What happened next… • The department requested a 2nd opinion with Provider B. • Provider B requested a copy of the letter sent by Provider A • Department contacted Provider A for a copy of their release form to be sure they were not violating any HIPAA laws. • Provider A refused to give them a copy of their release upon learning that the department had requested a 2nd opinion. Provider A also told the department that they were not permitted to release that information to Provider B. • What would you do if you were Provider A? • What would you do if you were Provider B?

18. Information from the Interview • Firefighter provided letter from his girlfriend in an attempt to displace blame for the domestic incident (hitting, pushing and shoving) • Firefighter said he had previous therapy to address anger management at the advice of his attorney following the first incident of domestic violence. He said that he experienced frustrated feelings related to his previous divorce. He denied any dx or sxs. • Provider B requested a release form to speak to the treatment provider. • Firefighter was hesitant but provided the release • Documentation from the Tx provider was inconsistent with the Firefighter’s report and indicated that he was been diagnosed with GAD, Depression, Alcohol Use Disorder and Narcissistic Personality Disorder. He was also prescribed Celexa. • Now what?

19. Fit or Unfit? • Are his off duty problems associated with a mental health disorder? • If we believe that they are associated with a mental health disorder, how do we support that conclusion? • What do we do about the knowledge that he has been less than forthcoming? • How can we best serve the department?

20. Contact Information • Heather K. McElroy, Ph.D., ABPP • hmcelroy@stonemcelroy.com • 770-431-6858