Educational Video-based Scenarios and Evaluative Instrument For Handoff Communication

1. Educational Video-based Scenarios and Evaluative Instrument For Handoff Communication Saba Berhie1; Vineet M Arora1; Leora Horwitz2; Mark Saathoff1; Paul Staisiunas1; Jeanne M. Farnan1 1University of Chicago, Chicago, IL; 2Yale University, New Haven ,CT 10 10 8 10 8 6 8 4 6 2 6 4 0 2 Unsatisfactory Superior 0 4 Satisfactory Superior Unsatisfactory 2 0 Unsatisfactory Satisfactory Superior Professionalism Setting p <0.001 p <0.001 Communication Background Workshop Objectives & Structure p <0.001 • Utilize videos which highlight varying levels of handoff performance. • Rate and assess each video utilizing Handoff CEX instrument • July 2011– ACGME sets new limits for resident duty hours • Unintended consequence is increase in number of handoffs • Few trainees receive education on effective handoff communication (Horwitz, 2006) • Lack of validated instruments to measure handoff performance • HOW ACCOMPLISH THIS? VIDEO DEVELOPMENT • 6 scripts developed & reviewed • Changing handoff quality in the 3 • domains of performance • De-brief instrument development • WORKSHOPOUTLINE • Two 60-minutes workshops offered • Six videos watched and rated • independently by faculty participants using Handoff CEX • Post-workshop roundtable held after viewing the videos Instrument Development: Handoff CEX • Based on “Mini- CEX” instrument widely used in internal medicine (Norcini,2003) • Peer Evaluation Pilot Testing • Internal Medicine residents at the University of Chicago: 366 handoff observations • Factor analysis revealed that three domains captured 82% of the variance of overall sign-out quality • Communication, Professionalism and Setting Results Conclusions • 14 faculty from 2 departments participated • 73 of a possible 90 (82%) handoff observations were captured • Reliability • Cronbach’s alpha: 0.81 • Kendall’s coefficient of concordance: 0.59 • 3-item Handoff CEX shows promise as a reliable and valid tool to assess varying levels of videos depicting handoff performance • Raters had difficulty distinguishing between satisfactory and superior levels • Correlation between communication and professionalism • Future work in real-time clinical scenarios Figure 2: Faculty Ratings of Setting • Validity • Test of trend across ordered groups: p<0.001 • Two-way ANOVA • No rater bias; results significant only for difference between “superior” & “unsatisfactory” levels of performance Limitations • Single institution and small sample size • Future work planned at partner institutions Figure 3: Faculty Ratings of Professionalism Figure 1: Faculty Ratings of Communication MedEd Portal Submission Specific Aims • Arora V, Farnan J, Paro J, Vidyarthi A, Johnson J. Teaching Video:"Handoffs: A Typical Day on the Wards” MedEdPORTAL; 2011 • To create video-based examples of varying levels of handoff performance for education, adapting the approach in Arora et al MedEd Portal: • “Handoffs: A Typical Day on the Wards” • To validate an assessment instrument: • the Handoff CEX Research Funding • With generous funding from: • AHRQ R03 Grant # 1R03HS018278-01 • NIA T35 Grant #5T35AG029795-02 • Ability to detect level of performance for communication impacted by level of professionalism depicted