Using Simulation in TeamSTEPPS ® Training

1. Using Simulation in TeamSTEPPS® Training

2. Objectives • To know and be able to apply the Event Based Approach to Training • To know and be able to develop TeamSTEPPS training scenarios • To know and be able to develop TeamSTEPPS performance measures • To know and be able to conduct effective debriefs of team performance

3. Course Outline • Phase 1. Scenario Development • Exercise • Phase 2. Measurement Development • Exercise • Phase 3. Debriefing • Lessons Learned

4. Simulation • Team skills are developed through practice and feedback • Simulation allows health care professionals to practice TeamSTEPPS skills and strategies in a safe learning environment • Simulation includes • Role play • Part-task trainers • Human patient simulation • Standardized patients

5. Keys to Success • Proper scenario design • Focus on learning objectives • Provide more than one opportunity to practice team behaviors • Accurate measurement • Include process and outcome measures • Capture behaviors • Debriefing • Learning happens in the debrief • Include feedback on how to improve performance

6. TeamSTEPPS Resources • Scenarios – Tab I • 131 Vignettes • Organized by department and TeamSTEPPS skill and tool • Team Performance Observation Tool – Tab A, Appendix C • Tool for observing team performance • Observers should practice using the tool • Can be adopted to a particular type of care

7. EBAT • Event-Based Approach to Training (EBAT) • Not a new method • U.S. Navy, Targeted Acceptable Responses to Generated Events (TARGETs) (Fowlkes, Lane, Salas, Franz, & Oser, 1994) • FAA Advisory Circular 120-35C • Line Operational Simulations • How to design simulation scenarios • Health care domain • Simulation Module for Assessment of Resident Targeted Event Responses (SMARTER) (Rosen, Salas, Silvestri, Wu, & Lazzara, 2008)

8. Phase 1 • Scenario Development • Specify teamwork skills • Define learning objectives • Choose clinical context • Define event sets • Define targeted responses

9. Specify Teamwork Skills • Skills are general categories of behavior you want to train • Teamwork is too complex to train in a single scenario • Need to focus on a subset of competencies • There are four core skills in TeamSTEPPS

10. Define Learning Objectives • Learning objectives should be explicit and measurable • Focus on specific TeamSTEPPS behaviors • Include the following information • Performance – Specifies the behavior • Condition(s) – Specifies the conditions under which the behavior occurs • Standards – Specifies the level of expected performance

11. Example Objective Communication Learning Objective: Uses two-challenge rule or CUS to advocate for a patient in labor experiencing frequent, strong contractions

12. Choose a Clinical Context Considerations • Medicine is a notoriously task-specific domain • Not all contexts are equal for training purposes • Choose a clinical context that affords opportunities to perform • TeamSTEPPS provides 131 scenarios (Tab I) • Contexts should be appropriate for eliciting the team behavior of interest

13. TeamSTEPPS Scenario 86 Sally Rodgers, a 25-year-old nullip in labor at term who is dilated 3 cm. This is a change from 2cm over the previous 90 minutes. Sally is having frequent, strong to palpation contractions that are extremely uncomfortable. She is trembling, complaining of nausea, and begging her nurse for pain relief. The patient’s primary nurse believes epidural anesthesia would be appropriate and informs the obstetrician (OB) attending. The OB attending states he wants the patient to be dilated 4–5 cm before she receives the epidural. The nurse reiterates to the attending OB that her assessment is that the patient is in active labor. Although Sally’s cervix has not demonstrated active labor yet, her nurse believes the pain relief and relaxation resulting from an epidural would be beneficial for the patient. The attending OB agrees to the epidural placement. The patient is fully dilated and begins pushing 3 hours after the placement.

14. Clinical Context • Scenario 86 • Sally Rodgers, a 25-year-old nullip in labor at term who is dilated 3 cm. This is a change from 2cm over the previous 90 minutes. Sally is having frequent, strong to palpation contractions that are extremely uncomfortable. She is trembling, complaining of nausea, and begging her nurse for pain relief.

15. Define Event Sets • Event sets are the building blocks of a scenario • Event sets consist of: • Trigger – The incident to elicit the team behavior • Distracters – Characteristics of the situation (i.e., conditions) in which the behavior is to occur • Examples • Trigger:The OB attending states he wants the patient to be dilated 4–5 cm before she receives the epidural • Distracter: Sally has a strong contraction and screams

16. Define Targeted Responses • Targeted responses • Behavioral responses to each event set trigger • Expected levels of performance (i.e., standards) • Example • Trigger:The OB attending states he wants the patient to be dilated 4–5 cm before she receives the epidural • Response: The nurse uses two-challenge rule or CUS to advocate to the physician that Sally should receive pain medication now

17. Guidance • Simulation scenarios should consist of 3-5 event sets • Provides more than one opportunity to demonstrate behaviors • Creates reliability • Each event set should not include more than one trigger event • Event sets can be created by breaking a clinical procedure into chunks • Embed triggers into these chunks

18. Trauma ExamplePre-Hospital/Transport

19. Trauma ExamplePrimary Survey

20. Scenario Development • Select a scenario from Tab I • Identify/develop the following items • TeamSTEPPS skill • Learning objective(s) • Clinical context • Event set • Trigger and distracters • Targeted responses • Be prepared to present your results

21. Phase 2 • Develop Measures • Consider level of analysis • Clarify purpose • Decide what to measure • Select a measure

22. Leadership Utilizes resources efficiently to maximize team performance Balances workload within the team Delegates tasks or assignments, as appropriate Conducts briefs, huddles, and debriefs Empowers team members to speak freely and ask questions Situation Monitoring Includes patient/family in communication Cross monitors fellow team members Applies the STEP process when monitoring the situation Fosters communication to ensure team members have a shared mental model Mutual Support Rating Provides task-related support Provides timely and constructive feedback to team members Effectively advocates for the patient Uses the Two-Challenge rule, CUS, and DESC script to resolve conflict Collaborates with team members Communication Rating Provides brief, clear, specific, and timely information to team members Seeks information from all available sources Verifies information that is communicated Uses SBAR, call-outs, check-backs and handoff techniques Team Performance Observation Tool

23. Consider Level of Analysis • Individuals • MD • Nurse • Anesthesia • Teams • Delivery team • C-Section team • Multi-team – Team Structure • Core team • Contingency team • Administrative team

24. Clarify the Purpose • Diagnose root causes of performance deficiencies • Identify specific weaknesses • Provide feedback • Relay information regarding strengths and weaknesses as a remediation plan • Assessment • Evaluate the level of proficiency or readiness

25. Decide What to Measure • Outcomes tend to be more quantifiable and answer the question “What happened?” • Processes answer the question “Why did it happen?”

26. Outcomes • Outcomes • Sometimes referred to as measures of effectiveness (MOEs) • Provide an indication of the extent to which the outcome of the task was successful • Are important for most measurement purposes • What outcomes could we assess? • Accuracy – Precision of performance (e.g., correct medication) • Timeliness – How long (e.g., time to incision) • Productivity – How much (e.g., patient volume in ED) • Efficiency – Ratio of resources required versus used (e.g., OR supplies)

27. Process • Processes • Sometimes referred to as measures of performance (MOPs) • Explain how and why certain outcomes may have happened (“Was the decision made right?” versus “Was the right decision made?”) • Important when diagnosing root causes of performance deficiencies and providing feedback or follow-on training • Types of Process • Procedural – Taskwork • Non-procedural – Taskwork • Teamwork

28. Measurement Tips • Assess processes as well as outcomes when diagnosing performance deficiencies or providing feedback • Consider measuring at multiple levels to identify the weak link • Provide multiple opportunities to perform the same task or skill over the course of an exercise

29. Select a Measure • Checklists • Consist of items/actions that have dichotomous answers such as Yes/No; Right/Wrong; Performed/Not Performed • Example: Primary Survey ABCs • Frequency counts • Indication of the number of times that a behavior, action, or error occurs • Example: Use of CUS, SBAR, Two-Challenge • Rating scales • Numerical or descriptive judgment of how well a task was performed

30. Checklist Tips • Best used with a scripted scenario versus “free play” • Items should be related to triggers embedded into the scenario • Each item should represent a single action taken by the individual, team, or battlegroup • Label or define the response category that you are using

31. Checklist Action/Behavior Yes No Assess Airway Breathing Circulation/FAST Exam Disability Exposure and Environment

32. Frequency Count Tips • Frequency counts are better when measuring acts of commission than acts of omission • Overt actions or errors versus failing to demonstrate a particular behavior • Frequency counts are good when you want to know how often a specific action is taken or task is performed • Frequency counts can be recorded during a critical event in an exercise or throughout the entire scenario

33. Frequency Count Positive Instances Communication Check-back Call-outs SBAR Unintelligible Communications Mutual Support Positive Instances Two-Challenge CUS Task Assistance

34. Graphic Rating Scale • The team leader assigned roles to the Trauma Team. • 2. The PGY2 used check-back to confirm orders. Very Effective Ineffective x 1 2 3 4 5 6 Strongly Disagree Strongly Agree x 1 2 3 4 5 6

35. Anchored Rating Scale 1 2 3 4 5 Used check-back once to confirm care plan at end of case Used check-back to confirm all medication orders Used check-back to confirm critical orders, primary and secondary survey Used check-back to confirm all critical orders Did not use check- back Communication: Used check-back during trauma resuscitation. x

36. Rating Scales • Uses • Rating scales are good for assessing quality when it does not equate to a sum of quantity • Rating scales are good for assessing tasks that are less procedural in nature • Rating scales are better to use when you are evaluating performance on a continuum • Tips • Define or clearly describe what you are measuring • Include a specific stem or item to accompany the rating scale

37. Measures Develop a measure to assess performance for the scenario you developed earlier Be prepared to present your measures

38. Phase 3 • Debriefing • Introduce the debrief process • Describe what happened • Conduct an analysis of performance • Identify lessons learned

39. Description Phase • Recap of what happened in the scenario • Team members share their perspectives on what occurred during the scenario and reach common ground • This helps ensure everyone takes away similar lessons from the experience • How measurement can help • Provides a structure for understanding the scenario • The event sets or behavior categories can be used to structure discussion • Focus on critical aspects of performance • Helps to keep the discussion focused on events relevant to the learning objectives

40. Analysis Phase • A systematic investigation of why things happened in the scenario as they did • The team focuses on understanding what went well and what could have been done better • How measurement can help • Can help compare the team’s performance with standards of performance • Were the TeamSTEPPS behaviors performed when necessary? If so, were they performed correctly or could they be improved?

41. Application/Generalization Phase • A look ahead to how the team can generalize what they learned in the scenario to their daily practice • The team generates lessons learned and discuss what needs to be corrected and how to correct it • How measurement can help • Explicit event sets can be used to draw parallels between the scenario and the actual clinical environment • Explicit measures associated with these events help promote reflection about how to transfer what went well to the actual clinical environment

42. Tip for Success 1 • Tip 1: Don’t overwhelm learners or observers — Keep it simple • People can integrate only a few key learning points from a scenario • Observers have a limited attention span and frequently have to multi-task. Don’t ask too much; you won’t like what you get • Rule of thumb: Have a key event every 1 to 2 minutes of scenario time

43. Tip for Success 2 • Tip 2: Telling someone how well they did is not good enough • They need diagnostic feedback • Specific • Behaviorally focused • Descriptive

44. Tip for Success 3 • Tip 3: Train observers • Training, by definition, is systematic. For measurement to guide this process, it too must be systematic, reliable, and valid • Make sure everyone has common expectations about performance • Develop and use a scoring guide

45. Rater Scoring Guide

46. Tip for Success 4 • Tip 4: Keep teamwork and clinical skills separate • Use “dual debriefs” • Provide feedback on teamwork skills as a team • Correct an individual’s major clinical deficiencies in a follow-up session • Don’t overcomplicate the clinical nature of the scenario when your main purpose is to train teamwork • Teamwork novices especially should be given the chance to focus on teamwork in the scenario, not complex clinical issues • As the team members’ teamwork skills advance, more complex clinical scenarios can be used

47. Tip for Success 5 • Tip 5: Event-based methods involve more than just measurement • Good training design practices • Good scenario design practices • Good debrief facilitation practices

48. Teamwork Actions Apply the Event Based Approach to Training Able to develop TeamSTEPPS training scenarios Able to develop TeamSTEPPS performance measures Know how to conduct effective debriefs of team performance