Agency for Healthcare Research and Quality Technical Assistance Conference Call on the

1. Agency for Healthcare Research and Quality Technical Assistance Conference Call on the Hospital Survey on Patient Safety Culture (HSOPSC) February 15, 2005

2. Objectives • Introduce the Hospital Survey on Patient Safety Culture (HSOPSC) and its related tools • Convey support from public and private organizations • Showcase the stories of two early adopters who have implemented the survey in their hospitals • Answer questions AHRQ slide

3. Speakers Carolyn M. Clancy, MD is the Director of the Agency for Healthcare Research and Quality (AHRQ). Dr. Clancy is a general internist and health services researcher, and a graduate of Boston College and the University of Massachusetts Medical School. James B. Battles, PhD is aSenior Service Fellow for Patient Safety and Medical Errors at the Agency for Healthcare Research and Quality (AHRQ). Dr. Battles is a senior content specialist in patient safety for AHRQ’s research in patient safety initiative. (jbattles@ahrq.gov) Gina Pugliese, RN MS is the Vice President of the Safety Institute, Premier Inc., and editor of Premier’s Safety Share eNewsletter. She has associate faculty appointments at the University of Illinois School of Public Health and Rush University College of Nursing, Chicago and writes and lectures frequently on patient safety-related topics. (safety_culture@premierinc.com) Deborah McKay, CAPT, NC, USN is the Division Director of the Patient Safety Program within the Office of the Assistant Secretary of Defense for Health Affairs, TRICARE Management Activity. Capt. McKay is responsible for overseeing Patient Safety Programs for all U.S. military healthcare facilities throughout the world. (deborah.mckay@tma.osd.mil) Barbara Horne is a Vice President and Chief Nursing Officer at Indian River Memorial Hospital in Vero Beach, Florida and has worked there for 11 years. She has past experience as a risk manager and quality director as well as a nursing leader. (barbara.horne@irmh.org) Deborah Cutts is a Patient Safety Manager at the White River Junction VA Hospital in Whiter River Junction, Vermont. She has worked for the Department of Veterans Affairs (VA) for 20 years and has extensive experience with Joint Commission hospital surveys. (deborah.cutts@med.va.gov) Veronica Nieva, PhD is a Vice President and Director of the Organizational and Management Research Group at Westat. Dr. Nieva has a background in social/organizational psychology and is one of the principal investigators responsible for the development of the Hospital Survey on Patient Safety Culture under an AHRQ contract. (veronicanieva@westat.com) Joann Sorra, PhD is a Senior Study Director at Westat. Dr. Nieva has a background in industrial/ organizational psychology and is one of the principal investigators responsible for the development of the Hospital Survey on Patient Safety Culture under an AHRQ contract. (joannsorra@westat.com) AHRQ slide

4. Survey Background • Sponsored by the Medical Errors Workgroup of the QuIC, funded by AHRQ, developed by Westat • The Quality Interagency Coordination Task Force (QuIC) consists of representatives from 11 Federal agencies • Survey development process involved: • A review of literature & existing safety culture tools • Interviews with hospital staff • Cognitive testing • Input/comment from researchers & healthcare stakeholders • Pilot testing with over 1,400 respondents from 21 hospitals AHRQ slide

5. Safety Culture Dimensions 1. Overall perceptions of safety 2. Frequency of events reported 3. Supervisor/mgr. expectations & actions promoting patient safety 4. Organizational learning—continuous improvement 5. Teamwork within units 6. Communication openness 7. Feedback & communication about error 8. Nonpunitive response to error 9. Staffing 10. Hospital management support for patient safety 11. Teamwork across hospital units 12. Hospital handoffs & transitions Also-- • Patient safety “grade” • Number of events individuals have reported in last 12 months • Total of 51 survey items: 3 to 4 items per dimension AHRQ slide

6. HSOPSC Toolkit • Toolkit contents • Survey User’s Guide: Step-by-step guidance on how to implement the survey • Survey • Safety culture dimensions & reliabilities • Powerpoint template for displaying survey results • Description of development of survey & pilot study (Appendix A) • Quality & Safety in Healthcare article on uses of safety culture assessment (Appendix B) • On AHRQ web site at www.ahrq.gov/qual/hospculture/ • AHRQ has plans to support a benchmarking database for the survey AHRQ slide

7. Safety Institute, Premier Inc. • Developed a “Customized XL Data Tool” • Excel™ tool for easy entry/rapid display of AHRQ HSOPSC survey data • Input raw data - Create graphs and tables to display results • Analyze - safety culture dimensions by work area, unit, or profession • Share - results electronically • Benchmark - tool calculates overall safety culture composite scores for benchmarking • Available for download at www.premierinc.com/safety • A special thank you to Westat and Indian River Memorial Hospital Safety Institute, Premier Inc. slide

8. Department of Defense/TRICARE • Initiated dialogue with ARHQ/Westat in 2002 as a means of assessing the culture of safety within the Military Health System (MHS) • DoD had limited tools to assess PS culture • Army’s Patient Safety Survey • Medical Team Training Facility Assessment Tool • Will launch the AHRQ safety culture survey as a web survey in 2005 • Surveying 125,000 staff from 150 Army, Navy and Air Force hospitals and clinics in the U.S. and internationally • Current plan is to use tool for one cycle and evaluate DoD/TRICARE slide

9. Indian River Memorial Hospital • Barbara Horne, VP/Chief Nursing Officer • 335-bed not-for-profit community hospital in Vero Beach, Florida • Early adopter of survey—late summer 2004 • Decided to conduct a patient safety survey • Establish a baseline measurement • Perceived mistrust in past; was it dissipating? Indian River Memorial Hospital slide

10. Decision to Use AHRQ Survey • Learned of AHRQ survey at workshop • Research-based • Easily administered • Support available during administration and evaluation • Can be used across the organization • After discussion with developers at Westat, removed 4 staffing questions • Concerned about setting up an unrealistic expectation that staffing patterns would be changed based on results of the survey Indian River Memorial Hospital slide

11. Survey Sample • Conducted census of about 1,200 staff • Initial discussion about including non-clinical staff, but decided they should participate • Non-clinical staff spend time in care delivery areas, make observations, see results of care delivered, or talk with patients after discharge • Offer a different perspective in regard to culture • Wanted across-the-board assessment to be able to compare clinical versus non-clinical responses Indian River Memorial Hospital slide

12. Data Collection Procedure • Information systems department developed a web-based version and placed it on our intranet • Meal ticket incentive • A certificate could be printed upon completion of the web survey • Survey promotion activities • Letter sent to employees homes before survey • Notice on pay checks • Posters and e-mails sent out • Follow-up letter midway through time period • 6-week data collection time period • Due to hurricanes, extended the field period! Indian River Memorial Hospital slide

13. Survey Response • 619 surveys completed; 52% response rate largely due to incentive of meal ticket attached to completion • 75% of respondents were clinical staff • More manager support and encouragement may have improved our response rate, but we did much better than we anticipated Indian River Memorial Hospital slide

14. Survey Results • Hospital technical staff developed a survey results reporting tool in Excel • Bar charts, 5-point Likert scales condensed into just three points (negative, neutral, positive) • Can display results in aggregate, by department, length of service or by job classification • Distributed survey results electronically through the Excel tool • We provided initial input for Premier’s Customized Excel tool which is similar to what we developed Indian River Memorial Hospital slide

15. Sharing Survey Results • Results shared with all directors and managers • Not yet formally distributed housewide but departments have shared results in staff meetings • Surprised and pleased with the extent of participation • Encouraged overall with the results (less negative than anticipated) • Concerned that pockets of staff may perceive there to be a risk of retribution for bringing safety issues to light Indian River Memorial Hospital slide

16. Action Planning/Follow-up • VP/CNO had one-on-one conversations with managers in areas of concern; plan to do further follow-up • Currently developing individual goal-based incentive pay program • For managers, would like one goal to address issues raised in survey as appropriate • Results to be presented during patient safety module at the Leadership Academy of the Advisory Board Company in Washington, DC (in Feb 2005) Indian River Memorial Hospital slide

17. Lessons Learned & Future Plans • Needed 3 hospital staff who worked together on the survey project • All were detail-oriented, good at follow through • Next time, plan to improve survey results follow-up • Give staff access to results more quickly • Initiate action plans more quickly • Definitely plan to redo the survey after 1 year, hopefully in the fall • Not during the peak of hurricane season! Indian River Memorial Hospital slide

18. White River Junction VA Medical Center • Deborah Cutts, Patient Safety Manager • 60-bed, small, rural, acute care facility in White River Junction, Vermont • Pilot test site—summer 2003 • Decided to conduct a patient safety survey • To investigate the event reporting process in our facility • Meet JCAHO standards for obtaining staff perceptions of patient safety • Have a replicable / data driven process White River Junction VA slide

19. Decision to Use AHRQ Survey • The survey was being offered free of charge • We administered the survey and entered the data • Westat included our data as a pilot site, analyzed our data & sent us a survey results report • Westat was incredibly “user friendly” offering support/assistance when needed • Implemented the longer, pilot test version of the survey • Contained same items as current version plus additional items that were being tested White River Junction VA slide

20. Survey Sample • Since this was our first survey, decided to conduct a census of all 573 staff • Surveys were anonymous • Believe anonymity was a driver behind candid responses we got back White River Junction VA slide

21. Data Collection Procedure • Administered paper survey • Survey promotion activities • Announced that we would • Initiate process improvement based on survey results • Share results at our annual Patient Safety Fair • Announced upcoming survey through weekly employee bulletin • Mailed prenotification letter of support from our Director to all employees • Six-week time period, but accepted late surveys White River Junction VA slide

22. Survey Response • 162 surveys completed; 28% response rate which we believe is good • Would have been lower without publicity and pre-notification that was done • Demographic data on respondents • 64% had direct interaction with patients • 30% nurses; 17% unit assistants/clerks/secretaries; 11% physicians/residents White River Junction VA slide

23. Survey Results • Westat provided an executive summary and report showing item responses • Also provided a benchmarking report called “Comparing Your Results” • 1-page summary comparing our hospital against 20 pilot hospitals on each safety culture area • Comparative information was priceless White River Junction VA slide

24. Sharing Survey Results • Results presented to senior leaders, followed by medical and nursing staff • Distributed to a variety of services within the hospital • Shared survey results and small improvement projects which resulted from our report at our Patient Safety Fair White River Junction VA slide

25. Additional Benefits of the Survey • Went through a scheduled JCAHO survey • Surveyors very interested in the survey tool and discussion of the results • Specifically told us they “loved it”! • In 2004, our hospital was a Malcolm Baldridge Quality Award finalist • Baldridge team was duly impressed that we had measured our hospital’s patient safety culture and taken initiatives to improve White River Junction VA slide

26. Follow-up & Taking Action • Buy-in of results was initially difficult • Initially, senior leaders were defensive • Now several cycles of improvement initiatives/refinement are paying off • Implemented a number of improvement initiatives • Made changes to our adverse event/near miss reporting process • Enhanced process for communicating/fostering a culture of patient safety based on no-blame White River Junction VA slide

27. Lessons Learned & Future Plans • Next time will be ready to share the survey results as soon as possible • Don’t want staff to forget they completed the survey • Want them to link survey results to improvement initiatives • Talk it up! • Plan to conduct second, follow-up survey • Through Westat, we are launching a web-based version of survey in March during Patient Safety Awareness Week • Would like entire VA to adopt this patient safety culture survey White River Junction VA slide

28. General Feedback about the Survey • Still early, but feedback has been generally positive • Some potential adopters concerned about length, but staff find it only takes about 10 minutes to complete • Understand there is broad interest in benchmarking on the survey • AHRQ is planning to support a benchmarking database Westat slide

29. Reactions from Pilot Hospitals • At 6 to 9-month follow-up, what did pilot sites find useful about the survey? • Some results were surprising • “We thought we were doing great, but realized the staff did not have the same frame of mind so we decided to keep pushing those efforts.” • Some results confirmed what they already knew • “Energized” leaders to improve things they knew needed to be improved, served as a catalyst to finally move forward and implement action plans in areas of weakness • Written comments were useful • Illustrated points brought out by the numeric data • Some comments were very specific and could be acted upon quickly • Good that it was “not just a nursing or physician survey” but representative of hospital staff as a whole Westat slide

30. Reactions from Pilot Hospitals • What was difficult? • Not as many physician responses as desired • Logistics of survey administration (Takes time, staff, resources to implement) • Not sure what to do with the survey results • Don’t know where to start • Don’t know what interventions would improve things Westat slide

31. Future HSOPSC Conference Calls • March call: “Survey Administration Logistics” • How to implement the survey • Deciding whether to use a web, paper, or scannable surveys • What results to present • How to interpret survey feedback data • April call: “Taking Action” • You have your survey results, Now what? • How to communicate results throughout the hospital • Determining areas of priority • Deciding when to “resurvey” Westat slide

32. How Does My Hospital Get Started? • Download and review the toolkit materials on the AHRQ web site and the Premier Excel data reporting tool • Read over the “Frequently Asked Questions” document on AHRQ web site • Watch email for details about March and April technical assistance conference calls • Email questions to Westat at safetyculturesurvey@westat.com Westat slide