What is a Significant Healthcare Event?

1. Health Foundation SHINE Award 2012/13Enhanced Significant Event Analysis: A Human Factors Systems Approach for Primary CarePaul Bowie, Elaine McNaughton, Deirdre Holly, David Brucewww.nes.scot.nhs.uk/shine/

2. What is a Significant Healthcare Event?

3. Investigations of Significant Events Poorly Conducted: Problems and Impacts • Incidents are highly selective (or non-engagement) • Negative feedback (interferes with ability to assimilate & process information beyond the ‘self’ level) • Second-victim syndrome (impact on health & wellbeing of clinician: guilt, embarrassment, shame…) • Perceived blame culture (fear, distrust, punitive action, litigation…) • Lack of a structured analytical framework (long standing issue) • Many SEAs demonstrate a lack of ‘systems thinking’ • Most clinicians attribute events to their own actions/inactions

4. Lack of meaningful and constructive investigations Missed opportunities to learn & improve (personal, team & organisational) SEA becomes a tick-box exercise Increased workforce stress, frustration & sick levels Wasted time, energy and resource Low engagement in formal incident reporting Investigations of Significant Events Poorly Conducted: Problems and Impacts

5. History of SEA(compared to hospital based techniques) Case-based discussion (Bradley, 1992; Buckley, 1992; Pringle et al, 1994) Flanagan’s Critical Incident Technique (John C Flanagan, 1954) Embedded in practice (most Scottish primary care professions) Quality of SEA (Bowie et al, 2009; McKay et al, 2007)

6. What We Set Out to AchieveWe aimed to design, develop and test a theory-informed ‘guiding tool’ to support the SEA process in primary care settings. Overcome SEA deficiencies by introducing human factors systems principles Highlight and differentiate the interactions between the individual professional and the immediate workplace and wider organisational issues at play. Individual Level: guide clinicians to reflect upon and contextualise their emotional reactions - achieve a state of psychological readiness to move on. Team Level: a systems-centred analysis of the significant event. Underlying assumptions: gain a deeper understanding of the human-system interactions at play may lessen –ve emotional reactions and apportioning of personal blame may lead to more meaningful and effective action plans for improvement.

7. How did we do it? • Multi-professional steering group/project manager • Tapped into existing human factors and safety science expertise • Reviewed relevant literature • Aim 1. Design a conceptual framework (error theory and ergonomic model) • Aim 2. Develop ‘guiding tools’ (individual & team levels) • Aim 3: Recruited clinicians and managers (qualified and in-training) to test our interventions • Formal evaluation of impact • Final project report

8. Methods – Recruitment, Testing & Evaluation • Professional Groups: Community Pharmacy, General Dental Practice, General Medical Practice, Optometry and Clinical Psychology across NHS Scotland (n=180) • Educational Leaders identified suitable groups to approach (e.g. speciality or vocational training, trainers/tutors) • Participants undertook enhancedSEA as part of existing educational, appraisal or contractual arrangements (or none) • Emailed Flyer and directed to dedicated website • Website – one-stop-shop • Evaluation (mixed methods): captured learning - attitudes, experiences, perceptions of feasibility/usability/impact of guiding tools etc.

9. Our Results1. Development Outputs

10. CONCEPTUAL FRAMEWORK for enhancedSEA

11. THE GUIDING TOOLs enhancedSEA Method • The new approach is divided into three parts:  • Small Personal Booklet (with 4 card inserts) • A3 size Desk Pad • New written format for enhancedSEA reports

12. 1. A small 12-page Personal Booklet (with 4 card inserts) to help individuals reflect on the potential emotional impacts of a significant event - and their own role in the event - by using human factors principles to gain a clearer understanding of all of the contributory factors involved.

13. 2. An A3 size Desk Pad for the care team, the sheets from which can be distributed to all those who attend a team meeting to analyse significant events.  Each sheet contains instructions and prompts to guide the care team to take a systems-based approach to analysing the event in question and take notes on what was agreed – a small set of card prompts may also be used in conjunction.

14. 3. A re-designed written report format for enhancedSEA – to prompt a systems based analysis.

15. Our Results2. Recruitment & Testing242 registered via website126 submitted enhancedSEA reports117 completed post-study survey

16. Participation and Completion Rates

17. Our Results3. Evaluation of Interventions

18. Evaluation(n=117)

19. Evaluation (n=117) *P<0.05

20. Comments related to the Personal Booklet • “The step by step guide helped me to carry out significant event analysis in a quick and comprehensive manner making it more likely I will carry out formal SEA in the future with more confidence and frequency”. • “I don’t think I looked at the cards. The booklet itself was very clear, breaking everything down nicely to help write the report. The enclosed example was also very helpful and I will certainly refer to the booklet again next time I complete a significant event analysis”. • “I think that this tool might be more useful for some SEAs than others – for example it might be useful in understanding emotions surrounding a serious event, but many SEAs are more minor or administrative in nature. I did however find it helpful as an aide-memoire to ensure that I had thought of all possible aspects contributing to the event”. • “I still found it difficult to analyse the situation and feelings but this did encourage me to be more objective and see why/how things happened a certain way”. • “I found it a bit complicated, not all was relevant”

21. Comments related to the A3 Desk Pad • “I think it is a persistent reminder of what eSEA is trying to achieve. I would not generally refer to it when doing my eSEA but having it on my desk continually reminds of the human factors approach and so was useful (similar to having a pen with a drug name on it!!)” • “The content on the desk pad sheet was useful for our meeting but we ended up taking notes on a separate sheet and we could really have just used the personal booklet for guidance”. • “Even in a meeting scenario-our staff are not around a table but just in our waiting room so big and bulky to use. Could do with more space for note taking if used. Found the smaller tools more appropriate in this setting/environment”. • “Quite a large thing to have and wasn’t really sure where to keep it-significant events are hopefully relatively infrequent and not sure I needed something so big that don’t use that often”. • “A cumbersome waste of space and paper”

22. Other Selected Quotes • “I think SEA is a difficult and demanding process but these tools simplified the process for me and made it easier to try to be more objective” • “I think the human factors model is a very useful approach to looking at an SEA and without doubt caused me to think differently about this SEA and feel the outcomes were more meaningful”. • “Very worthwhile. Definitely made the SEA feel more in depth / thorough. Whole team very approving. Developed some really useful action points” • “If eSEA was recognised, it would be more beneficial and more useful than existing SEA” • “We’ll look at things more closely, when an SEA comes in use, use of the eSEA will have greater impact, will influence initial discussions” • “eSEA breaks stuff down further that you wouldn’t have considered....which I wouldn’t have considered before” • “I found it made the process more laborious and confusing in some ways. I think the booklet was helpful but the report format needs to be simplified. Some of it felt like writing in order to fill in boxes. I think we would have come to the same conclusions if I had used our normal format”.

23. Early Feedback from SEA Peer ReviewersSubjective/Objective Comparisons – ‘Old’ vs ‘New’ • Acts as a ‘forcing function’ – deeper analysis • More straightforward to actually understand and peer assess • Feeling that more challenging events are being selected • Overall standard has improved compared with ‘old’ system • Potential for objective comparison (old vs new assessment ratings) • Understanding of why event occurred • Learning needs identified • Change and improvements • Overall global rating

24. Overall Impact of enhancedSEA • “Use of eSEA might improve things a bit. Events are broken down into so many things that it would never be down to one person as there are so many links in the chain” • “I think it was definitely the most thorough SEA that we’ve done for a while and everybody commented that it went very well and it had some very good outcomes for us all things that I really think will have a decent impact” • “I think it’ll get people to think more into just why significant event analysis happens and it’s like a big elephant in the room, of course you’re embarrassed and you will have these emotions and I think it encourages people to realise that those emotions are there, we’re not working as robots if you like. And it gets people to think more about being human” • “…it made you less stuck for sort of thinking about things that’d gone wrong and why there’s sort of different aspects had gone wrong because there was lots of problems there to get you going”.

25. What has gone well? • We made the project aims more realistic and manageable. • Our core group has stayed together and functioned well – migrating to broader Q&S organisational group • National Educational Leaders clearly interested, very supportive and helpful, also Executive buy-in • The approach developed based on solid theory and has high face validity (particularly amongst those informed in safety) • Post-pilot implementation looks promising, but may take a while to embed for some professions (e.g. optometry and psychology) • Infrastructure to improve and build upon – a lasting legacy • TROJAN HORSE STRATEGY SEEMS TO HAVE WORKED!

26. What could have been better/different? • Website design and access issues – bit rushed • Tool content and design – took longer than expected, not sure if we had a shared mental model of final output (but a Pilot) • Impacted slightly on testing period – knocked some out of sync. (e.g. Pharmacy) • Quality of design could be much better • Different interpretations of ‘human factors’ science and role in project/and also of evaluation expectations • Test numbers plucked out of thin air – over-confidence and naivety? • Email bombardment irritated a fair few GPs wearing different hats! • Thematic analysis of enhancedSEA reports only just started.

27. Project Learning and Outputs • Once evaluation feedback incorporated, enhancedSEA method ready for roll out – good interest from Educational Leaders/SPSP/Others • Conference Presentations • Basic e-learning (BMJ Learning, NES website) • Book Chapters (NES book and Good Practice GP Training Guide) • Journal Submissions (conceptual framework, evaluation…) • NES team learning (human factors science, psychological safety, logic models, error theory…) • Organisational impacts (e.g. inter-professional working, good publicity)

28. Positive External Interest • GP Specialty Training/Dental/Pharmacy/ • RCGP • Vocational Training for GPNs/PMs • SPSP-PC and Scottish Government • Department of Health in England/PS Toolkit Enquiries • Some NHS Boards – as part of local enhanced services

29. Thank You – Any Questions?www.nes.scot.nhs.uk/shine/P Bowie & C de Wet [eds.] ‘Safety and Improvement in Primary Care: The Essential Guide’ Radcliffe Publishing Ltd.