Bringing EBM into EM Residency: Tricks, Tools, Trade Secrets

1. Bringing EBM into EM Residency:Tricks, Tools, & Trade Secrets Lee Wilbur, MD Bryan Kane, MD Merle Carter, MD Charles Graffeo, MD Raymond Lucas, MD Christopher Carpenter, MD

2. Charlie Graffeo Prof. and Asst. PD Eastern Virginia Medical School Norfolk, Virginia

3. Today�s objectives Describe successful EM GME EBM curricula Discuss faculty development ideas to engender the growth of EBM role models Discuss EBM strategies that satisfy RRC practice-based learning requirements Provide participants with a toolbox to develop sustainable EBM curricula locally Highlight current EM GME EBM-related research opportunities �in other words

4. Objectives What comprises a successful EBM curriculum? What practices can truly translate into lifelong EB knowledge acquisition? What do residents really want? Faculty development issues Collaborative research

5. �How to teach EBM� McMaster University 2001 McMaster, Eddy Lang, P Wyer, Great group of participants, EM Journal Club, Description of the group, Question formulation, JC activities. McMaster, Eddy Lang, P Wyer, Great group of participants, EM Journal Club, Description of the group, Question formulation, JC activities.

6. Post McMaster� Immediate integration into JC Used critical appraisals from EBM course Orientation lectures on EBM JC Search strategy EBM interest group

7. Successes Resident driven clinically relevant PICO�s CATS & the clinical bottom line Critical Appraisals & Clinical Epidemiology Private Practice CQI Committee buy-in Shift in didactic format CAT Archive / Website development High levels of enthusiasm for JC

8. Ha!Ha!�keep yours in Ha!Ha!�keep yours in

11. Challenges Faculty development in EBM expertise Community based EM program Use of EBM tools in real time Discordant practices by community faculty Pharma II

12. Exciting timesU.S. Healthcare 2.0 EBM, GME Training & Healthcare Policy Comparative Effectiveness Research Accountable Care Model The EMR An EM Checklist Manifesto?

16. Housekeeping Issues Contents of package Please fill out your survey Brief presentations Forward questions Your own experiences Sign up sheet for JC research proposals Would like to think we have a room full of experts and hope to broaden the discussion to everyone here today. Handout/package contents. <20 question survey.Would like to think we have a room full of experts and hope to broaden the discussion to everyone here today. Handout/package contents. <20 question survey.

17. Ray Lucas

18. Teaching EBM Model teaching EBM the same way we teach other skills and content knowledge Group learning and individual learning Shared responsibility between teachers and learners Didactic learning should provide a model for clinical practice Regurgitated textbook chapters are a waste of time We have a 2 year content curriculum based on the clinical model taught in a variety of formats We try to model classroom learning to be similar to clinical practice Example � non EM physicians do not give lectures � they come as invited experts who supplement with their expertise � much in the same we use consultants in the ED. Not to take over a case, but to add expertise we may not have Learning EBM should be �in synch� with how we practice.We have a 2 year content curriculum based on the clinical model taught in a variety of formats We try to model classroom learning to be similar to clinical practice Example � non EM physicians do not give lectures � they come as invited experts who supplement with their expertise � much in the same we use consultants in the ED. Not to take over a case, but to add expertise we may not have Learning EBM should be �in synch� with how we practice.

19. Teaching EBM

20. EBM Workshop Intern orientation Taught with reference librarian Emphasis of workshop How to formulate a question Search strategies On-the-fly learning to augment patient care

21. Journal Club Monthly, led by senior resident and a faculty member Structured review of 1-2 articles with worksheets Emphasis Critical appraisal skills Basic statistics

22. EM Study group Individualized learning and practice in EBM Serves as double duty for core content delivery Emphasis �Putting it together� Forming a question, searching for answers Evaluating the information Establishing a �best answer� based on the available knowledge

23. EBM � study group Led by a PGY-2 or PGY-3 resident The resident is given 3-5 clinical questions around the topic to be covered They are charged with doing an evidence based investigation to find the best available answers to the question Proctored by a faculty member ( usually me)

24. Preparing Each year, a group of faculty and chief residents write the questions, following the monthly curriculum. Archives of past years are reviewed The resident prepares in advance � meets with a faculty preceptor at least 2 weeks in advance of their presentation

25. Example - Asthma �How useful is the PEFR � really?� �When should you prescribe a steroid inhaler?� �Does heliox really make a difference in severe asthma?� Is Bipap appropriate in severe asthma and does it change outcome?�

26. Example - Pericarditis �How common is the classic EKG findings of ST elevation in all leads?� Which patients with pericarditis should be admitted?� If using colchicine to treat, do you still give the patient NSAIDS?�

27. Advantages Makes a more interesting conference � which keeps residents AND FACULTY interested Makes the resident assess the available information because there is not a randomized control trial for everything. Promotes the notion that book reading is important because we are not going to cover everything in conference Individualized approach ensures each resident has the opportunity to be an active participant and have their EBM skills assessed and improved by a faculty member

28. Disadvantanges Not every core topic lends itself to a good EBM review Takes a good working knowledge of the literature to write good questions Takes faculty time on an ongoing basis

29.  Multi-layered Approach to EBM Integration into Residency Training

30. Integrating EBM in to Residency Early Goals Increase EBM as a content area in the curriculum Single-content area Blended into pre-existing lecture series Increase use of evidence-based medicine in the ED Provide opportunities for residents to lead evidence-driven seminars

31. Integrating EBM in to Residency Challenges & Barriers Experience & knowledge gaps of the residents & faculty How to incorporate an EBM series in an already bulging conference schedule? No identified faculty champion 1. Residents come with varied backgrounds and familiarity with research topics, comprehension of research principles & vocabulary, etc� created difficulty with 1. Residents come with varied backgrounds and familiarity with research topics, comprehension of research principles & vocabulary, etc� created difficulty with

32. Addressing Challenges Multi-layered Approach: Faculty Development Research Curriculum Journal Club Landmark, abstracts, theme-based, current Evidence-based Medicine Conference 1. Residents come with varied backgrounds and familiarity with research topics, comprehension of research principles & vocabulary, etc� created difficulty with 1. Residents come with varied backgrounds and familiarity with research topics, comprehension of research principles & vocabulary, etc� created difficulty with

33. Evidence-Based MedicineConference for Seniors PGY 3 and 4 Resident-run Faculty-precepted Seminar format Twice a month (1hr each) Two clinical questions covered during each conference On-going file kept of resident presentations

34. Setting the Groundwork for the Year Review principles of EBM Formulating searchable questions Searching the literature Critical appraisal of the literature Incorporating EBM in clinical practice Clinical controversies discussed & topic goals set for the year Faculty and resident assignments distributed Packets are distributed to both residents & faculty Goals & objectives of the conference EBM, library, critical appraisal, statistics, & on-line resources �How to prepare for your presentation� instructions

35. Evidence-Based Medicine Series:Resident Responsibilities Topic selection: Evidence may change one�s practice Controversial topic in the literature with evidence supporting alternate points of view Areas where faculty or EM & other specialties may disagree Literature review & written critical appraisal Handouts for presentation Seminar discussant Leads group discussion of the clinical question presented Discusses available evidence & evaluates its quality Summarizes findings in context of patient values, cost effectiveness, local experience, �best practices�, etc.

36. Evidence-Based Medicine Series:Faculty Responsibilities Initial review of topic to ensure suitability for seminar Clinically relevant to the practice of EM ? Sufficient literature available on the topic ? Reviews resident�s appraisal and handouts prior to presentation Ensures thorough review was undertaken Ensures accurate understanding of literature & analyses Serves as seminar preceptor Highlights core concepts Advances research design and statistical principles

37. Eliminating the Early Barriers Knowledge gaps & variability in experience of the residents & faculty Faculty Development, Research Lecture Series, Journal Clubs How to incorporate an EBM series in an already bulging conference schedule? Two-track template for conference No identified faculty champion All faculty serve as EBM preceptors Precepting EBM conferences are additional faculty development toward a team of champions Process yields a publishable work

38. Lee WilburIndiana University _____________________________________ �If physicians would read two articles a day out of the six million published annually, they would fall 82 centuries behind in their reading� Miser WF; Critical Appraisal of the Literature. J Am Board of Fam Pract (12)4:315-333, 1999.

39. EBM skill acquisition� Didactic lectures Journal club MS1 EBM course CATs Knowledge Translation shifts

41. Create the culture� Chairman supported (exempt from CME) Since 2002 10 McMasters 7 NYAM 20 have taught MS1 EBM course Rawle (Tony) Seupaul, MD Statewide Director for EBM and Biostatistics

42. Didactic lectures Core curriculum repeats every 2 years Resident must review with PD EBM specific: Orientation block 2 EBM lectures per academic year 1 R1 year-directed conference on EBM

43. Journal Club 4th Thursday of every block Began 1987, EBM approach 1995 General topics pre-determined 1 yr ahead 3 � 4 faculty, 3-4 residents assigned to each JC Individual articles selected by above group

44. Journal Club - sessions Clinical scenario presented to group Residents/ faculty teach EBM mini-lesson Critical appraisal forms used Then, re-convene for discussion Merge with nursing JC in 2009

45. MS1 EBM course Required freshman EBM Course Tutor Trainee � R3 Immersion via small sessions Scholarly activity completion Annually: 17 � 20 groups (EM faculty/ residents) 150 students Indianapolis/ 350 statewide

46. CAT�s Residents required to complete CAT�s 1 � 2 per academic year Following EM block What�s required: Clinical question (PICO) format Search criteria Appraisal of the evidence How evidence influenced PICO question

47. KT Shift Completed in ED Began 2008 PGY-2 ( 4 � 5/ year), 9 hr duration Real time EBM/KT KT sheets submitted for 6mo evals Novel CORD SAEM IEME

48. �Daily Practice Initiative� achieved through� Didactic lectures Journal club MS1 EBM course CATs Knowledge Translation shifts

49. Chris CarpenterA Sustainable EBM-Based Journal Club Format

50. Conflicts of Interest Chris Carpenter is Co-Chair of Best Evidence in Emergency Medicine Senior Editor of PEPID Chief Clinical Editor of Emergency Physicians Monthly Associate Editor of Academic Emergency Medicine Faculty member BEEM, EP Monthly Chief Clinical Editor, Senior EM Editor PepidFaculty member BEEM, EP Monthly Chief Clinical Editor, Senior EM Editor Pepid

52. EBM Training Survey Barriers Lack of trained faculty Lack of time Lack of familiarity with EBM resources Insufficient funding Lack of interested faculty Survey mailed to program directors of all 122 accredited EM residency programs (53% response rate). 80% reported teaching some EBM, though only 22% provided more than 5 hours per year. 63% reported using JAMA Users Guide Series in Journal Club and 83% reported efforts to link Journal Club to patient care.Survey mailed to program directors of all 122 accredited EM residency programs (53% response rate). 80% reported teaching some EBM, though only 22% provided more than 5 hours per year. 63% reported using JAMA Users Guide Series in Journal Club and 83% reported efforts to link Journal Club to patient care.

53. Journal Club Frustrations Lack of focus Insufficient preparation time No standardized approach Feedback often lacking No clear conclusions Discussion forgotten after Journal Club Lifelong learning skills neglected

54. Journal Club IdeasThe Wash U Experience Ask a single, answerable question based around a realistic EM vignette Discuss search strategy Distribute articles 3-4 weeks in advance Use standardized critical appraisal forms Routinely invite content-experts

55. Journal Club IdeasContinued Provide hands-on approach to statistical analysis each month Provide private feedback to each resident�s critical appraisal form Reward participation Maintain Journal Club conclusions on website for later retrieval and real-time use I�m from the show-me state so let me give you an example of our JC format.I�m from the show-me state so let me give you an example of our JC format.

56. PICO Question Patient population Intervention/Finding Control Outcome Non-newborn children presenting with ED clinical concern for bacterial meningitis CSF pleocytosis, protein elevation and/or glucose depression Normal CSF Bacterial meningitis, Aseptic meningitis, mortality Intervention in PICO can also be interpreted as diagnostic finding (for a diagnostic study) or exposure (for a prognostic study).Intervention in PICO can also be interpreted as diagnostic finding (for a diagnostic study) or exposure (for a prognostic study).

57. Search Strategy � do a search on OVID, but yield an incomprehensibly long list of over 2000 references so you decide to search the term meningitis through the various EM journals and obtain the Academic EM reference below. Finally, you scour the bibliography of this Academic EM selection to find the Nigrovic article and perform a Web of Science review of the Nigrovic article to obtain the Bonsu article. How to find the evidence is just as important as how to analyze and utilize what you find.How to find the evidence is just as important as how to analyze and utilize what you find.

58. Critical Appraisal Forms 1st Year Residents use the Clinical Practice Guidelines Critical Review Form 2nd Year Residents use the Prognosis Critical Review Form 3rd Year Residents use the Clinical Decision Rule Critical Review Form 4th Year Residents use the Therapy Critical Review Form

59. RRC Practice-Based Learning Core Competency? How does your program demonstrate the PBL core competency for the RRC? Whether or not residents can physically attend any given evening�s Journal Club, they are asked to turn in their individual critical appraisal forms for anonymous, individualized �grading� by that month�s Journal Club preceptor. They have the vignette, articles, and critical appraisal form for 3 weeks before the Journal Club & have up to one week after the JC evening to turn it in.How does your program demonstrate the PBL core competency for the RRC? Whether or not residents can physically attend any given evening�s Journal Club, they are asked to turn in their individual critical appraisal forms for anonymous, individualized �grading� by that month�s Journal Club preceptor. They have the vignette, articles, and critical appraisal form for 3 weeks before the Journal Club & have up to one week after the JC evening to turn it in.

60. Journal Club DiscussionKeys to Facilitating Participation Adequate preparation time (for participants) Adequate preparation time (by facilitator) Encouraging organized presentation Optimal time for presentation Emphasis on important factoids, limitations, and clinical bottom line Provocative questions or ancillary literature

61. Reward Participation Attendance -1 point Verbal Participation Actively Led Discussion � 3 points Said Something � 2 points Was silent as a mouse � 1 point Not present � 0 points Written Summary Exceptional � 3 points Average � 2 points Sub-par � 1 point Not turned in � 0 points

66. Textbooks You�ve got the Users� Guide to the Medical Literature in PDF format on the CD �Journal Club Toolbox� you were provided today. Users� Guide = $53.51, Teaching EBM = $46, EM Decision Making = $36.95.You�ve got the Users� Guide to the Medical Literature in PDF format on the CD �Journal Club Toolbox� you were provided today. Users� Guide = $53.51, Teaching EBM = $46, EM Decision Making = $36.95.

69. EBEM website organized as per Core Model.EBEM website organized as per Core Model.

70. We solicit author updates to SRAs and EBEM Reviews when indicated.We solicit author updates to SRAs and EBEM Reviews when indicated.

71. �BEEM� Me the Evidence PleaseMay 2010 BEEM Raters receive twice-monthly e-mail of 15-20 potentially high-impact EM-relevant abstracts The highest rated article each month will be critically appraised and e-mailed to all BEEM Raters Free! BEEM has over 35 secondary peer-reviewed publications. Additionally, BEEM has had a dozen CME courses in 5 countries with over 500 participants. BEEM offers the spectrum of teaching participants detailed critical appraisal teaching skills to simply becoming aware of the best-evidence available today for your practice-setting.BEEM has over 35 secondary peer-reviewed publications. Additionally, BEEM has had a dozen CME courses in 5 countries with over 500 participants. BEEM offers the spectrum of teaching participants detailed critical appraisal teaching skills to simply becoming aware of the best-evidence available today for your practice-setting.

73. Can You Think of Anything You Can Teach Which Will Remain True and Unchanged When Your Students Retire?

74. Bryan KaneLehigh Valley

75. Drinking From a Fire Hose Estimated that 627.5 hours need per month for a clinician to keep up with the literature (in primary care) And our residents are trying to both learn the old and keep up with the new

76. An Intervention Metric Intern class cohort and a convenience sample of residents Not getting it in medical school Schwart BMJ 2003 Three or four 2 hour sessions Recognition Searching Measures of difference Review***

77. It Worked, But Can We Use It?

79. The Fresno Test Write a PICO question Sources of information Match methods to question Search strategies Article relevance Article validity Article impact: Magnitude and significance

80. The Fresno Test: �Stats� Complete a 2x2 grid Sensitivity, Specificity PPV, NPV Odds ratio Other stats Absolute risk, Relative risk reduction NNT 95% CI Identify methodology for: Diagnosis Prognosis

81. Scoring Matrix Objective scoring matrix Open ended questions (Weighted) 1-7 PICO (12) Multiple descriptors, detailed/specific, POEM (3) Single descriptor, specific, DOE (2) Vague, non-searchable (1) None (0) Closed questions 8-12

82. Grading Metric

83. Teaching EBM -> Phase 3 Data 35 attendings and 12 residents Syllabus, text and 7 hours of lecture 262 consecutive pre, 275 post EBM �best Rx� from 74 to 82% (p=.046) Use of RCT Rx from 49 to 62% (p=.016) Strength of evidence improved as well Cardiac specific 87 to 98% (p=.02)

85. Tracking Behavior Motivated library services Accessed digital library Who, what (source), when Which can be tied to� Specialty PGY (and faculty) Lectures Journal Club On shift �PBL� recording EMR links

86. Bryan.Kane@lvh.com

87. In Summary Designate a faculty EBM champion Invest in faculty-wide EBM development Develop an EBM Toolbox (BEEM) Introduce early in the residency curriculum Provide useful feedback to learners Structure didactics around best available evidence Further EBM research� Complete your survey Sign up to hear more about JC research protocols

88. Contact Information lwilbur@iupui.edu carterm@einstein.edu carpenterc@wustl.edu cgraffeo@gmail.com rlucas@mfa.gwu.edu Bryan.kane@lvh.com

89. Questions?