Teaching the Art of Literature Analysis

1. Teaching the Art of Literature Analysis Evan Williams, PharmD, BCPS Assistant Professor of Pharmacy Practice Husson University School of Pharmacy

2. Overview • Why literature evaluation is important and why people hate it • How to make literature less terrible for students and preceptors • What parts of the study are important and what is beyond essential

3. Pre Test Questions • Students should focus on the statistical methods used as it relates to internal validity of the findings more so than the external validity of the study when discussing journal articles. • True or False

4. Pre Test Questions • Journal Article Presentations should • Focus on an area of practice relevant to the practice site • Tie into some other project or problem at the practice site • Be flexible enough to fit into the amount of time a practice site or preceptor can devote to the assignment • All of the above

5. Pre Test Questions • The following study for gorillastatin produced the following results after 5 years of follow up. Based on the results in the table, about how many patients do I need to treat with gorillastatin for 5 years to prevent one CAD event? a) 5 b) 8 c) 12 d) 16

6. Objectives • Discuss reasons literature evaluation is an essential skill for all pharmacists • Identify components of a quality “journal club” format • Describe ways to make student literature review useful to your practice • Briefly distinguish between useful and confounding statistical methods and demonstrate ability to calculate NNT and NNH

7. Are your students this last guy?

8. Literature Evaluation is Important! • Essential skill for any clinician • Everyone wants to follow “evidence-based medicine” • How do you know if recommended treatment is evidence based? • Needed to keep up with emerging therapies • New drugs • New guidelines • If you don’t do this, the next generation of pharmacists will be followers instead of leaders regarding drug therapy

9. Why People Hate Literature • Dry, boring reading • 50 Shades of Primary Literature? • Time considerations • Perceived lack of utility • This is all summarized in Pharmacists’ Letter anyway • Fear of statistics • What the devil is the difference between a t-test, f-test, and a z-test, anyway?

10. Why Students Hate “Journal Club” • There is nothing “clubby” about it • We force them to present some random article and belittle them for not knowing stuff we never taught them • It is detached from other learning outcomes • “Pick a recent article that has something to do with your internal medicine rotation.” • They pick terrible articles because of this • They lack good examples of what we are expecting

11. How to Make This Better • Decide what the desired learning outcome is • Presentation skills? • A refined, grammatically perfect handout? • Identify all of the trial’s strengths and weaknesses? • Distinguish important clinical implications of the trial presented? • Once the outcome is identified, make sure this is communicated to the student

12. How to Make Lit Eval Better • Selection of Articles • Student vs. Preceptor Selection • Ensure that the article relates to something happening on rotation! • This will make it relevant anduseful • Type of Article • Original Research • Review Article • Guideline Evaluation

13. How to Make Lit Eval Better • Selection of Articles • Good vs. Bad Articles • Clinical vs. Surrogate Outcomes • Sketchy statistics • Primary outcomes vs. Secondary outcomes • New vs. Old • Older articles aren’t bad or irrelevant!! • If the student doesn’t know it, it is a learning opportunity

14. How to Make Lit Eval Better • Identify what you’d like the student to present • Background • Do you need this? • Needs to be quick • Methods • Brief • These are fairly standard • Breeze through the stats

15. How to Make Lit Eval Better • Identify what you’d like the student to present • Results • Best presented in a table format • Identify primary vs secondary outcomes • Distinguish between clinical and statistical significance • Does power matter? • Calculate NNT and NNH

16. How to Make Lit Eval Better • Identify what you’d like the student to present • Discussion should take the most time • Is the outcome valid based on the test used? • Confounders • Are they looking at the right groups of patients? • Influence of uncontrolled variables? • Statistical vs. clinical significance? • Does this change my practice? • Conclusions • Do you agree with the author’s conclusion?

17. How to Make Lit Eval Better • Presentation Setting • Big time presentation and roast? • Conversational? • Panel Discussion? • CE style? • Do what makes sense for your practice site! • Make this useful to you

18. Other Points For Improvement • Give students enough time • Check in to make sure progress is being made • Suggest addressing questions prior to the presentation to improve the quality of the presentation • Read the article! • Show the student a clear example of a good handout to follow • Many are available, so pick one that you like • Students tend to do better with bullet points • Make them create their own tables

19. What You Need to Know • In order to properly evaluate the student’s critique, you need to have identified the key points that should have pointed out • What are the results? • Are the results valid? • How can I use the results in my practice?

20. What Are the Results? • Any statistically significant differences? • p-value vs confidence intervals • Little p-values don’t make results more significant • Confidence intervals can give an impression of how precise the results are • Confidence intervals that cross unity are not significant • Clinical significance?

21. What Are the Results? • Number Needed to Treat • Good way to quantify clinical significance • 1/difference in the event rate between groups • 25 of 500 people in the placebo group died - (5%) • 10 of 550 people in the drug group died – (1.8%) • The difference between the groups is 3.2%, or 0.032 • 1/0.032 = 31.25 ~ 31 • If I treat 31 people with the drug, I will prevent one death • Compare this to the Number Needed to Harm

22. What Are the Results? • Number Needed to Harm • Same calculation, but for adverse events • 10 of 500 people had major bleeding in the placebo group – (2%) • 22 of 550 people had major bleeding in the drug group – (4%) • The difference between the groups is 2%, or 0.02 • 1/0.02 = 50 • If I give 50 people the drug, I will cause 1 major bleed • Does the risk vs benefit ratio seem reasonable?

23. Are the Results Valid? • Was the primary endpoint clearly stated and reported? • Was it relevant? • Surrogate vs clinical endpoints • Composite endpoints • Was the designed to reveal the primary endpoint? • Powered enough to detect a difference if one wasn’t found? • Does the discussion focus on subgroup analyses?

24. Are the Results Valid? • Where the right people included? • Intent to Treat (ITT) or Per Protocol Analysis? • Inclusion and exclusion criteria • Do they control for confounders? • Do they limit external validity? • Was bias effectively dealt with? • Blinding • Valid control groups • Placebo vs active control?

25. How Can I Apply the Results? • NNT vs NNH • Are the patients studied similar to my patients? • How does this evidence jive with other studies regarding the same subject? • Is this study important enough to shift clinical practice?

26. Make This Useful to You • New Drugs • What studies got this approved? • Who was looked at? • Side effect profiles? • New Guidelines • Check the references • Things that have strong/low evidence • Things you don’t know/always get questions about • When is aspirin indicated? • Evidence that vaccines don’t cause autism? • Does LDL really correlate with MI? • Why do we still use warfarin?

27. Summary • Pharmacists need to be able to evaluate literature • We should be showing students why this is important • Make sure journal article presentations focus on preparing pharmacists that can critically evaluate literature • Presentation style, format, handouts, length can all vary and should be adapted to the practice site • Focus on the key issues within the studies presented • Don’t get bogged down in statistics • Consider the clinical impact • Make the exercise beneficial for you and the student

28. Post Test Questions • Students should focus on the statistical methods used as it relates to internal validity of the findings more so than the external validity of the study when discussing journal articles. • True or False

29. Post Test Questions • Journal Article Presentations should • Focus on an area of practice relevant to the practice site • Tie into some other project or problem at the practice site • Be flexible enough to fit into the amount of time a practice site or preceptor can devote to the assignment • All of the above

30. Post Test Questions • The following study for gorillastatin produced the following results after 5 years of follow up. Based on the results in the table, about how many patients do I need to treat with gorillastatin for 5 years to prevent one CAD event? a) 5 b) 8 c) 12 d) 16

31. Questions? Thank you!