EBM & evaluating and grading evidence

1. EBM & evaluating and grading evidence Phil Wiffen UK & Chinese Cochrane Centre

2. The rationale • Evidence based medicine as a model • What is evidence ? • What are systematic reviews ? • Tools to present data

3. League table of NNTs to produce at least 50% pain relief over 4-6 hours compared to placebo in pain of moderate or severe intensity

4. What evidence-based medicine is: Evidence-based medicine is the conscientious, explicit and judicious use of current best evidence in making decisions about the care of individual patients. Sackett (BMJ 1996; 312: 71-2)

5. What evidence-based medicine is: • The practice of EBM requires the integration of • individual clinical expertise • with the • best available external clinical evidence from systematic research.

6. Another definition of EBM Evidence based medicine is an approach to health care that promotes the collection, interpretation and integration of valid, important and applicable patient reported, clinician observed and research derived evidence. The best available evidence, moderated by patient circumstances and preferences, is applied to improve the quality of clinical judgements. McKibbon KA et al ‘The medical literature as a resource for Evidence Based Care’ http://hiru.mcmaster.ca/hiru/medline/mdl-ebc.htm

7. How effective are the medicines we use? Do more harm than good Unknown effect and no research Do more good than harm Unknown effect and research in progress

8. “There are perhaps 30000 biomedical journals in the world, and they have grown steadily by 7% a year since the seventeenth century. Yet about 15% of medical interventions are supported by solid scientific evidence... ...only 1% of the articles in medical journals are scientifically sound” R. Smith quoting Prof. D. Eddy, BMJ 1991; 303: 798-99 “...approximately 17000 new biomedical books are published annually.” Lowe and Barnett, JAMA 1994; 271: 1103-8 More than 20 000 RCTs have been published in pain relief research since 1950

9. The size of the task • How many biomedical papers are there ? • Medline 12 million records, 4600 journals, 30 languages • Embase 9 million records, 4000 journals, 70 countries • CINAHL 0.87 million records 1600 journals. 13 languages • Others: ???? Nov.2003

10. UNBIASED ‘good’ RCT numbers / inclusion / exclusion blinding power enthusiast + systematic reviews open ‘biased’ study local use ‘expert’ clinical practice

11. Bias in clinical trials Over-estimation of treatment effect Not random 40% Not double-blind 17% Duplicate information 20% Small trials 30% Poor reporting quality 25%

12. Randomisation Non randomised studies increase treatment effect by 41% Unclear randomisation by 30% Schulz, Chalmers et al JAMA 1995 273 408-12

13. Blinding of Patient and/or Observer if not double-blind: overestimation 17%Schulz et al, JAMA 1995; 273:408-12 is this important in every setting? (unconscious) practicable in every setting? (surgery)

14. Effect of duplicate publication From Tramèr et al BMJ 1997;315 635-40

15. Tools not Rules

16. Type & Strength of Evidence I Strong evidence from at least 1 systematic review of multiple well-designed randomised controlled trials II Strong evidence from at least 1 properly designed randomised controlled trial of appropriate size III Evidence from well designed trials without randomisation, single group pre-post, cohort, time series or matched case-controlled studies IV Evidence from well-designed non experimental studies from more than 1 centre or research group V Opinions of respected authorities, based on clinical evidence, descriptive studies or reports of expert committees

17. Information Making and gathering information Single papers RCTs Knowledge Distillation Integration Quality Systematic reviews meta-analyses Wisdom Experience Values Conditions EB place in clinical practice Evidence-based Practice and Policy

18. What is a systematic review ? • Filing Cabinets • Friends • Foreigners ? • The world literature on a subject

19. Systematic Reviews “Clinical review articles should be as scientific as the articles they review” Haynes, BMJ 1992; 304: 330-1 “ The fundamental difference between a review and a primary study is the unit of analysis, not the scientific principles that apply” Oxman & Guyatt, CMAJ 1988; 138: 697-703

20. Tools to present data Meta-analysis NNTs L'Abbé plots

21. Ibuprofen 400 mg vs. paracetamol 1000 mg for acute postoperative pain favours paracetamol favours ibuprofen no difference 7 Cooper et al, 1984 6 Cooper, 1984 5 Cooper et al, 1989 difference between the mean effects within the trial and 95% CI 4 Schachtel al, 1989 3 Mehlisch et al, 1990 2 Overall weighted difference 1 0 -20 -10 0 10 20 30 mean differences and 95% CI ( % of the maximum possible TOTPAR value)

22. McCrory et al Cochrane Library Issue 4 2003

23. Numbers needed to treat (NNTs) The Number of people who have to be treated for ONE to benefit

24. Number-needed-to-treat (NNT) • Controls • Ncon • Impcon • Actives • Nact • Impact Number of patients Improved = Clinical end point 1 NNT = Impact Impcon - Nact Ncon

25. 1 NNT = 100 0 - 100 100 Number-needed-to-treat (NNT) • NNT is treatment specific-takes into account the • event rate in controls: • may be a placebo effect • may be the effect of another treatment

26. active control • improved 80 20 • N 100 100 Relative Risk (RR) = (Impact/Nact) / (Impcon/Ncon) Relative Risk Reduction (RRR) = (1-RR) / 100 Absolute Risk (AR) = (Impact/Nact) - (Impcon/Ncon) Number Needed to Treat (NNT) = 1/AR RR = 4; AR = 0.6; NNT = 1.7 (best 1.25)

27. L'Abbé plot for treatment 100 Treatment better than control equality Proportion improved with treatment 75 50 Control better than treatment 25 0 0 25 50 75 100 Proportion improved with control

28. 100 90 200 80 100 70 0 60 50 40 600/650 30 500 20 1000 10 0 0 10 20 30 40 50 60 70 80 90 100 World literature on paracetamol At least 50% pain relief with paracetamol At least 50% pain relief with placebo Moore et al Pain 1997;70:193