GRADE From Evidence to Recommendations

1. Holger Schünemann, MD, PhD Professor Utrecht, NL September 18 - 19, 2008 GRADE From Evidence to Recommendations

2. Evidence based clinical decisions Clinical state and circumstances Patient values and preferences Expertise Research evidence Equal for all Haynes et al. 2002

3. The GRADE approach Separation of 2 issues: 1) 4 categories of quality of evidence: very low, low, moderate, or high quality? • methodological quality of evidence • likelihood of bias • by outcome and across outcomes 2) Recommendation: 2 grades - weak or strong (for or against)? • Quality of evidence only one factor www.GradeWorkingGroup.org

4. Grades of recommendation:Strength of recommendations Strong recommendations • high quality methods with large precise effect • benefits much greater than downsides, or downsides much greater than benefits • we recommend • Grade 1 Weak/conditional recommendations • lower quality methods • benefits not clearly greater or smaller than downsides • values and preferences uncertain or very variable • we suggest • Grade 2

5. Case scenario A 13 year old girl who lives in rural Indonesia presented with flu symptoms and developed severe respiratory distress over the course of the last 2 days. She required intubation. The history reveals that she shares her living quarters with her parents and her three siblings. At night the family’s chicken stock shares this room too and several chicken had died unexpectedly a few days before the girl fell sick.

6. Relevant clinical question?Example from a not so common disease Clinical question: Population: Avian Flu/influenza A (H5N1) patients Intervention: Oseltamivir (or Zanamivir) Comparison: No pharmacological intervention Outcomes: Mortality, hospitalizations, resource use, adverse outcomes, antimicrobial resistance Schunemann et al. The Lancet ID, 2007

7. Methods – WHO Rapid Advice Guidelines for management of Avian Flu • Applied findings of a recent systematic evaluation of guideline development for WHO/ACHR • Group composition (including panel of 13 voting members): • clinicians who treated influenza A(H5N1) patients • infectious disease experts • basic scientists • public health officers • methodologists • Independent scientific reviewers: • Identified systematic reviews, recent RCTs, case series, animal studies related to H5N1 infection

8. Evidence Profile • Oseltamivir for treatment of H5N1 infection: - -

9. Oseltamivir for Girl with Avian Flu Summary of findings: • No clinical trial of oseltamivir for treatment of H5N1 patients. • 4 systematic reviews and health technology assessments (HTA) reporting on 5 studies of oseltamivir in seasonal influenza. • Hospitalization: OR 0.22 (0.02 – 2.16) • Pneumonia: OR 0.15 (0.03 - 0.69) • 3 published case series. • Many in vitro and animal studies. • No alternative that is more promising at present. • Cost: ~ Euro 40$ per treatment course

10. Determinants of the strength of recommendation

11. Example: Oseltamivir for Avian Flu Recommendation: In patients with confirmed or strongly suspected infection with avian influenza A (H5N1) virus, clinicians should administer oseltamivir treatment as soon as possible (????? recommendation, very low quality evidence). Schunemann et al. The Lancet ID, 2007

12. Example: Oseltamivir for Avian Flu Recommendation: In patients with confirmed or strongly suspected infection with avian influenza A (H5N1) virus, clinicians should administer oseltamivir treatment as soon as possible (strong recommendation, very low quality evidence). • Values and Preferences • Remarks: This recommendation places a high value on the prevention of death in an illness with a high case fatality. It places relatively low values on adverse reactions, the development of resistance and costs of treatment. Schunemann et al. The Lancet ID, 2007

13. Other explanations Remarks: Despite the lack of controlled treatment data for H5N1, this is a strong recommendation, in part, because there is a lack of known effective alternative pharmacological interventions at this time. The panel voted on whether this recommendation should be strong or weak and there was one abstention and one dissenting vote.

14. ACCP: Acute coronary syndrome Would a recommendation ignoring V & P be possible? For all patients presenting with NSTE ACS, without a clear allergy to aspirin, we recommend immediate aspirin, 75 to 325 mg po, and then daily, 75 to 162 mg po (strong recommendation, high quality evidence).

15. Value sensitive recommendation • Idiopathic deep venous thrombosis (DVT) is potentially life threatening condition • Patients usually receive blood thinners for one year • Continuing therapy will reduce a patients absolute risk for recurrent DVT by 7% per year for several years • The burdens include: • taking a blood thinner daily • keeping dietary intake of vitamin K constant • blood tests to monitor the intensity of anticoagulation • living with increased risk of minor and major bleeding

16. Value sensitive recommendations → Patients who are very averse to a recurrent DVT would consider the benefits of avoiding DVT worth the downsides of taking warfarin. Other patients are likely to consider the benefit not worth the harms and burden. For patients with idiopathic DVT, without elevated bleeding risk, we suggest long term warfarin therapy (weak recommendation, high quality evidence). Values and preferences: this recommendation ascribes a low value to bleeding complications and burden from therapy and a high value to avoiding DVTs

17. Quality assessment criteria

18. Strength of recommendation • “The strength of a recommendation reflects the extent to which we can, across the range of patients for whom the recommendations are intended, be confident that desirable effects of a management strategy outweigh undesirable effects.” • Strong or weak

19. Quality of evidence & strength of recommendation • Linked but no automatism • Other factors beyond the quality of evidence influence our confidence that adherence to a recommendation causes more benefit than harm • Systems/approaches failed to make this explicit • GRADE separates quality of evidence from strength of recommendation

20. Implications of a strong recommendation • Patients: Most people in this situation would want the recommended course of action and only a small proportion would not • Clinicians: Most patients should receive the recommended course of action • Policy makers: The recommendation can be adapted as a policy in most situations

21. Implications of a weak recommendation • Patients: The majority of people in this situation would want the recommended course of action, but many would not • Clinicians: Be prepared to help patients to make a decision that is consistent with their own values/decision aids and shared decision making • Policy makers: There is a need for substantial debate and involvement of stakeholders

22. Respiratory disease guidelines ?

23. Factors determining strength of recommendation

24. Values & Preferences • Patients’ perspectives, beliefs, expectations, and goals for health and life. • Underlying processes used in considering the benefits, harms, costs, and inconveniences patients will experience with each management option and the resulting preferences for each option.

25. Relative importance of outcomes and management approaches • Guideline panels should be explicit about the relative value they place on the range of relevant patient-important outcomes. If values and preferences vary widely, a strong recommendation becomes less likely • Example: Patients vary widely in their view of how aversive they find the risk of a stroke versus the risk of a gastrointestinal bleed when deciding about oral anticoagulation for atrial fibrillation.

26. Desirable and undesirable effects • desirable effects • Mortality • improvement in quality of life, fewer hospitalizations • reduction in the burden of treatment • reduced resource expenditure • undesirable consequences • deleterious impact on morbidity, mortality or quality of life, increased resource expenditure

27. Conclusion • clinicians, policy makers need summaries • quality of evidence • strength of recommendations • explicit rules • transparent, informative • GRADE • four categories of quality of evidence • two grades for strength of recommendations • transparent, systematic by and across outcomes • applicable to diagnosis • wide adoption

29. Design and Execution • limitations • lack of concealment • intention to treat principle violated • inadequate blinding • loss to follow-up • early stopping for benefit • Example: RCT suggests that danaparoid sodium is of benefit in treating HIT complicated by thrombosis • Key outcome: clinicians’ assessment of when the thromboembolism had resolved • Not blinded – subjective judgement

30. Consistency of results • consistency of results • if inconsistency, look for explanation • patients, intervention, outcome, methods • unexplained inconsistency downgrade quality • Bleeding in thrombosis-prophylaxed hospitalized patients • seven RCTs • 4 lower, 3 higher risk

31. Example: Bleeding in the hospital Dentali et al. Ann Int Med, 2007

32. Directness of Evidence • indirect comparisons • interested in A versus B • have A versus C and B versus C • formoterol versus salmeterol versus tiotropium • differences in • patients (mild versus severe COPD) • interventions (all inhaled steroids) • outcomes (long-term health-related quality of life, short –term functional capacity, laboratory exercise, spirometry)

33. Reporting Bias & Imprecision • reporting bias • reporting of studies • publication bias • number of small studies • reporting of outcomes • small sample size • small number of events • wide confidence intervals • uncertainty about magnitude of effect

34. Example: Bleeding in the hospital Dentali et al. Ann Int Med, 2007

35. What can raise quality? • large magnitude can upgrade (RRR 50%) • very large two levels (RRR 80%) • common criteria • everyone used to do badly • almost everyone does well • Oral anticoagulation for mechanical heart valves • dose response relation (higher INR – increased bleeding)

36. Questions for you • Are systematic reviews for every recommendation in your guidelines a reality/possibility? • What about cost – how do you deal with cost and how should we deal with it?

37. Content • Study design – bias • Levels/quality of evidence - GRADE • Guidelines/Recommendations

38. Content • Study design – bias • Levels/quality of evidence - GRADE • Guidelines/Recommendations

39. Confidence in evidence • There always is evidence • “When there is a question there is evidence” • Better research  greater confidence in the evidence and decisions • Evidence alone is never sufficient to make a clinical decision

40. Evidence based clinical decisions Clinical state and circumstances Patient values and preferences Expertise Research evidence Equal for all Haynes et al. 2002

41. About GRADE • Since 2000 • Researchers/guideline developers with interest in methodology • Aim: to develop a common, transparent and sensible system for grading the quality of evidence and the strength of recommendations • Evaluation of existing systems

42. GRADE Evidence Profiles

43. The GRADE approach Clear separation of 2 issues: 1) 4 categories of quality of evidence: very low, low, moderate, or high quality? • methodological quality of evidence • likelihood of bias • by outcome and across outcomes 2) Recommendation: 2 grades - weak or strong (for or against)? • Quality of evidence only one factor *www.GradeWorking-Group.org

44. Determinants of quality • RCTs start high • observational studies start low • what can lower quality? • detailed design and execution • inconsistency • indirectness • reporting bias • imprecision

46. The GRADE approach Separation of 2 issues: 1) 4 categories of quality of evidence: very low, low, moderate, or high quality? • methodological quality of evidence • likelihood of bias • by outcome and across outcomes 2) Recommendation: 2 grades - weak or strong (for or against)? • Quality of evidence only one factor www.GradeWorkingGroup.org

47. Grades of recommendation:Strength of recommendations Strong recommendations • high quality methods with large precise effect • benefits much greater than downsides, or downsides much greater than benefits • we recommend • Grade 1 Weak/conditional recommendations • lower quality methods • benefits not clearly greater or smaller than downsides • values and preferences uncertain or very variable • we suggest • Grade 2

48. Case scenario A 13 year old girl who lives in rural Indonesia presented with flu symptoms and developed severe respiratory distress over the course of the last 2 days. She required intubation. The history reveals that she shares her living quarters with her parents and her three siblings. At night the family’s chicken stock shares this room too and several chicken had died unexpectedly a few days before the girl fell sick.

49. Relevant clinical question?Example from a not so common disease Clinical question: Population: Avian Flu/influenza A (H5N1) patients Intervention: Oseltamivir (or Zanamivir) Comparison: No pharmacological intervention Outcomes: Mortality, hospitalizations, resource use, adverse outcomes, antimicrobial resistance Schunemann et al. The Lancet ID, 2007

50. Methods – WHO Rapid Advice Guidelines for management of Avian Flu • Applied findings of a recent systematic evaluation of guideline development for WHO/ACHR • Group composition (including panel of 13 voting members): • clinicians who treated influenza A(H5N1) patients • infectious disease experts • basic scientists • public health officers • methodologists • Independent scientific reviewers: • Identified systematic reviews, recent RCTs, case series, animal studies related to H5N1 infection