slide1 n.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
Why Grade Recommendations? PowerPoint Presentation
Download Presentation
Why Grade Recommendations?

Loading in 2 Seconds...

play fullscreen
1 / 74

Why Grade Recommendations? - PowerPoint PPT Presentation


  • 138 Views
  • Uploaded on

Why Grade Recommendations?. strong recommendations strong methods large precise effect few down sides of therapy weak recommendations weak methods imprecise estimate small effect substantial down sides. Which grading system to use?. many available Australian National and MRC

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Why Grade Recommendations?' - brandice


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
why grade recommendations
Why Grade Recommendations?
  • strong recommendations
    • strong methods
    • large precise effect
    • few down sides of therapy
  • weak recommendations
    • weak methods
    • imprecise estimate
    • small effect
    • substantial down sides
which grading system to use
Which grading system to use?
  • many available
    • Australian National and MRC
    • Oxford Center for Evidence-based Medicine
    • Scottish Intercollegiate Guidelines (SIGN)
    • US Preventative Services Task Force
    • American professional organizations
      • AHA/ACC, ACCP, AAP, Endocrine society, etc....
  • cause of confusion, dismay
a common international grading system
A common international grading system?
  • GRADE (Grades of recommendation, assessment, development and evaluation)
  • international methodologists, guideline developers
    • Australian NMRC, SIGN, USPSTF, WHO, NICE, Oxford CEBM, CDC, CC
  • ~ 20 meetings over last eight years
      • (~10 – 50 attendants)
  • BMJ 2004, six part series 2008
grade uptake
GRADE Uptake
  • UpToDate World Health Organization
  • British Medical Journal American Thoracic Society
  • American College of Physicians Cochrane Collaboration
  • BMJ Clinical Evidence KDIGO
  • Polish Institute for EBM EBM Guidelines Finland
  • Society of Vascular Surgery Society of Pediatric Endocrinology
  • European Respiratory Society American Endocrine Society
  • Society of Critical Care Medicine Surviving sepsis campaign
  • American College of Chest Physicians European Soc of Thoracic Surgeons
  • Allergic Rhinitis in Asthma Guidelines Society of Vascular Surgery
  • Infectious Disease Society of America
  • National Institute for Clinical Excellence (NICE)
  • Agency for Health Care Research and Quality (AHRQ)
  • Swedish National Board of Health and Welfare
  • Canadian Agency for Drugs and Technology in Health
  • Ontario MOH Medical Advisory Secretariat
  • Agencia sanitaria regionale, Bologna, Italia
  • The German Agency for Quality in Medicine
  • Evidence-based Nursing Sudtirol, Alta Adiga, Italy
  • Norwegian Knowledge Centre for the Health Services
  • University of Pennsylvania Health System Center for EB Practice
  • Journal of Infection in Developing Countries – International
  • Japanese Society of Oral and Maxilofacial Radiology
  • Emergency Medical Services for Children National Resource Center
what are we grading
What are we grading?
  • two components
  • quality of body of evidence
    • confidence in estimate of effect
      • high, moderate, low, very low
  • strength of recommendation
      • strong and weak
slide8

Studies

S1

S2

S3

S4

S5

Health Care Question

(PICO)

Systematic reviews

Outcomes

OC1

OC2

OC3

OC4

Important

outcomes

Critical

outcomes

OC1

OC2

OC3

OC4

Generate an estimate of effect for each outcome

Rate the quality of evidence for each outcome, across studies

RCTs start high, observational studies start low

(-)

Study limitations

Imprecision

Inconsistency of results

Indirectness of evidence

Publication bias likely

Final rating of quality for each outcome: high, moderate, low, or very low

(+)

Large magnitude of effect

Dose response

Plausible confounders would ↓ effect when an effect is present or ↑ effect if effect is absent

Rate overall quality of evidence

(lowest quality among critical outcomes)

Decide on the direction (for/against) and grade strength (strong/weak*) of the recommendation considering:

Quality of the evidence

Balance of desirable/undesirable outcomes

Values and preferences

Decide if any revision of direction or strength is necessary considering: Resource use

*also labeled “conditional”

or “discretionary”

structured question
Structured question
  • patients: lymphoma patients at risk of developing chemotherapy-induced febrile neutropenia
  • granulocyte colony-stimulating (G-CSF)
  • alternative not using G-CSF
design and execution
Design and Execution
  • well established
    • concealment
    • intention to treat principle observed
    • blinding
    • completeness of follow-up
  • more recent
    • early stopping for benefit
    • selective outcome reporting bias
consistency of results
Consistency of results
  • consistency of results
  • if inconsistency, look for explanation
    • patients, intervention, outcome, methods
  • judgment of consistency
    • variation in size of effect
    • overlap in confidence intervals
    • statistical significance of heterogeneity
    • I2
directness of evidence
Directness of Evidence
  • differences in patients
    • age, sex, ethnicity, condition – avian versus regular influenza
  • interventions
    • dose, class
  • outcomes
    • health-related quality of life, functional capacity, laboratory exercise
slide15

Critical

for decision making

Important,

butnot critical for

decision making

Of low patient-

importance

Figure 6: Hierarchy of outcomes according to their patient-importance to assess the effect of phosphate lowering drugs in patients with renal failure and hyperphophatemia

Importance

of endpoints

Surrogates of declining importance

Mortality 9

Coronary

calcification

Ca2+/P-

Product

Myocardial infarction 8

Bone

density

Ca2+/P-

Product

Fractures 7

Pain due to soft tissue

Calcification / function 6

Soft tissue calcification

Ca2+/P-

Product

5

4

Lower by one level for indirectness

3

2

Flatulence

Lower by two levels for indirectness

1

slide16

Directness

interested in A versus B

available data A vs C, B vs C

Alendronate

Risedronate

Placebo

imprecision
Imprecision
  • small sample size
    • small number of events
  • wide confidence intervals
    • uncertainty about magnitude of effect
  • how to decide if CI too wide?
    • grade down one level?
    • grade down two levels?
  • extent of confidence in estimate of effect
offer all effective treatments
Offer all effective treatments?
  • atrial fib at risk of stroke
  • warfarin increases serious gi bleeding
    • 3% per year
  • 1,000 patients 1 less stroke
    • 30 more bleeds for each stroke prevented
  • 1,000 patients 100 less strokes
    • 3 strokes prevented for each bleed
  • where is your threshold?
    • how many strokes in 100 with 3% bleeding?
publication bias
Publication bias
  • high likelihood could lower quality
  • reporting of studies
    • publication bias
      • number of small studies
      • industry sponsored
what can lower quality
What can lower quality?
  • detailed design and execution
  • inconsistency
  • indirectness
  • imprecision
  • publication bias
what can raise quality
What can raise quality?
  • large magnitude can upgrade one level
    • very large two levels
  • common criteria
    • everyone used to do badly
    • almost everyone does well
    • quick action
  • hip replacement for severe osteoarthritis
  • dialysis vs no dialysis for prolonging life
dose response gradient
Dose-response gradient
  • childhood lymphoblastic leukemia
  • risk for CNS malignancies 15 years after cranial irradiation
  • no radiation: 1% (95% CI 0% to 2.1%)
  • 12 Gy: 1.6% (95% CI 0% to 3.4%)
  • 18 Gy: 3.3% (95% CI 0.9% to 5.6%).
strength of recommendation
Strength of Recommendation
  • strong recommendation
    • benefits clearly outweigh risks/hassle/cost
    • risk/hassle/cost clearly outweighs benefit
  • what can downgrade strength?
    • low quality evidence
    • close balance between up and downsides
risk benefit tradeoff
Risk/Benefit tradeoff
  • aspirin after myocardial infarction
    • 25% reduction in relative risk
    • side effects minimal, cost minimal
    • benefit obviously much greater than risk/cost
  • warfarin in low risk atrial fibrillation
    • warfarin reduces stroke vs ASA by 50%
    • but if risk only 1% per year, ARR 0.5%
    • increased bleeds by 1% per year
strength of recommendations
Strength of Recommendations

Aspirin after MI – do it

Warfarin rather than ASA in Afib

-- probably do it

-- probably don’t do it

significance of strong vs weak
Significance of strong vs weak
  • variability in patient preference
    • strong, almost all same choice (> 90%)
    • weak, choice varies appreciably
  • interaction with patient
    • strong, just inform patient
    • weak, ensure choice reflects values
  • use of decision aid
    • strong, don’t bother
    • weak, use the aid
  • quality of care criterion
    • strong, consider
    • weak, don’t consider
when evidence is low quality
When evidence is low quality
  • choice more preference dependent
  • risk aversion
  • steroids for pulmonary fibrosis
    • low quality evidence in support of benefit
    • high quality evidence of toxicity
when evidence is low quality1
When evidence is low quality
  • recommendation to the hopeful patient
    • I’m likely to deteriorate
    • if something might work, let’s try it
    • damn the torpedoes
  • recommendation to the fearful patient
    • doctor, you mean you know it’s toxic
      • diabetes, skin changes, body habitus, infection, osteoporosis
    • you don’t know for sure it works?
    • are you crazy?
  • discretionary recommendation mandated
strong recommendation when evidence is low quality
Strong recommendation when evidence is low quality?
  • recommendations against
    • uncertainty of benefit
    • confidence in down sides
  • whole body CT or MRI screening
    • maybe benefit, maybe not
    • true positives some harm
    • false positive some harm
strong recommendation when evidence is low quality1
Strong recommendation when evidence is low quality?
  • known benefit, strong recommendation for one of two alternatives
    • antipyretics in children with chickenpox
    • but which one: ASA or acetaminophen
  • benefit: high quality evidence of equivalence
  • harm: low quality evidence that harm differs appreciably
    • Reye syndrome from ASA
  • strong recommendation for acetaminophen?
strong recommendation when evidence is low quality2
Strong recommendation when evidence is low quality?
  • Blastomycosis
    • low quality evidence amphotericin more effective than itraconazole
    • high quality evidence more toxic
  • patients with life threatening blasto
    • life and death situation
    • strong recommendation for ampho
value and preference statements
Value and preference statements
  • underlying values and preferences always present
  • sometimes crucial
  • important to make explicit
values and preferences
Values and preferences

Stroke guideline: patients with TIA clopidogrel over aspirin (Grade 2B).

Underlying values and preferences: This recommendation to use clopidogrel over aspirin places a relatively high value on a small absolute risk reduction in stroke rates, and a relatively low value on minimizing drug expenditures.

values and preferences1
Values and preferences

peripheral vascular disease: aspirin be used instead of clopidogrel (Grade 2A).

Underlying values and preferences: This recommendation places a relatively high value on avoiding large expenditures to achieve small reductions in vascular events.

flavanoids for hemorrhoids
Flavanoids for Hemorrhoids
  • venotonic agents
    • mechanism unclear, increase venous return
  • popularity
    • 90 venotonics commercialized in France
    • none in Sweden and Norway
    • France 70% of world market
  • possibilities
    • French misguided
    • rest of world missing out
systematic review
Systematic Review
  • 14 trials, 1432 patients
  • key outcome
    • risk not improving/persistent symptoms
    • 11 studies, 1002 patients, 375 events
    • RR 0.4, 95% CI 0.29 to 0.57
  • minimal side effects
  • is France right?
  • what is the quality of evidence?
what can lower quality1
What can lower quality?
  • detailed design and execution
    • lack of detail re concealment
    • questionnaires not validated
  • rate down quality for study limitations?
  • indirectness – no problem
  • inconsistency, need to look at the results
publication bias1
Publication bias?
  • size of studies
    • 40 to 234 patients, most around 100
  • all industry sponsored
what can lower quality2
What can lower quality?
  • detailed design and execution
    • lack of detail re concealment
    • questionnaires not validated
  • inconsistency
    • almost all show positive effect, trend
    • heterogeneity p < 0.001; I2 65.1%
  • indirectness
  • imprecision
    • RR 0.4, 95% CI 0.29 to 0.57
  • reporting bias
    • 40 to 234 patients, most around 100
is france right
Is France right?
  • no
    • recommend against use
  • strong or weak
  • yes
    • ensure venotonics available
    • recommend for use in individuals
  • strong or weak?
poldermans nejm 1999
Poldermans, NEJM, 1999
  • elective vascular surgery
    • positive dobutamine stress echo
  • compared bisoprolol to placebo
    • unblinded
  • primary endpoint death or nonfatal MI
poldermans nejm 19991
Poldermans NEJM 1999
  • planned sample size 266
  • prior planned single look at 100 pts
    • stop if exceeded O’Brien-Fleming boundary
      • p < 0.001
  • stopped after 112 patients
  • primary endpoint
    • 2 of 59 (3.4%) in bisoprolol group
    • 18 of 53 (34%) in placebo
  • RR 0.09, 95% CI 0.02 to 0.37, P< 0.001
slide56

406

3548

2005

1506

886

Final Recruitment8351 pts from 190 sites in 23 countries

validity
Validity
  • randomization concealed
  • blinded
    • patients
    • clinicians
    • date collectors
    • adjudicators
  • follow-up 99.8%
  • followed ITT principle
recommendations
Recommendations
  • beta blockers
    • for
    • against
  • Strength of recommendation
    • strong
    • weak
acetylcysteine paracetamol overdose
Acetylcysteine paracetamol overdose
  • patients: acetaminophen overdose
  • intervention: aceteylcysteine
  • comparator: no acetylcysteine
  • outcomes
    • death
    • liver failure
acetylcysteine keays 1991
Acetylcysteine Keays 1991
  • randomized trial fulminant hepatic failure after paracetamol before acetylcysteine
  • loading dose and infusion of acetylcysteine until resolution or death vs no acetylcysteine
  • treatment began 33 to 96 hours after OD
  • validity
    • concealment: envelopes
    • blinding – none
    • LFUP – none
  • 12/25 (48%) acetylcysteine survived vs 5/25 (20%) of controls
    • (RR dying 0.65, 95% CI 0.43 to 0.99; p=0 037)
quality of evidence
Quality of evidence
  • RCT starts high
    • no study limitations
    • imprecise (CI too wide)
    • directness (possibly indirect)
    • consistency (no other RCT)
    • publication bias – no suggestion
  • overall moderate quality
acetylcysteine for paracetamol od observational studies
Acetylcysteine for paracetamol ODobservational studies
  • Prescott BMJ 1979
    • severe liver damage AST or ALT > 1,000
    • 1/62 treated within 10 hours (2%)
    • retrospective series 22/57 (58%)
  • Smilkstein NEJM 1988
    • same criteria for damage
    • 32/537 (6.1%) treated in 10 hours
    • 247/935 (26.4%) treated 10 to 24 hours
acetylcysteine for paracetamol od observational studies1
Acetylcysteine for paracetamol ODobservational studies
  • observational studies start low quality
  • very large effect
    • rate up 1 level to moderate or 2 to high
recommendations1
Recommendations
  • acetylcysteine
    • for
    • against
  • strength of recommendation
    • strong
    • weak
healthy asymptomatic postmenopausal qomwn hrt in 1992

Can GRADE lead to change?

healthy asymptomatic postmenopausal qomwn: HRT in 1992?

Possible benefits

  • CHD, Hip fracture, Colorectal cancer

Possible harms

  • Breast cancer
  • Stroke
  • Thrombosis
  • Gall bladder disease
conclusion
Conclusion
  • clinicians, policy makers need summaries
    • quality of evidence
    • strength of recommendations
  • explicit rules
    • transparent, informative
  • GRADE
    • simple, transparent, systematic
    • increasing wide adoption