grading evidence and recommendations the grade initiative l.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
Grading evidence and recommendations The GRADE initiative PowerPoint Presentation
Download Presentation
Grading evidence and recommendations The GRADE initiative

Loading in 2 Seconds...

play fullscreen
1 / 54

Grading evidence and recommendations The GRADE initiative - PowerPoint PPT Presentation


  • 383 Views
  • Uploaded on

Grading evidence and recommendations The GRADE initiative. Holger Schünemann, MD, PhD Associate Professor Italian National Cancer Institute Regina Elena, Rome.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Grading evidence and recommendations The GRADE initiative' - Jimmy


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
grading evidence and recommendations the grade initiative

Grading evidence and recommendationsThe GRADE initiative

Holger Schünemann, MD, PhD

Associate Professor

Italian National Cancer Institute Regina Elena, Rome

slide2

Professional good intentions and plausible theories are insufficient for selecting policies and practices for protecting, promoting and restoring health.

Iain Chalmers

slide3

How can we judge the extent of our confidence that adherence to a recommendation will do more good than harm?

grade

GRADE

Grades of Recommendation Assessment, Development and Evaluation

what do you know about grade
What do you know about GRADE?
  • Have prepared a guideline
  • Read the BMJ paper
  • Have prepared a systematic review and a summary of findings table
  • Have attended a GRADE meeting, workshop or talk
about grade
About GRADE*
  • Began as informal working group in 2000
  • Researchers/guideline developers with interest in methodology
  • Aim: to develop a common system for grading the quality of evidence and the strength of recommendations that is sensible and to explore the range of interventions and contexts for which it might be useful*
  • 13 meetings (~10 – 35 attendants)
  • Evaluation of existing systems and reliability*
  • Workshops at Cochrane Colloquia, WHO, GIN and various conferences since 2000

*Grade Working Group. CMAJ 2003, BMJ 2004, BMC 2004, BMC 2005

grade working group
David Atkins, chief medical officera

Dana Best, assistant professorb

Martin Eccles, professord

Francoise Cluzeau, lecturerx

Yngve Falck-Ytter, associate directore

Signe Flottorp, researcherf

Gordon H Guyatt, professorg

Robin T Harbour, quality and information director h

Margaret C Haugh, methodologisti

David Henry, professorj

Suzanne Hill, senior lecturerj

Roman Jaeschke, clinical professork

Regina Kunx, Associate Professor

Gillian Leng, guidelines programme directorl

Alessandro Liberati, professorm

Nicola Magrini, directorn

James Mason, professord

Philippa Middleton, honorary research fellowo

Jacek Mrukowicz, executive directorp

Dianne O’Connell, senior epidemiologistq

Andrew D Oxman, directorf

Bob Phillips, associate fellowr

Holger J Schünemann, associate professorg,s

Tessa Tan-Torres Edejer, medical officert

Jane Thomas, Lecturer, UK

Helena Varonen, associate editoru

Gunn E Vist, researcherf

John W Williams Jr, professorv

Stephanie Zaza, project directorw

a) Agency for Healthcare Research and Quality, USA

b) Children's National Medical Center, USA

c) Centers for Disease Control and Prevention, USA

d) University of Newcastle upon Tyne, UK

e) German Cochrane Centre, Germany

f) Norwegian Centre for Health Services, Norway

g) McMaster University, Canada

h) Scottish Intercollegiate Guidelines Network, UK

i) Fédération Nationale des Centres de Lutte Contre le Cancer, France

j) University of Newcastle, Australia

k) McMaster University, Canada

l) National Institute for Clinical Excellence, UK

m) Università di Modena e Reggio Emilia, Italy

n) Centro per la Valutazione della Efficacia della Assistenza Sanitaria, Italy

o) Australasian Cochrane Centre, Australia

p) Polish Institute for Evidence Based Medicine, Poland

q) The Cancer Council, Australia

r) Centre for Evidence-based Medicine, UK

s) National Cancer Institute, Italy

t) World Health Organisation, Switzerland

u) Finnish Medical Society Duodecim, Finland

v) Duke University Medical Center, USA

w) Centers for Disease Control and Prevention, USA

x) University of London, UK

GRADE Working Group
what do users want from guidelines
What do users want from guidelines?
  • users looking for different things
  • just tell me what to do (recommendation)
  • what to do, and on strong or weak grounds
    • recommendation and grade
  • recommend, grade, evidence summary, values
    • systematic review, value statement
  • evidence from individual studies
when to make a recommendation
When to make a recommendation?
  • never
    • patient values differ
    • just lay out benefits and risks
  • when evidence strong enough
    • when very weak, too uncertain
  • clinicians need guidance
    • intense study demands decision
why bother about grading
Why bother about grading?
  • People draw conclusions about the
    • quality of evidence
    • strength of recommendations
  • Systematic and explicit approaches can help
    • protect against errors
    • resolve disagreements
    • facilitate critical appraisal
    • communicate information
  • However, there is wide variation in currently used approaches
who is confused
Evidence Recommendation

II-2 B

C+ 1

Strong Strongly recommended

Organization

USPSTF

ACCP

GCPS

Who is confused?
still not confused
EvidenceRecommendation

B Class I

C+ 1

IV C

Organization

AHA

ACCP

SIGN

Still not confused?

Recommendation for use of oral anticoagulation in patients with atrial fibrillation and rheumatic mitral valve disease

grading system
Grading System
  • current profusion: can there be consensus?
  • trade-off benefits and risks
    • do it (or don’t do it)
    • probably do it (or probably don’t do it)
  • quality of underlying evidence
    • high quality (well done RCT)
    • intermediate (quasi-RCT)
    • low (well done observational)
    • very low (anything else)
moving down
Moving down
  • poor RCT design, implementation
    • randomization, concealment, follow-up
    • inconsistency
  • indirect
    • patients, interventions, outcomes
    • A vs B, but have A to C, B to C
  • reporting bias
moving up
Moving up
  • magnitude of effect
  • dose-response
  • biases favor control
example accp
Example ACCP
  • First ACCP guidelines in 1986 (J. Hirsh; J. Dalen)
  • Initially aimed at consensus
  • Methodologists involved since beginning
  • Now formally convening every 2 to 3 years
  • Seventh conference held in 2003; > 200.000 copies published in Chest
  • 87 panel members, 22 chapters
  • Across subspecialties
  • 565 recommendations, 230 new
  • Evidence Based Recommendations
  • Next conference in 2006
what makes guidelines evidence based in 2005
What makes guidelines evidence based (in 2005)?
  • Evidence – recommendation: transparent link
  • Explicit inclusion criteria
  • Comprehensive search
  • Standardized consideration of study quality
  • Conduct/use meta-analysis
  • Evaluate overall quality of evidence
  • Grade recommendations
  • Acknowledge values and preferences

Schünemann et al. Chest 2004

judgements about the overall quality of evidence
Judgements about the overall quality of evidence
  • Most systems not explicit
  • Options:
    • strongest outcome
    • primary outcome
    • benefits
    • weighted
    • separate grades for benefits and harms
    • no overall grade
    • weakest outcome
  • Based on lowest of all the critical outcomes
  • Beyond the scope of a systematic review
quality of evidence
Quality of evidence

“The extent to which one can be confident that an estimate of effect or association is correct.”

It depends on the:

  • study design (e.g. RCT, cohort study)
  • study quality/limitations (protection against bias; e.g. concealment of allocation, blinding, follow-up)
  • consistency of results
  • directness of the evidence including the
    • populations (those of interest versus similar; for example, older, sicker or more co-morbidity)
    • interventions (those of interest versus similar; for example, drugs within the same class)
    • outcomes (important versus surrogate outcomes)
    • comparison (A - C versus A - B & C - B)
quality of evidence22
Quality of evidence

The quality of the evidence (i.e. our confidence) may also be REDUCEDwhen there is:

  • Sparse or imprecise data
  • Reporting bias

The quality of the evidence (i.e. our confidence) may be INCREASEDwhen there is:

  • A strong association
  • A dose response relationship
  • All plausible confounders would have reduced the observed effect
  • All plausible biases would have increased the observed lack of effect
categories of quality
Categories of quality
  • High: Further research is very unlikely to change our confidence in the estimate of effect.
  • Moderate: Further research is likely to have an important impact on our confidence in the estimate of effect and may change the estimate.
  • Low: Further research is very likely to have an important impact on our confidence in the estimate of effect and is likely to change the estimate.
  • Very low: Any estimate of effect is very uncertain.
strength of recommendation
Strength of recommendation

“The extent to which one can be confident that adherence to a recommendation will do more good than harm.”

  • quality of the evidence
  • translation of the evidence into practice in a specific setting
  • uncertainty about baseline risk
  • trade-offs (the relative value attached to the expected benefits, harms and costs)
clarity of the trade offs between benefits and the harms
Clarity of the trade-offs between benefits and the harms
  • the estimated size of the effect for each main outcome
  • the precision of these estimates
  • important factors that could be expected to modify the size of the expected effects in specific settings; e.g. proximity to a hospital
  • the relative value attached to the expected benefits and harms
  • the variation in values between people
slide27

← Option 1 (pink card)

Option 2 → (green card)

slide29

← Option 1 (pink card)

Option 2 → (green card)

slide31

Strawberry

← Option 1 (pink card)

Chocolate

Option 2 → (green card)

slide33

← Option 1 (pink card)

Red Ferrari

Option 2 → (green card)

Yellow fox

judgements about the balance between benefits and harms
Judgements about the balance between benefits and harms
  • Before considering cost and making a recommendation
judgements about recommendations36
Judgements about recommendations
  • “We recommend”…”should” …“Do it”
  • “We suggest”…”may” … “Probably do it”
  • “We recommend not”… “may not” …“Probably don’t do it”
  • “We suggest not”…”should not”… “Don’t do it”

No recommendation

This could include considerations of costs; i.e. “Is the net gain (benefits-downsides) worth the costs?”

slide37

Will GRADE lead to change?

Should healthy asymptomatic postmenopausal women have been given oestrogen + progestin for prevention in 1992?
  • Quality of evidence across studies for
    • CHD
    • Hip fracture
    • Colorectal cancer
    • Breast cancer
    • Stroke
    • Thrombosis
    • Gall bladder disease
  • Quality of evidence across critical outcomes
  • Balance between benefits and harms
  • Recommendations
further grade developments
Further GRADE developments
  • Diagnostic tests
  • Costs
  • (Equity)
  • Empirical evaluations
  • Free software application
grade profile
GRADE Profile
  • Excel, HTML, MS Word format
  • Linked to REVMAN (direct import from REVMAN)
comparison of grade and other systems
Comparison of GRADE and other systems
  • Explicit definitions
  • Explicit, sequential judgements
  • Components of quality
  • Overall quality
  • Relative importance of outcomes
  • Balance between health benefits and harms
  • Balance between incremental health benefits and costs
  • Consideration of equity
  • Evidence profiles
  • International collaboration
  • Software
  • Consistent judgements?
  • Communication?
who is interested in grade
Who is interested in GRADE
  • WHO
  • American Endocrine Society
  • American College of Chest Physicians (ACCP)
  • Italian National Cancer Institute, Rome
  • Clinical Evidence
  • Norwegian Centre for Health Services
  • UpToDate
  • Close relationship with Cochrane Collaboration
  • American Society of Clinical Oncology (ASCO)
  • American Thoracic Society (ATS)
  • Urologists worldwide
empirical evaluations
Empirical evaluations
  • Critical appraisal of other systems
  • Pilot test + sensibility
  • “Case law” + practical experience
  • Guidance for judgements
    • Single studies
    • Sparse data or imprecise data
  • Agreement
  • Validity?
  • Comparisons with other systems
  • Alternative presentations