assessing study quality for a systematic review n.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
Assessing study quality for a systematic review PowerPoint Presentation
Download Presentation
Assessing study quality for a systematic review

Loading in 2 Seconds...

play fullscreen
1 / 37

Assessing study quality for a systematic review - PowerPoint PPT Presentation


  • 108 Views
  • Uploaded on

Assessing study quality for a systematic review. Trudy Bekkering , PhD Center of Evidence-Based Medicine & Belgian Branch of the Dutch Cochrane Center Centre for Methodology of Educational Research Katholieke Universiteit Leuven, Belgium. Why is it so important?.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Assessing study quality for a systematic review' - lupita


Download Now An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
assessing study quality for a systematic review

Assessing study quality for a systematic review

Trudy Bekkering, PhD

Center of Evidence-Based Medicine &

Belgian Branch of the Dutch Cochrane Center

Centre for Methodology of Educational Research

KatholiekeUniversiteit Leuven, Belgium

why is it so important
Why is it so important?
  • Meta-analysis aims to increase precision
  • Meta-analysis of studies with bias in results gives very precise but wrong results
  • Garbage in, garbage out
bias versus imprecision
Bias versus imprecision

Bias:

  • A systematic error in the results or the inferences
  • Methodologicalflaw
  • Overestimation or underestimation
bias versus imprecision2
Bias versus imprecision

Imprecision:

  • A random error in the results
  • Sample variation
  • Direction of error is random
bias versus imprecision3
Bias versus imprecision

IMPRECISION

Ideal study

bias versus imprecision4
Bias versus imprecision

BIAS + IMPRECISION

bias versus imprecision5
Bias versus imprecision

BIAS ? IMPRECISION?

risk of bias versus bias
Risk of bias versus bias
  • Clearempiricalevidencethatparticularflaws in study design can lead to bias.
  • Usuallyimpossibletoknowtowhatextentbiaseshave affected the results.
  • Keyconsideration = should the resultsbebelieved
tools for assessing study quality
Tools for assessing study quality

Scales: discouraged

Checklist: between 3 and 57 items

Cochrane tool: “domain based”

Depends on study design

tool for rcts cochrane tool
Tool for RCTs: Cochrane tool

Risk of bias on 6 domaines:

  • Sequencegeneration
  • Allocationconcealment
  • Blinding
  • Incomplete outcome data
  • Selectivereporting
  • Other sources of bias
blinding of intervention
Blinding of intervention
  • Participants (patients, clients)
  • Care providers (doctors, nurses, teachers …)
  • Outcome assessors
incomplete outcome data
Incomplete outcome data

“Attrition” (drop-out): no data

  • Withdrawal
  • Do notattend follow-up appointment
  • Failure to complete questionnaire / diaries
  • Cannotbelocated (lost to follow-up)
  • Decisionby investigator tocease follow-up
  • Data or records are lost
incomplete outcome data1
Incomplete outcome data

“Exclusion”: data available, but excluded from analysis

  • Participants found to be ineligible after enrolment
  • An “as treated” (or per-protocol) analysis:participants are only included if they received the intended intervention in accordance with the protocol
selective outcome reporting
Selectiveoutcomereporting
  • = selection of a subset of the variables recorded for inclusion in publication,
  • on the basis of the results
  • For example:
  • Omission of non-significant outcomes
  • Choice of data for an outcome (e.g. osteoporosis)
  • Choice of analysis (e.g. blood pressure)
  • Reporting of subsets of data (e.g. sepsis)
  • Under-reporting of data (e.g. only “not significant”)
presentation in your review
Presentation in your review

“Risk of bias graph”

assessing risk of bias in nrs
Assessing risk of bias in NRS
  • Selection bias (how was groupallocation?)
  • Performance bias (blinding, fidelity of interventions)
  • Attrition bias (completeness of sample & follow-up)
  • Reporting bias (selectiveoutcomereporting)
  • Confoundingandadjustment
confounding
Confounding

?

?

Imbalance in prognostic factors

Comparison intervention - control

Intervention versus control

Difference in outcome

confounding1
Confounding

Confounding factor

Association between 2 factors

Presence of risk factor

Occurrence of outcome

confounding adjustment
Confounding & adjustment
  • At the stage of protocol: list potential confounding factors
  • Identify the factors the authors have considered and omitted
  • Assess balance between groups at baseline
  • What did authors do to control for confounders (matching, restricting to subgroups, stratification, regression modelling)
tool for nrs
Tool for NRS

Downs and Black instrument

(J Epidemiol Community Health 1998;52:377-84)

27 items:

  • Reporting (10)
  • Externalvalidity (applicability) (3)
  • Internalvalidity - bias (7)
  • Internalvalidity – confounding (6)
  • Power (1)
tool for nrs1
Tool for NRS

Newcastle-Ottawa Scale (NOS)

(Wells 2008)

8 items covering 3 perspectives:

  • Selection of studygroups
  • Comparibility of groups
  • Ascertainement of exposure (case-control) or outcome (cohort)
slide32

http://www.ohri.ca/programs/

clinical_epidemiology/oxford.asp

diagnostic studies
Diagnostic studies

QUADAS tool

Whiting BMC Medical Research Methodology 2003;3:25

Cochrane version: 11 items (out of 14 original)

Diagnostic Test Accuracy Working Group: handbook

http://srdta.cochrane.org/

handbook-dta-reviews

other risk of bias assessment tools
Other risk of bias assessment tools

SIGN (Scottish Intercollegiate Guidelines Network)

http://www.sign.ac.uk/methodology/

checklists.html

in summary
In summary
  • Risk of bias assessment is essential for systematic reviews
  • For RCT: use the Cochranetool
  • For NRS:
    • Higher risk of bias (selection bias & reporting bias)
    • Use the appropriate tool toassess risk of bias
    • Considerhowpotentialconfounders are addressed