Systematic reviews, meta-analysis and critical reading of medical literature: Evidence-based medicine. Phyllis W. Speiser, MD Chief, Div Ped Endo, CCMC Professor of Pediatrics Hofstra-NSLIJ School of Medicine. The need to be evidence-based . Wide variations in practice
Systematic reviews, meta-analysis and critical reading of medical literature:Evidence-based medicine
Phyllis W. Speiser, MD
Chief, Div Ped Endo, CCMC
Professor of Pediatrics
Hofstra-NSLIJ School of Medicine
Pitfalls of small clinical trials
- Bias in subject selection or reporting- Inappropriate endpoint selection- Chance associations of common outcomes/variables- True findings, but unrelated
Was an appropriate spectrum of patients included?
Avoids ascertainment bias
All patients subjected to a gold standard diagnostic test with blinded interpretation?
Avoids observer bias
Methods described & validated
Appropriate diagnostic tools used
Uniform definition of clinical & biochemical phenotype
Low inter-assay and intra-assay variability
Adequate number of subjects in each group
Low drop-out rate
Study design included power analysis
Appropriate statistical methods used
Lijmer et al. Empiric evidence of design-related bias in studies of diagnostic studies. JAMA 1999;282:1061
Optional part of a systematic review
Meta-analysis of RCTs
Systematic review of RCTs
test tube, animal, human physiology
Odds ratio for treatment efficacy =
AD / BC,
or TP x TN / FN x FP
Confidence intervals are based on the assumption that a study provides one sample of observations out of many possible samples that would be derived if the study were repeated many times.
For a 95% confidence interval, if the experiment were repeated many times, 95% of the intervals would contain the true treatment effect.
Endpoint #2: Pain at 2 – 7 days improved w/tx
Timing is important!
Effect of population size: Confidence interval and significance improve with larger or pooled samples
Any observed difference between two groups, no
matter how small, can be made “statistically
significant” - at any level of significance - by
taking a sufficiently large sample.
Forest plot of IHD prevalence in SCH and euthyroid controls: Age differences
Razvi, S. et al. J Clin Endocrinol Metab 2008;93:2998-3007
Thrombolytictherapy in MI: Power in N!
Importance of current textbooks!
43 21 059
54 22 051
65 47 185
67 47 531
70 48 154
Odds Ratio (Log Scale)
Growth of Cochrane Reviews and Protocols
2500 completed mid-2005
Clinical state and circumstances
…study of the organization, retrieval, appraisal, refinement, dissemination, and uptake of knowledge (eg, important new knowledge from health research)
KT Type 1
Based on Hulley et al. Designing Clinical Research, 2007, p 23
Step 1. Generating Research Evidence
drugs (not devices or services)
Most use for clinical decisions
Computerized decision support
Evidence-based journal abstracts
>57,000 EBM articles, 2009
Original journal articles
By the year 2020, 90% of clinical decisions
will be supported by accurate, timely, and
up-to-date clinical information and will reflect
the best available evidence.
IOM Roundtable on
This can’t happen without a better understanding
of the barriers to translating knowledge into practice
and ways to overcome them.
This can happen if
Absolute risk reduction (ARR): 14.7% (19/129 pbo) - 6.9%(17/245 rx)=7.8%
Relative risk reduction: 7.8% (ARR)/14.7% (pbo SAEs) = 53%
Number Needed to Treat: 1/ 7.8% (ARR) = 13 patients with flu need to be treated with tamiflu for 5 days to prevent one complication.
1. Victor Montori, MD
Mayo Clinic KER unit
2. McMaster Inst., Institute of Medicine
3. Online tutorial
4. Walker E, Hernandez AV, Kattan MW. Meta-analysis: Its strengths & limitations. Cleveland Clinic Journal of Medicine June 2008 vol. 75 6 431-439