Systematic reviews, meta-analysis and critical reading of medical literature: Evidence-based medicine. Phyllis W. Speiser, MD Chief, Div Ped Endo, CCMC Professor of Pediatrics Hofstra-NSLIJ School of Medicine. The need to be evidence-based . Wide variations in practice
Phyllis W. Speiser, MD
Chief, Div Ped Endo, CCMC
Professor of Pediatrics
Hofstra-NSLIJ School of Medicine
Pitfalls of small clinical trials medical literature:
- Bias in subject selection or reporting- Inappropriate endpoint selection- Chance associations of common outcomes/variables- True findings, but unrelated
Was an appropriate spectrum of patients included?
Avoids ascertainment bias
All patients subjected to a gold standard diagnostic test with blinded interpretation?
Avoids observer bias
Methods described & validated
Appropriate diagnostic tools used
Uniform definition of clinical & biochemical phenotype
Low inter-assay and intra-assay variability
Adequate number of subjects in each group
Low drop-out rate
Study design included power analysis
Appropriate statistical methods used
Lijmer et al. Empiric evidence of design-related bias in studies of diagnostic studies. JAMA 1999;282:1061
Optional part of a systematic review
Meta-analysis of RCTs
Systematic review of RCTs
test tube, animal, human physiology
Odds ratio for treatment efficacy =
AD / BC,
or TP x TN / FN x FP
Confidence intervals meta-analysis are based on the assumption that a study provides one sample of observations out of many possible samples that would be derived if the study were repeated many times.
For a 95% confidence interval, if the experiment were repeated many times, 95% of the intervals would contain the true treatment effect.Confidence intervals: Definition
Endpoint #2: Pain at 2 – 7 days improved w/tx meta-analysis
Timing is important!
Effect of population size: Confidence interval and significance improve with larger or pooled samples
Moral: significance improve with larger or pooled samples
Any observed difference between two groups, no
matter how small, can be made “statistically
significant” - at any level of significance - by
taking a sufficiently large sample.
Forest plot of IHD prevalence in SCH and euthyroid controls: Age differences
Razvi, S. et al. J Clin Endocrinol Metab 2008;93:2998-3007
Thrombolytic hypothyroidismtherapy in MI: Power in N!
Importance of current textbooks!
43 21 059
54 22 051
65 47 185
67 47 531
70 48 154
Odds Ratio (Log Scale)
2010: >4000 hypothyroidism
Growth of Cochrane Reviews and Protocols
2500 completed mid-2005
Clinical state and circumstances
…study of the organization, retrieval, appraisal, refinement, dissemination, and uptake of knowledge (eg, important new knowledge from health research)
KT Type 1
Based on Hulley et al. Designing Clinical Research, 2007, p 23
Step 1. Generating Research Evidence hypothyroidism
drugs (not devices or services)
Most use for clinical decisions
Computerized decision support
Evidence-based journal abstracts
>57,000 EBM articles, 2009
Original journal articles
By the year 2020, 90% of clinical decisions hypothyroidism
will be supported by accurate, timely, and
up-to-date clinical information and will reflect
the best available evidence.
IOM Roundtable on
This can’t happen without a better understanding
of the barriers to translating knowledge into practice
and ways to overcome them.
This can happen if
Absolute risk reduction (ARR): 14.7% (19/129 pbo) - 6.9%(17/245 rx)=7.8%
Relative risk reduction: 7.8% (ARR)/14.7% (pbo SAEs) = 53%
Number Needed to Treat: 1/ 7.8% (ARR) = 13 patients with flu need to be treated with tamiflu for 5 days to prevent one complication.
Other resources: hypothyroidism
1. Victor Montori, MD
Mayo Clinic KER unit
2. McMaster Inst., Institute of Medicine
3. Online tutorial
4. Walker E, Hernandez AV, Kattan MW. Meta-analysis: Its strengths & limitations. Cleveland Clinic Journal of Medicine June 2008 vol. 75 6 431-439