1 / 95

Systematic Review and Meta-Analysis: Lecture 2

Systematic Review and Meta-Analysis: Lecture 2. Dejana Braithwaite, PhD, MSc Assistant Professor Department of Epidemiology and Biostatistics.

inez
Download Presentation

Systematic Review and Meta-Analysis: Lecture 2

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Systematic Review and Meta-Analysis: Lecture 2 Dejana Braithwaite, PhD, MSc Assistant Professor Department of Epidemiology and Biostatistics

  2. “Meta-analysis begins with scientific studies, usually performed by academics or government agencies, and sometimes incomplete or disputed. The data from the studies are then run through computer models of bewildering complexity, which produce results of implausible precision.” Davis B. Wall Street Journal, 1992

  3. Learning objectives • Understand the principles of combining studies quantitatively • Understand key meta-analytic models - fixed effects models - random effects models 3. Understand methods for examining reasons for heterogeneity among studies

  4. Historical note on the importance of research synthesis • Karl Pearson is probably the first medical researcher to use formal techniques to combine data from different studies (1904): • He synthesized data from several studies on efficacy of typhoid vaccination • His rationale for pooling data: • “Many of the groups… are far too small to allow of any definite opinion being formed at all, having regard to the size of the probable error involved.” Egger et al. Systematic reviews in health care. London: BMJ Publications, 2001.

  5. Prof Archibald Cochrane, CBE (1909 - 1988) • The Cochrane Collaboration is named in honour of Archie Cochrane, a British researcher. • In 1979 he wrote, "It is surely a great criticism of our profession that we have not organised a critical summary, by specialty or subspecialty, adapted periodically, of all relevant randomized controlled trials” Source: http://www.cochrane.org/cochrane/archieco.htm

  6. The Cochrane Collaboration • Archie Cochrane’s challenge led to the establishment during the 1980s of an international collaboration to develop the Oxford Database of Perinatal Trials. • His encouragement, and the endorsement of his views by others, led to the opening of the first Cochrane centre (in Oxford, UK) in 1992 and the founding of The Cochrane Collaboration in 1993. Source: http://www.cochrane.org/cochrane/archieco.htm

  7. Systematic reviews/meta-analyses indexed in PubMed – 10 years Search: meta-analysis(MeSH) OR meta-analysis(tw) OR systematic review(tw)

  8. http://www.medepi.net/meta/

  9. Systematic Review in context

  10. (Systematic review) ≠ (Meta–analysis) • Meta-analysis is the statistical part of a systematic review • Meta-analysis is always part of a systematic review • Systematic review does not always have a meta-analytic part

  11. Meta-analyses IPD* Systematic reviews *IPD= individual participant data

  12. Systematic reviews: key points • Systematic review(s) when properly done: • are transparent • are NOT synonymous with “meta-analysis” • are replicable by others • are exploding in frequency in numerous health disciplines • Are valuable even when number of studies = 0

  13. Elements of a study protocol for a systematic review and meta-analysis Objectives Background Information retrieval Data collection Data analysis

  14. Today… Steps in data analysis • Create summary data • Examine heterogeneity • Examine publication bias • Consider conducting subgroup analysis (according to a priori hypotheses)

  15. Central questions of interest Are the results of the studies fairly similar (consistent)? Yes No What is the common, summary effect? What factors can explain the dissimilarities (heterogeneity) in the study results? How precise is the common, summary effect?

  16. Opening example of a meta-analysis: BCG and the risk of TB JAMA. 1994 Mar 2;271(9):698-702. • Efficacy of BCG vaccine in the prevention of tuberculosis. Meta-analysis of the published literature. • Colditz GA, Brewer TF, Berkey CS, Wilson ME, Burdick E, Fineberg HV, Mosteller F. Technology Assessment Group, Harvard School of Public Health, Boston, MA 02115.

  17. OBJECTIVE--To quantify the efficacy of BCG vaccine against tuberculosis (TB) DATA SOURCES-- MEDLINE, experts from the Centers for Disease Control and Prevention and the World Health Organization STUDY SELECTION --1264 articles or abstracts considered; 70 articles reviewed in depth --14 prospective trials and 12 case-control studies included DATA EXTRACTION --Recorded study design, age range of study population, number of patients enrolled, efficacy of vaccine, and items to assess the potential for bias in study design and diagnosis.

  18. DATA SYNTHESIS-- The relative risk (RR) or odds ratio (OR) of TB used to provide the measure of vaccine efficacy that we analyzed. The protective effect was then computed by 1-RR or 1-OR. • A random-effects model estimated a weighted average RR or OR from those provided by the trials or case-control studies. • In the trials, the RR of TB was 0.49 (95% confidence interval [CI], 0.34 to 0.70) for vaccine recipients compared with nonrecipients (protective effect of 51%). • In the case-control studies, the OR for TB was 0.50 (95% CI, 0.39 to 0.64), or a 50% protective effect. • 7 trials reporting tuberculous deaths showed a protective effect from BCG vaccine of 71% (RR, 0.29; 95% CI, 0.16 to 0.53) • 5 studies reporting on meningitis showed a protective effect from BCG vaccine of 64% (OR, 0.36; 95% CI, 0.18 to 0.70).

  19. Geographic latitude of the study site and study validity score explained 66% of the heterogeneity among trials in a random-effects regression model.

  20. CONCLUSION--On average, BCG vaccine significantly reduces the risk of TB by 50%. • Protection observed across many populations, study designs, and forms of TB. • Age at vaccination did not enhance predictiveness of BCG efficacy. • Protection against tuberculous death, meningitis, and disseminated disease is higher than for total TB cases, although this result may reflect reduced error in disease classification rather than greater BCG efficacy.

  21. Steps in data analysis • Create summary data • Examine heterogeneity • Consider examining study quality and publication bias • Consider conducting subgroup analysis (according to a priori hypotheses)

  22. Step 1. Create summary data • Prepare tables showing characteristics of primary studies: • Year • Setting • Patients • Design • Outcome (results) • Gives a ‘first hand’ feel for the data • Can make some assessment of quality and heterogeneity

  23. Decide what data to combine • Data types: • Continuous • Dichotomous • Examples of measures that can be combined: • Risk ratio • Odds ratio • Risk difference • Effect size (Z statistic; standardized mean difference) • P-values • Correlation coefficient (R)

  24. Summary dataExample: Cochrane albumin review Cochrane Injuries Group Albumin Reviewers. Human albumin administration in critically ill patients: systematic review of randomised controlled trials. BMJ 1998;317:235-40.

  25. Efficient way of presenting summary results • Forest plot: • Presents the point estimate and CI of each trial • Also presents the overall, summary estimate • Allows visual appraisal of heterogeneity • Other graphs: • Cumulative meta-analysis

  26. Example 1: Meta-analysis

  27. Example 2: Meta-analysis

  28. Forest Plot Cochrane albumin review BMJ 1998;317:235-240

  29. Braithwaite et al. JNCI 2004

  30. Cumulative Meta-analysis Plot Passive smoking and lung cancer review Hackshaw AK et al. BMJ 1997;315:980-88.

  31. Road Map to Generating Summary Effect Estimates • Fixed effects models • Mantel-Haenszel (ratio measures) • Peto (ratio measures) • General variance based (ratio and difference) • Random effects models • DerSimonian Laird (ratio and difference measures)

  32. Fixed versus random effects models • Fixed effects model assumes that the true effect of treatment is the same value in each study (fixed); the differences between studies is solely due to random error • In random effects models, the treatment effects for the individual studies are assumed to vary around some overall average treatment effect • Allows for random error plus inter-study variability • Results in wider confidence intervals (conservative) • Studies tend to be weighted more equally (relatively more weight is given to smaller studies)

  33. Steps in data analysis • Create summary data • Examine heterogeneity • Consider examining study quality and publication bias • Consider conducting subgroup analysis (according to a priori hypotheses)

  34. Step 2. Examine heterogeneity • Indicates that effect sizes vary considerably across studies • If heterogeneity is present, a common, summary measure is hard to interpret • Can be due to differences in: • Patient populations studied • Interventions used • Co-interventions • Outcomes measured • Study design features (eg. length of follow-up) • Study quality • Random error

  35. How to look for heterogeneity? • Visually • Forest plot: do confidence intervals of studies overlap with each other and the summary effect? • Statistically • Chi-square test for heterogeneity (Mantel-Haenszel test or Cochran Q test) • Tests whether the individual effects are farther away from the common effect, beyond what is expected by chance • Has poor power • P-value < 0.10 indicates significant heterogeneity

  36. Visual appraisal of heterogeneity Zinc for common cold: Summary and incidence odds ratios for the incidence of any cold symptom at 1 wk Jackson JL, et al. Zinc and the common cold: a meta-analysis revisited. J of Nutrition. 2000;130:1512S-1515S

  37. Dealing with Heterogeneity • If significant heterogeneity is found: • Find out what factors might explain the heterogeneity • Can decide not to combine the data • If no significant heterogeneity: • Can perform meta-analysis and generate a common, summary effect measure

  38. Examples for calculating summary statistics and assessing heterogeneity

  39. Road Map to Generating Summary Effect Estimates • Fixed effects models • Mantel-Haenszel (ratio measures) • Peto (ratio measures) • General variance based (ratio and difference) • Random effects models • DerSimonian Laird (ratio and difference measures)

  40. Example 1 • Count data (cohort or case-control studies) • Outcome dichotomous (disease+, disease-) • Goal is either an average RR (CIR or OR) over all of the studies or an average RD • Issue—how to weight the studies in the overall average • Issue—how to develop inference about the overall result (i.e. is it significant, what is the CI?, etc.)

  41. Example 2 • Outcome is continuous • Goal is an average difference (in means) over all of the studies • Issue—how to weight the studies in the overall average difference • Issue—how to develop inference about the overall result (i.e. is the difference significant, what is the CI?, etc.)

  42. Pooled estimates (summary ratios or differences)

More Related