PH 401: Meta-analysis

1. PH 401: Meta-analysis Eunice Pyon, PharmD eunice.pyon@liu.edu (718) 488-1246, HS 506

2. Meta-analysis • Quantitative systematic review • Combines data from previously conducted clinical trials (and epidemiologic research) and performs statistical analyses on pooled results NOTE: different from a review article

3. Meta-analysis Useful when: • definitive clinical trials are impossible, unethical or impractical • randomized trials have been performed but results are conflicting • results from definitive trials are being awaited • new questions not posed at the beginning of the trial need to answered • sample sizes are too small

4. Meta-analysis Purposes include: • to increase statistical power for primary endpoint and or subgroups • to resolve uncertainty when reports disagree • to improve estimates of size of effect • to answer new questions not posed at the start of the trials • to bring about improvements in quality of primary research

5. Meta-analysis: SSRIs • Whittington CJ, et al. Selective serotonin reuptake inhibitors in childhood depression: systematic review of published versus unpublished data. Lancet 2004;363:1341-1345.

6. Meta-analysis: Cox-2 Furberg CD, et al. Parecoxib, valdecoxib, and cardiovascular risk. Circulation 2005;111:249.

7. Meta-analysis: the process • Problem formulation • Data collection • Evaluation of the collected data • Analysis and interpretation • Presentation of Results

8. Problem formulation • Clearly define the clinical question • specify variables • evaluate relationship between variables (cause and effect)

9. Data collection • Describe details of literature search • databases • published vs. unpublished • additional sources (i.e.., reference lists, meetings) • Describe inclusion/exclusion criteria • study design • participants • treatment • outcome measures

10. Evaluation of data Even before the “data” • Author • Funding • Relevant information Important part of data evaluation. Different ways to incorporate into meta-analysis: exclusion, weighting, stratifying

11. Evaluation of data • Raw data, individual patient data • preferred • difficult to obtain • Summary data • more commonly utilized

12. Evaluation of data • Homogeneity vs. heterogeneity • L’abbe plot • Cochran-Q • Publication Bias • Funnel plot

14. Funnel Plot

15. Meta-analysis continued Remember: Meta-analysis is an observational study of evidence. It is retrospective.

16. Evaluation of data • Scrutinize validity of trials • randomization techniques • sample size • compliance • blinding • intention to treat vs. per protocol Primary studies may be weighted to reflect quality of research design. Weighting of data is controversial. Investigators should be blinded to: authors, institutions, journals, funding, acknowledgements.

17. Analysis and interpretation • Appropriate statistical analyses • standardized outcome measure • Continuous (i.e., blood pressure): differences, standard deviations • Binary (i.e., dead or alive): odds ratio, relative risk • overall effect; combining data • fixed effects model--assumes same effect across studies • random effects model--assumes different underlying effect for all studies • and others…

18. Analysis and interpretation • Odds Ratio

19. Lung cancer cases Controls Smokers 647 622 Non-smokers  2 27 Analysis and Interpretation: Odds Ratio Cigarette smoking and lung cancer (Doll and Hill BMJ 1950 ii 739-748). Results for men. OR=?

20. Lung cancer cases Controls Smokers 647 622 Non-smokers  2 27 Analysis and Interpretation: Odds Ratio Cigarette smoking and lung cancer (Doll and Hill BMJ 1950 ii 739-748). Results for men. Odds ratio = (647x27) / (2x622) = 14.04 Lung cancer cases 14 x more likely to be smokers.

21. Analysis and interpretation • Relative Risk

22. Analysis and interpretation • Sensitivity analysis • Overall effect calculated by different methods (fixed vs. random) • Reanalysis with exclusion of poor-quality studies • Reanalysis with exclusion of small studies • Reanalysis of exclusion of studies with short duration of follow-up

23. Presentation of results • Often graphically displayed with confidence intervals • Type I and II error should be discussed • Robustness of findings/sensitivity should be discussed

24. Strengths • Can summarize from available studies the effects of interventions across many patients • Can reveal research designs as moderators of study results • Can reduce false negative results • Can clarify heterogeneity between study results

25. Strengths • Can assist in accurate calculation of sample size needed in future studies • Can suggest promising research questions for future study • Can allow more objective assessment of evidence and thereby reduce disagreement

26. Weaknesses • Can pass along inflated estimates of size effects based previously reported results • Cannot overcome subjectivity in choice of outcomes and their weighting in analysis • Can be compromised by publication bias

27. Weaknesses • Arithmetic nature of meta-analysis can produce false impression of certainty in an inherently uncertain process with many subjective elements

28. Cochrane Collaboration • The Cochrane Collaboration is an international not-for-profit organization, providing information about the effects of health care • Source of qualitative and quantitative systematic reviews with good methodological rigor www.cochrane.org

29. Conclusions • Interpret with caution remembering that conclusions depend on the quality of the studies included • Findings of subsequent randomized controlled trials may differ

30. References • Malone PM et al. Drug information: a guide for pharmacists. McGraw-Hill. New York. 2nd edition. 2001. • Noble Jr JH. Meta-analysis: methods, strengths, weaknesses, and political uses. J Lab Clin Med 2006;147:7-20.