slide1 n.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
Grant Stephen: Chair of the MBC Life Science Informatics Group & CEO, Tessella Inc: grant.stephen@tessella.com PowerPoint Presentation
Download Presentation
Grant Stephen: Chair of the MBC Life Science Informatics Group & CEO, Tessella Inc: grant.stephen@tessella.com

Loading in 2 Seconds...

play fullscreen
1 / 70

Grant Stephen: Chair of the MBC Life Science Informatics Group & CEO, Tessella Inc: grant.stephen@tessella.com - PowerPoint PPT Presentation


  • 147 Views
  • Uploaded on

Life Science Informatics Group. Introduction to Meta-Analysis Christopher H. Schmid, PhD Tufts-New England Medical Center 6 June 2008. Grant Stephen: Chair of the MBC Life Science Informatics Group & CEO, Tessella Inc: grant.stephen@tessella.com.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Grant Stephen: Chair of the MBC Life Science Informatics Group & CEO, Tessella Inc: grant.stephen@tessella.com' - gefjun


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
slide1

Life Science Informatics Group

Introduction to Meta-Analysis

Christopher H. Schmid, PhD

Tufts-New England Medical Center

6 June 2008

Grant Stephen: Chair of the MBC Life Science Informatics Group

& CEO, Tessella Inc: grant.stephen@tessella.com

Creating Insight & Understanding from Scientific Data

the weight of medical knowledge

The weight of medical knowledge

Weight of the Index Medicus According to 10-Year Periods from 1879 to 1977

introduction
Introduction
  • Medicine requires hard evidence, often clinical trials
  • Evidence inconsistent across studies

Different study populations

Different treatments or protocols

Quality of technical design or execution

Random variation

reasons for combining data
Reasons for combining data
  • Get an overall estimate of treatment effect
  • Appreciate the degree of uncertainty
  • Appreciate heterogeneity
  • Forces you to think rigorously about the data
meta analysis
Meta-Analysis
  • A scientific discipline which applies a protocol to critically evaluate and uses statistical methods to combine the results of (previous) research
  • Provides a quantitative summary of the overall treatment effect (typically an overall estimate and confidence interval)
  • Increasingly used to understand differences among studies to explain discrepancies of results and to generate hypotheses of interactions
  • To guide future research
some terms used for meta analysis
Some terms used for meta-analysis
  • Systematic Review
  • Overview
  • Quantitative overview
  • Research (evidence) synthesis
  • Research Integration
  • Pooling (implies lumping data altogether)
  • Combining (implies performing procedures on data)
slide7
No one will criticize you for doing a systematic review. But as soon as you combine data, you will most likely have controversy
why is that
Why is that?
  • Apples and oranges (heterogeneity)
  • Garbage in, garbage out (quality)
  • Selection of outcomes (soft or hard)
  • Selection of studies
  • Publication bias
  • Assumptions used in quantifying results
slide9

Characteristics of Meta-Analyses

  • More heterogeneity than multicenter trial
  • Meta-analysis addresses random variation
  • Gives pooled estimate of treatment effect
  • Can be confirmatory or exploratory
  • Pooling may not be best solution
  • Need to explain variation
systematic review protocol
Systematic review protocol
  • Well-focused study question
  • Identification of studies (design, source, search strategy)
  • Eligibility criteria (study, patient, and disease characteristics, treatments, outcomes)
  • Data extraction (definition of outcomes, quality assessment)
  • Data summary and analysis (outcomes, intention to treat)
issues in formulating a question
Issues in formulating a question
  • Narrow versus broad (for individuals/ subgroups or entire population)
  • Scientifically meaningful and useful (based on sound biological and epidemiological principles)
  • Very broadly defined questions may be criticized for mixing apples and oranges
  • Narrowly focused questions have limited generalizability and may lead to biased conclusions
literature processing
Literature processing
  • Questions
  • Search strategy
  • Screening of abstracts
  • Retrieve potential articles
  • Screen full articles
  • Data extraction on qualifying articles
issues in finding and retrieving evidence
Issues in Finding and Retrieving Evidence
  • Search strategies
  • Sources
  • Language selection
  • Published vs. unpublished literature
  • Use of abstracts
  • File drawer problem (publication bias)
  • Multiple publications on same subjects
  • Disproportionate amount of data for topic
types of multiple publications
Types of Multiple Publications
  • Overlapping data (preliminary and later reports)
  • Same data but different authors
  • Similar data (same authors) but different cohort
    • need to verify with authors
slide17
Most meta-analyses are retrospective exercises, suffering from all the problems of being an observational design. We cannot fix bad data.
basic principles in combining data
Basic principles in combining data
  • For each analysis, one study should contribute only one effect
  • Effect may be single outcome or composite of several outcomes
  • Effect being combined should be same or similar across studies
what kinds of control
What kinds of control?
  • No treatment control
  • Placebo
  • Active comparator
types of data that could be combined
Types of data that could be combined
  • dichotomous (events, e.g. deaths)
  • measures (odds ratios, correlations)
  • continuous data (mmHg, pain scores)
  • survival curves
  • diagnostic test (sensitivity, specificity)
  • individual patient data
  • “effect size”
issues in choosing method to combine studies
Issues in choosing method to combine studies
  • Metrics
  • Fixed vs. random effects model
  • Treatment effect heterogeneity
  • Baseline rate heterogeneity
  • Weight
heterogeneity diversity
Heterogeneity (diversity)
  • Is it reasonable (are studies and effects sufficiently similar) to estimate an average effect?
  • Types of heterogeneity
    • Conceptual (clinical) heterogeneity
    • Statistical heterogeneity
conceptual scientific heterogeneity

Conceptual (scientific) Heterogeneity

Are the studies of similar treatments, populations, settings, design, etc., such that an average effect would be scientifically meaningful?

statistical heterogeneity

Statistical Heterogeneity

Is the observed variability of effects greater than that expected by chance alone?

meta analytic approaches
Meta-analyticapproaches
  • Summary point estimation (random or fixed effect model)
  • Meta-regression - modeling aggregate data heterogeneity
  • Baseline risk meta-regression
  • Response surface - individual patients’ data analysis
sensitivity analyses
Sensitivity analyses
  • Exclude studies
  • Analyze subgroups
  • Change assumptions
  • Use different metric
  • Compare fixed versus random effects model
  • Perform cumulative meta-analysis
dichotomous outcomes
Dichotomous outcomes
  • Binary outcomes, event or no event, yes or no
  • Most common type of outcome reported in clinical trials (about 70%)
  • Examples: dead/alive, stroke/no stroke, cure/failure
  • 2x2 tables commonly used to report their results
  • Sometimes continuous variables are forced into dichotomous outcomes.
    • E.g., a threshold could be used for pain scores and reported as improved or not improved.
available metrics for combining dichotomous outcome data
Available metrics for combining dichotomous outcome data
  • Odds ratio (OR)
  • Risk ratio (RR)
  • Risk difference (RD)
  • NNT (Number Needed to Treat) can be derived (inverse of the combined risk difference) = 1/RD
calculating treatment effects in isis 2
Calculating treatment effects in ISIS-2

TR = 791/8592 = 0.0921

CR = 1029 / 8595 = 0.1197

RR = 0.0921 / 0.1197 = 0.77

OR = (791 x 7566) / (1029 x 7801) = 0.75

RD = 0.0921 – 0.1197 = -0.028

slide36

Beta-Blockers after Myocardial Infarction - Secondary Prevention

Experiment Control Odds 95% CI

N Study Year Obs Tot Obs Tot Ratio Low High

=== ============ ==== ====== ====== ====== ====== ===== ===== =====

1 Reynolds 1972 3 38 3 39 1.03 0.19 5.45

2 Wilhelmsson 1974 7 114 14 116 0.48 0.18 1.23

3 Ahlmark 1974 5 69 11 93 0.58 0.19 1.76

4 Multctr. Int 1977 102 1533 127 1520 0.78 0.60 1.03

5 Baber 1980 28 355 27 365 1.07 0.62 1.86

6 Rehnqvist 1980 4 59 6 52 0.56 0.15 2.10

7 Norweg.Multr 1981 98 945 152 939 0.60 0.46 0.79

8 Taylor 1982 60 632 48 471 0.92 0.62 1.38

9 BHAT 1982 138 1916 188 1921 0.72 0.57 0.90

10 Julian 1982 64 873 52 583 0.81 0.55 1.18

11 Hansteen 1982 25 278 37 282 0.65 0.38 1.12

12 Manger Cats 1983 9 291 16 293 0.55 0.24 1.27

13 Rehnqvist 1983 25 154 31 147 0.73 0.40 1.30

14 ASPS 1983 45 263 47 266 0.96 0.61 1.51

15 EIS 1984 57 858 45 883 1.33 0.89 1.98

16 LITRG 1987 86 1195 93 1200 0.92 0.68 1.25

17 Herlitz 1988 169 698 179 697 0.92 0.73 1.18

general weighted average effect size
General Weighted Average Effect Size

where: di = effect size of the ith study

wi = weight of the ith study

k = number of studies

calculation of weights
Calculation of weights
  • Generally the inverse of the variance of treatment effect
  • Different formula for odds ratio, risk ratio, risk difference
  • Readily available in books and software
effect size
Effect Size
  • Dimensionless metric
  • Combine standard deviations of diverse types of related effects
  • Availability and selection of reported effects may be biased
  • Variable importance of different effects
  • Frequently used in education, social science literature
  • Difficult to interpret results
statistical models of pooling 2x2 tables
Statistical Models of Pooling 2x2 Tables
  • Fixed Effect Model
  • Random Effect Model
slide44
David Bowers. Statistics from scratch. An introduction for Health Care Professionals. John Wiley & Sons, 1996.
random sampling from container with fixed number of white and black balls different sample size
Random sampling from container with fixed number of white and black balls (different sample size)
summary point estimation
Summary point estimation

Principle - common truth

Main Model - fixed effects weighted average

Advantages - easy to interpret, applies to whole population

Disadvantages - often simplistic; not applicable with heterogeneity

summary point estimation1
Summary point estimation
  • Principle - range of truth
  • Main Model - random effects weighted average
  • Advantages – realistic; allows between-study heterogeneity
  • Disadvantages - less intuitive; no insight into heterogeneity
chi square homogeneity test
Chi-Square Homogeneity Test

NOTE: d = ln(ORi d+ = ln(ORMH) wi = 1/variance (ORi)

Variance (ORi) = 1/ai + 1/bi + 1/ci + 1/di

findings of cumulative meta analysis
Findings of Cumulative Meta-analysis
  • Clinical experts’ recommendations often are unreliably synchronized with developing RCT evidence.
  • Large clinical trials often echo findings from meta-analyses of several smaller studies.
  • Trends established by cumulative meta-analyses of previous studies are unlikely to be reversed.
slide60

Bayesian Models

  • Model parameters are random variables, not unknown constants
  • Probability distribution quantifies this uncertainty
    • Prior before collecting data
    • Posterior after observing data
    • Likelihood is probability of observing data under model
  • Statements about probability of hypotheses and parameter values
  • Obtained by combining prior and likelihood with Bayes rule
slide61

Advantages of Bayesian Models

  • Quantify knowledge of parameters
  • Incorporate all sources of variation in single model
  • Inference not restricted just to model parameters
  • Can be generalized to nonnormal priors and likelihoods
  • (e.g. binomial)
  • Posterior densities need not be normally distributed
  • like maximum likelihood estimates
slide62

Magnesium for AMI

  • Infamous because random effects and fixed effects analysis lead to different conclusions
    • Random effects OR = 0.59
    • Fixed effects OR = 1.02
  • Very large, influential clinical trial showed treatment gave no benefit
  • Contradicted earlier MA with large trial showing significant benefit
slide63

Meta-analysis for Magnesium Studies

          • Pooled Odds Ratio
          • Mortality Observed Posterior
  • Study Treated Control Est 95% PI Est 95% PI Pr(OR<1)
  • Morton 1/40 2/36 0.44 0.0, 5.0 0.54 0.2, 1.6 0.89
  • Smith 2/200 7/200 0.28 0.1, 1.5 0.46 0.1, 1.1 0.96
  • Abraham 1/48 1/46 0.96 0.1, 15.8 0.61 0.2, 1.9 0.84
  • Feldstedt 10/150 8/148 1.25 0.5, 3.3 0.86 0.4, 1.9 0.70
  • Rasmussen 9/135 23/135 0.35 0.2, 0.8 0.43 0.2, 0.9 0.99
  • Ceremuz. 1/25 3/23 0.28 0.0, 2.9 0.49 0.1, 1.4 0.92
  • Shechter I 1/59 9/56 0.09 0.0, 0.7 0.38 0.1, 1.4 0.97
  • LIMIT 2 90/1159 118/1157 0.74 0.6, 1.0 0.73 0.6, 1.0 0.99
  • ISIS-4 2216/29011 2103/29039 1.06 1.0, 1.1 1.06 1.0, 1.1 0.04
  • Shechter II 2/89 12/80 0.13 0.0, 0.6 0.36 0.1, 0.9 0.99
  • Singh 6/76 11/75 0.50 0.2, 1.4 0.54 0.2, 1.1 0.95
  • Pooled 0.59 0.4, 0.9 0.55 0.3, 0.9 0.99
indirect comparisons of multiple treatments
Indirect Comparisons of Multiple Treatments

Trial

1 A B

2 A B

3 B C

4 B C

5 A C

6 A C

7 A B C

We want to compare A vs. B

Direct evidence from trials 1, 2 and 7

Indirect evidence from trials 3, 4, 5, 6 and 7

slide66

Meta-Regression

  • Regression analysis to identify correlations between treatment effects (outcomes) and covariates of interest (predictors)
  • Unit of analysis is the individual study
  • Correlation implies treatment interaction
  • Factors may be study-level or subject-level
  • Study-level factors include blinding, randomization, dosage, protocol
  • Subject-level factors include age, gender, race, blood pressure
        • i = 0 + 1Xi1 + 2Xi2 + … + ui
slide67

Sources of Heterogeneity

  • Different study populations
  • Different treatments or protocols
  • Quality of technical design or execution
  • Random variation
problems with meta regression
Problems with Meta-Regression
  • Number of studies usually small
  • Data may be unavailable (not conceived or not reported)
  • Covariates pre-selected (biased?)
  • Little variation in range of mean predictor
  • Subject-level factors can be affected by ecological bias
  • Causality uncertain
slide69

Patient Level Analysis

  • Principle: risk factors differ between patients
  • Main model: multivariate regression of individual patient data
  • Advantages
    • maximum information using patient as unit of analysis
    • direct interpretation for individual patient
    • no reporting bias
    • no ecologic bias
  • Disadvantages
    • data difficult to obtain and frequently unavailable
    • retrieval bias
    • causality uncertain
    • costly
meta regression vs individual patient regression
Meta-Regression vs. Individual Patient Regression

Meta-Regression Individual Patient Regression

Cost cheap expensive

Factors study patient and study

Outcomes reported updated, complete

Data Cleaning not possible reconciliation, missing data

Bias publication data retrieval, publication