Meta analysis the art and science of combining information
Download
1 / 46

Meta-Analysis: The Art and Science of Combining Information - PowerPoint PPT Presentation


  • 79 Views
  • Uploaded on

Ora Paltiel, October 28, 2014. Meta-Analysis: The Art and Science of Combining Information. DEFINITIONS. The statistical analysis of a large collection of results from individual studies for the purpose of integrating the findings

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about ' Meta-Analysis: The Art and Science of Combining Information' - griffith-newton


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Meta analysis the art and science of combining information

Ora Paltiel, October 28, 2014

Meta-Analysis: The Art and Science of Combining Information


Definitions
DEFINITIONS

  • The statistical analysis of a large collection of results from individual studies for the purpose of integrating the findings

  • A quantitative review and synthesis of results of related but independent studies

  • “overview”

  • “data pooling”

  • “data synthesis”

  • systematic review


“Meta “

  • Webster’s dictionary:

    a) occurring later than or in succession to

    b) situated behind or beyond

    c) change, transformation

    Examples: metaphysics,

    metamorphosis.


Over 2 million medical articles are published each year

The findings of new studies not only “ differ from previously established truths but disagree with one another, often violently”

-Morton Hunt, How Science Takes Stock, P.1

The Problem

Over 2 million medical articles are published each year.


The goal of meta analysis making order of scientific chaos
The Goal of Meta-Analysis: previously established truths but disagree with one another, often violently”“Making Order of Scientific Chaos”

  • Began as a tool in Social Sciences

    21 citations in 1986

    431 citations in 1991

    more than 45000 today

    In Medicine – at first only RCTs

    Now – thousands of meta-analyses of observational studies


  • is a group of over 15,000 volunteers in previously established truths but disagree with one another, often violently”

    more than 90 countries who review the

    effects of health care interventions tested

    in biomedicalrandomized controlled trials

  • reviews have also studied the results of

    non-randomizedobservational studies.

  • The results of thesesystematic reviewspublished as "Cochrane Reviews" in theCochrane Library

  • Founded in 1993 under the leadership of Iain Chalmers.

  • developed in response to Archie Cochrane's call for up-to-date, systematic reviews of all relevant randomized controlled trials of health care.


Cochrane collaboration
Cochrane collaboration previously established truths but disagree with one another, often violently”

Goal : to help people make well informed decisions about health care by preparing, maintaining and ensuring the accessibility of systematic reviews of the effects of health care interventions.

The principles of the Cochrane Collaboration are:

  • collaboration

  • building on the enthusiasm of individuals

  • avoiding duplication

  • minimizing bias

  • keeping up to date

  • striving for relevance

  • promoting access

  • ensuring quality

  • continuity

  • enabling wide participation


Major goals of meta analysis
Major goals of Meta-Analysis previously established truths but disagree with one another, often violently”

  • Objective summaries

  • Increase powerto detect true effects

  • Estimate effect size

  • Resolve uncertainty

  • Explore heterogeneity and reasons for itIf the studies produced dissimilar results, How did they differ? Why? Study design, quality, populations, subtle intervention differences etc

  • Tool for conducting evidence-based medicine and for setting policy


How to do a meta analysis
How to do a Meta-Analysis previously established truths but disagree with one another, often violently”

1. Define research question, including intervention, population, and outcome to be assessed

2. Define eligibility criteria (types of study, design)

3. Identify all studies (published or un) which deal with the specified problem

4. Evaluate each article for inclusion or exclusion, on the basis of predefined criteria

5. Summarize, numerically, the results of these studies

6. Interpret these findings, with emphasis on explaining differences as well as summarizing the data


Literature review
Literature review previously established truths but disagree with one another, often violently”

  • A comprehensive, systematic literature review should be conducted

  • Sources: citation indexes, abstract databases, clinical trials registers, references ,

  • Issues: language, “grey literature”, conference abstracts, unpublished findings

  • Meta-analysis is research, which should be reproducible, methods incl key words must be able to be replicated

    Problem of publication bias


SEARCH STRATEGY- example previously established truths but disagree with one another, often violently”

Horvath et al BMJ 2010;340:c1395


Information assembled
Information Assembled previously established truths but disagree with one another, often violently”

  • The report ( author, year)

  • The study (population)

  • The patients (demographic and clinical characteristics)

  • The design

  • The treatment

  • The effect size ( estimate , SE)

    Methods, reliability and validity of recording information need to be documented


Thirty three trials of streptokinase vs. conventional treatment for Acute Myocardial Infarction

  • “Head-Counting - Statistical”: Count the number of significant results in each direction Result: 6 favor treatment, 0 favor placebo, 27 nonsignificant

  • “Head-Counting”: Count the direction of the results in the studiesResult: 24 favor treatment, 9 favor placebo


Streptokinase summary
Streptokinase - Summary treatment for Acute Myocardial Infarction

  • Streptokinase reduces mortality by about 22%

  • Efficacy proven by 2 large RCTs in 1986 and 1988

  • Meta-analysis proved efficacy in 1971

  • 6380 lives could have been saved in large RCTs alone


What can we learn from the forest plot
What can we learn from the Forest Plot? treatment for Acute Myocardial Infarction

Meta-analysis of gestational diabetes outcomes – 1. Maternal

Horvath et al BMJ 2010;340:c1395


Statistical methods
Statistical Methods treatment for Acute Myocardial Infarction

  • We have a series of measures of association, one for each study

  • We wish to summarize these measures

  • This can be carried out using a weighted average of the estimates taken from each study.


Classic meta analysis
Classic Meta-Analysis treatment for Acute Myocardial Infarction

  • Analyzes RR, OR, or absolute differences in percentages between groups.

  • Uses the the inverse of the variance of the estimate provided by each participating trial for the weights. This gives a minimum variance unbiased estimate of the effect.

  • Large trials carry more weight than small trials.


Inference fixed v random effects
Inference: fixed .v. random effects treatment for Acute Myocardial Infarction

If interest is centered on making inferences for the populations that have been sampled, and we assume that there is a single effect of treatment - then a fixed effects approach is used.

In this approach the only source of uncertainty is that resulting from sampling patients into the studies. Variation stems

from within-study variation study.

The population to which we wish to generalized the results consists of a set of studies having identical characteristics


Random effects
Random-effects treatment for Acute Myocardial Infarction

  • In random-effects approach the existing studies are considered as a random sample from a population of studies

  • Random-effects approach is used when inferences are to be generalized to a population in which studies may differ in their effect and characteristics

  • Random effects approach integrate also the between-study variability


Fixed vs random effects
Fixed vs. Random-effects treatment for Acute Myocardial Infarction

  • The use of random-effects will produce somewhat larger 95% CI

  • A good practice is to first perform a test of heterogeneity between studies. If no significant variation is found between studies - a fixed-effects approach can be used

  • There are a number of ways to model random-effects


Heterogeneity
Heterogeneity treatment for Acute Myocardial Infarction

Horvath et al BMJ 2010;340:c1395


Sensitivity analysis comparators or control groups
Sensitivity analysis- treatment for Acute Myocardial Infarctioncomparators or control groups


Sensitivity analyses excluding studies with predefined less desirable characteristics as follows
Sensitivity analyses treatment for Acute Myocardial Infarctionexcluding studies with predefined less desirable characteristics, as follows:

Risk of bias

When the analysis was limited to two studies with a low risk of bias for random sequence generation and/or allocation concealment the add-on effect of acupuncture on patient-reported global assessment remained significant (RR 0.39, 95% CI 0.18–0.88, I2 = 0%).

Sample size

When four studies with ≥ 40 participants per group were pooled, there was no significant difference in the risk of symptoms persisting or worsening between the acupuncture and control groups (RR 0.50, 95% CI 0.24–1.05, I2 = 55%).


Assessing quality
Assessing Quality treatment for Acute Myocardial Infarction

A systematic approach should be used in order to assess the quality of the studies and to determine inclusion/exclusion of studies

Explicit methods limit bias in identifying and rejecting studies

Scales such as Jaddad scale


Domains to be assessed
Domains to be assessed treatment for Acute Myocardial Infarction

  • Methodological quality ( bias)

  • Precision in estimation

  • External validity





Further exploring heterogeneity
Further Exploring Heterogeneity text

  • In case of substantial heterogeneity between studies, exploring its causes can be performed by considering covariates on the study level that could ‘explain’ differences between studies.

  • Such analyses are called meta-regression



Publication bias. textSome studies are not published, selective presentation in those published.Do a comprehensive search. Use a funnel plot


Publication bias use of the funnel plot
Publication bias textuse of the funnel plot

1-SAMPLE SiZE



Conclusions
Conclusions text

  • In times of increasing amount of information-a systematic approach to synthesizing information has many advantages.

  • A systematic approach enables exploring heterogeneity between studies

  • As any other type of research systematic review should be carried out methodically and cautiously


Problems with meta analysis in real life
Problems with Meta-Analysis in Real Life text

  • “Meta-analysis” often not done, or very few studies combined

  • Retrospective study

  • Publication Bias

  • Heterogeneity


Future
Future text

  • Expect to see lots of meta-analyses

  • Good ones and bad ones

  • Scientific community will decide whether it is useful

    Be skeptical of everything




Robustness of results meta regression
Robustness of results-meta-regression text

an investigation of how a categorical study characteristic is associated with the intervention effects in the meta-analysis. For example, studies in which allocation sequence concealment was adequate may yield different results from those in which it was inadequate. Here, allocation sequence concealment, adequate /inadequate, is a categorical characteristic at the study level. MR in principle allows the effects of multiple factors to be investigated simultaneously (although this is rarely possible due to inadequate numbers of studies) (Thompson 2002). Meta-regression should generally not be considered when there are fewer than ten studies in a meta-analysis.

Meta-regressions are similar in essence to simple regressions, in which an outcomevariable is predicted according to the values of one or more explanatory variables. In meta-regression, the outcome variable is the effect estimate (for example, a mean difference, a risk difference, a log odds ratio or a log risk ratio). The explanatory variables are characteristics of studies that might influence the size of intervention effect


ad