**1. **How to Conduct a Meta-Analysis Arindam Basu MD MPH
About the Author
Required Browsing

**2. **Objectives Following the course, you will be able to:
Define Meta-analysis
Select Studies for a Meta-analysis
Identify different types of Models
Calculate Summary Effects
Interpret Results of a Meta-analysis
This is an introductory tutorial on how to quickly search, review and analyze informaiton contained in health care literature. You can conduct meta-analysis at a number of levels: from a full scale study to for very personal reasons to summarize results across studies. Generally has four steps:
Develop a research question and identify studies
Select Studies
Select between fixed effects model and random effects model
Calculate a summary effect and interpret the results in the light of findings.
In this tutorial, it is assumed that you already have a research question in mind, and you are familiar with using spreadsheets. This is an introductory tutorial on how to quickly search, review and analyze informaiton contained in health care literature. You can conduct meta-analysis at a number of levels: from a full scale study to for very personal reasons to summarize results across studies. Generally has four steps:
Develop a research question and identify studies
Select Studies
Select between fixed effects model and random effects model
Calculate a summary effect and interpret the results in the light of findings.
In this tutorial, it is assumed that you already have a research question in mind, and you are familiar with using spreadsheets.

**3. **What is Meta-analysis? Synthesis of previous studies
Providing a Summary estimate
Steps
Identify studies
Define Eligibility Criteria
Abstract Data
Do Statistical Analysis
Meta-analysis is essentially synthesis of available literature about a topic. Ideally, synthesis of randomized trials to arrive at a single summary estimate is used. Other types of studies are used for meta-analysis as well.
Begin with a systematic process of identifying your studies. Search your personal files and computerized databases, for example, Medline. Write down the search strategy.
After identifying all studies, define the ones you want to keep for the meta-analysis. This will help another researcher faced with the same body of literature applying the same criteria to find and work with the same studies.
Now create structured formats to key in information taken from the selected studies.
Finally, combine the data to arrive at a summary estimate of the effect, it?s 95% confidence interval, and a test of homogeneity of the studies. Meta-analysis is essentially synthesis of available literature about a topic. Ideally, synthesis of randomized trials to arrive at a single summary estimate is used. Other types of studies are used for meta-analysis as well.
Begin with a systematic process of identifying your studies. Search your personal files and computerized databases, for example, Medline. Write down the search strategy.
After identifying all studies, define the ones you want to keep for the meta-analysis. This will help another researcher faced with the same body of literature applying the same criteria to find and work with the same studies.
Now create structured formats to key in information taken from the selected studies.
Finally, combine the data to arrive at a summary estimate of the effect, it?s 95% confidence interval, and a test of homogeneity of the studies.

**4. **Identify Studies: Overview Steps:
Search Your Personal Files First
Search Electronic Databases
Review Reference Lists of Articles
Contact Experts and Researchers
Retrieve New Articles
Evaluate Quality of the Studies
Set Up Eligibility Criteria
Begin your search in your personal files archive before moving on to electronic databases. First, read their titles and abstracts. Eliminate those studies that do not match with your requirements. Then, for the remaining articles, read their reference lists, retrieve the abstracts of articles you think are important to your research, and continue the process. You should aim to retrieve all relevant articles. You also need to search experts and researchers working in the area of your interest. It?s a good idea to search newsgroups and mailing lists, in addition to contacting individual authors.
Begin your search in your personal files archive before moving on to electronic databases. First, read their titles and abstracts. Eliminate those studies that do not match with your requirements. Then, for the remaining articles, read their reference lists, retrieve the abstracts of articles you think are important to your research, and continue the process. You should aim to retrieve all relevant articles. You also need to search experts and researchers working in the area of your interest. It?s a good idea to search newsgroups and mailing lists, in addition to contacting individual authors.

**5. **Searching Electronic Databases First, Define a Search Strategy
Limitations of Databases
incomplete and imperfect queries
language problems
problems with fugitive literature
Publication Bias is important
What is publication bias
How to deal with publication bias
Identify documents in electronic databases by searching for words that appear in the title, in the abstract or in the body of the text. Medline can also be searched by using the name of the author or the source journals.
Remember, records retrieved from electronic databases depend both on the structure of these databases and the way you query the database. Also, if you limit the studies to only those published in english language, you will leave out important studies in other languages. Finally, several documents are published in dissertation theses, or government documents which are not archived in these databases. These are oftent termed as fugitive literature.
Publication Bias refers to the greater likelihood of research with statistically significant results to be reported in comparison to those with null or nonsignificant results.
Acknowledge the problem in the study report
Try to retrieve all studies
Construct a Funnel Plot with effect size in the X axis and Sample Size in the Y axis. If the plot resembles a funnel with base down, shows that publication bias is minimum.Identify documents in electronic databases by searching for words that appear in the title, in the abstract or in the body of the text. Medline can also be searched by using the name of the author or the source journals.
Remember, records retrieved from electronic databases depend both on the structure of these databases and the way you query the database. Also, if you limit the studies to only those published in english language, you will leave out important studies in other languages. Finally, several documents are published in dissertation theses, or government documents which are not archived in these databases. These are oftent termed as fugitive literature.
Publication Bias refers to the greater likelihood of research with statistically significant results to be reported in comparison to those with null or nonsignificant results.
Acknowledge the problem in the study report
Try to retrieve all studies
Construct a Funnel Plot with effect size in the X axis and Sample Size in the Y axis. If the plot resembles a funnel with base down, shows that publication bias is minimum.

**6. **Evaluating Study Quality Define Study Quality Criteria Early
Set Up A Good Scoring System
Develop A Form for Assessment
Calculate Quality for each Study
Use this for Sensitivity Analysis
stratify studies according to quality
Decide which studies you want to include based on the quality of the research reported in them. Decide the study quality standards early on, preferably before beginning the analysis. A standardized form helps a lot. Give a score to each study and analyze the studies based on the quality scores of the studies.Decide which studies you want to include based on the quality of the research reported in them. Decide the study quality standards early on, preferably before beginning the analysis. A standardized form helps a lot. Give a score to each study and analyze the studies based on the quality scores of the studies.

**7. **Defining Eligibility of Studies Select Eligible Studies Based On:
Study Designs
Years of Publication
Language
Choice among multiple articles
Sample-size or follow-up issues
Similarity of Exposure and/or Rx
Completeness of information
If you are reviewing both randomized and non-randomized studies in the same meta-analysis, report the effect sizes separately for the RCTS and non-randomized studies. This is because the effect of a new treatment is likely to be larger in studies that employ a non-randomized design.
Try to be as up-to-date as possible. Mention the cut-off dates in the analysis so that it becomes clear that studies that were published before and after the time mentioned were not missed but not included in the study as part of the design. If you are including only English language articles in the study, mention the rationale for doing so.
Importantly, if you are faced with multiple reports published from the same study on similar topic, include only one study so that information from the same study population should contribute ONLY ONCE to the analysis.
Try to exclude studies with small sample size to avoid overemphasizing small studies. Similarly, make an early decision about the period of follow up of studies. More of these in the sensitivity analysis.
Decide early on whether you are going to include studies with similar exposures or outcomes in your analysis. Base this decision on your experience and understanding of the research question. Remember, if you are too restrictive, you gain face validity, though you may end up omitting important studies. If you are reviewing both randomized and non-randomized studies in the same meta-analysis, report the effect sizes separately for the RCTS and non-randomized studies. This is because the effect of a new treatment is likely to be larger in studies that employ a non-randomized design.
Try to be as up-to-date as possible. Mention the cut-off dates in the analysis so that it becomes clear that studies that were published before and after the time mentioned were not missed but not included in the study as part of the design. If you are including only English language articles in the study, mention the rationale for doing so.
Importantly, if you are faced with multiple reports published from the same study on similar topic, include only one study so that information from the same study population should contribute ONLY ONCE to the analysis.
Try to exclude studies with small sample size to avoid overemphasizing small studies. Similarly, make an early decision about the period of follow up of studies. More of these in the sensitivity analysis.
Decide early on whether you are going to include studies with similar exposures or outcomes in your analysis. Base this decision on your experience and understanding of the research question. Remember, if you are too restrictive, you gain face validity, though you may end up omitting important studies.

**8. **Abstract Data - Review! Steps:
Identify Relevant Articles
Sort out Eligible Articles
Set up a Form for Abstraction
Enter the Eligible Studies
Use this as your database
Statistical Analysis is next...
Before you proceed on to the statistical analyses, make sure you have the following:
All your relevant articles on hand.
You have prepared a format to enter the studies.
You have tabulated the results of all eligible studies systematically.
Now proceed with the statistical analysis.Before you proceed on to the statistical analyses, make sure you have the following:
All your relevant articles on hand.
You have prepared a format to enter the studies.
You have tabulated the results of all eligible studies systematically.
Now proceed with the statistical analysis.

**9. **Statistical Analysis - Overview Select An Estimate of Effect
Choose An Effects Measure
Select An Effects Model
For Each Model:
Calculate Summary Effect Size
Calculate Confidence Intervals
Calculate Q-statistic for Homogeneity
Perform Sensitivity Analysis
The purpose of conducting a statistical analysis is to determine a summary estimate of effect. The effect measures could be rate difference, relative risk estimate (e.g. an odds? ratio or relative risk) or a rate ratio. Selecting an effects model is important. Usually the choice is between fixed effects model, which indicates that the conclusions derived in the meta-analysis are valid for the studies included in the analysis, and random effects model, which assume that the studies included in the meta-analysis belong to a random sample of a universe of such studies. When the studies are found to be homogeneous, random and fixed effects models are indistinguishable. Finally, conduct a sensitivity analysis to test that the model you propose is a good model. The purpose of conducting a statistical analysis is to determine a summary estimate of effect. The effect measures could be rate difference, relative risk estimate (e.g. an odds? ratio or relative risk) or a rate ratio. Selecting an effects model is important. Usually the choice is between fixed effects model, which indicates that the conclusions derived in the meta-analysis are valid for the studies included in the analysis, and random effects model, which assume that the studies included in the meta-analysis belong to a random sample of a universe of such studies. When the studies are found to be homogeneous, random and fixed effects models are indistinguishable. Finally, conduct a sensitivity analysis to test that the model you propose is a good model.

**10. **Selecting Estimate of Effect Choose Only One Estimate
For RCTS, choose the one with
Once randomized always randomized
For nonrandomized trials, choose:
estimate adjusted only for age
that, and for a known confounder
the ?most adjusted? estimate
estimate presented in the abstract
Usually most studies come with many estimates of effects and sub-analyses. Which ones should you accept? If you are studying randomized control trials (RCTs), take only that estimate where the rule of randomization is strictly followed. All other effect estimates are not important. For studies that use non-randomized study designs, look for those estimates that are adjusted for age. For models that include more than one independent variable, take the estimate where age and additionally, effect of a well known confounders (on theoretical grounds) are used for adjustment. Alternatively, use the effect measure in a model where most variables are used for adjustment. A good rule of the thumb is to use the effect measure that is presented in the abstract.Usually most studies come with many estimates of effects and sub-analyses. Which ones should you accept? If you are studying randomized control trials (RCTs), take only that estimate where the rule of randomization is strictly followed. All other effect estimates are not important. For studies that use non-randomized study designs, look for those estimates that are adjusted for age. For models that include more than one independent variable, take the estimate where age and additionally, effect of a well known confounders (on theoretical grounds) are used for adjustment. Alternatively, use the effect measure in a model where most variables are used for adjustment. A good rule of the thumb is to use the effect measure that is presented in the abstract.

**11. **Choosing An Effect Measure RCTs or Cohort Studies
Rate Difference between Treatment and Control Groups
Ratio of Disease Rates
Case Control Studies
Odds? Ratio
Rate Ratio
For Randomized Control Trials or Cohort Studies, difference in the rates of the disease in the treatment group (or the group subjected to exposure) and that in the control group is usually measured. The rates could be cumulative incidence rates or incidence-density rates. Use Odds? Ratio for case control studies.For Randomized Control Trials or Cohort Studies, difference in the rates of the disease in the treatment group (or the group subjected to exposure) and that in the control group is usually measured. The rates could be cumulative incidence rates or incidence-density rates. Use Odds? Ratio for case control studies.

**12. **Selecting An Effects Model Available Types:
Fixed Effects Model
Random Effects Model
Difference Between the Two
Special Cases:
When Outcomes are not binary
Methods to be Used for them
Fixed Effects Model: Answers the question whether the studies included in the meta-analysis show that the treatment or exposure produced the effect on average
Random Effects Model: Answers the question, on the basis of the studies that are examined, is it possible to comment that the treatment or the exposure will produce a result?
A random effects model is computationally more intense than a fixed effects model. Also, if the studies are homogeneous, fixed effects and random effects models are similar.
In this tutorial, we will also examine special situations when the outcomes are on a continuous scale, rather than discrete counting of events.
Finally, check out this table about the methods to be used for different types of models. Fixed Effects Model: Answers the question whether the studies included in the meta-analysis show that the treatment or exposure produced the effect on average
Random Effects Model: Answers the question, on the basis of the studies that are examined, is it possible to comment that the treatment or the exposure will produce a result?
A random effects model is computationally more intense than a fixed effects model. Also, if the studies are homogeneous, fixed effects and random effects models are similar.
In this tutorial, we will also examine special situations when the outcomes are on a continuous scale, rather than discrete counting of events.
Finally, check out this table about the methods to be used for different types of models.

**13. **Fixed Effects Model Methods:
Mantel Haenszel Method
Peto?s Method
General Variance Based Methods
For Rate Difference
For Rate Ratios
When only RR and 95 CI given
Tests of Homogeneity
Calculation of Q Statistic Both Mantel Haenszel and Peto?s methods are popularly used. Mantel-Haenszel Method is a method of stratified analysis of data. In Meta-analysis, this is used to pool together the effects of individual studies. Peto?s Method is similar to Mantel Haenszel?s but computationally simpler.
Use Calculators for General Variance Based Methods for Rate Difference and for the situation when only Relative Risk Estimate and 95% Confidence Interval are provided. Both Mantel Haenszel and Peto?s methods are popularly used. Mantel-Haenszel Method is a method of stratified analysis of data. In Meta-analysis, this is used to pool together the effects of individual studies. Peto?s Method is similar to Mantel Haenszel?s but computationally simpler.
Use Calculators for General Variance Based Methods for Rate Difference and for the situation when only Relative Risk Estimate and 95% Confidence Interval are provided.

**14. **Mantel Haenszel Method Download Spreasheet Calculator
Strength of Mantel Haenszel
Very powerful
Widely Used
Limitation
Cannot Control For Confounding!
The Mantel Haenszel Method is a widely used method to calculate stratified summary effects. In this tutorial, a spreadsheet calculator is provided to calculate the summary effect from a group of studies. Ideal for a variety of study designs including RCTs. However, calculation of summary effect using the Mantel Haenszel Method needs a data matrix that can be entered into a 2*2 matrix. Besides, it cannot control for the effects of confounding factors. The Mantel Haenszel Method is a widely used method to calculate stratified summary effects. In this tutorial, a spreadsheet calculator is provided to calculate the summary effect from a group of studies. Ideal for a variety of study designs including RCTs. However, calculation of summary effect using the Mantel Haenszel Method needs a data matrix that can be entered into a 2*2 matrix. Besides, it cannot control for the effects of confounding factors.

**15. **Peto?s Method Similar to Mantel Haenszel
Download Calculator
Simpler Computation
No Control for Confounding
Good for RCTs
Requires 2 X 2 Table
Peto?s method is very similar to Mantel Haenszel method, except is?s computationally simpler. Like Mantel Haenszel, it requires a 2*2 table for computation of summary effect. It cannot control for confounding. Peto?s method is very similar to Mantel Haenszel method, except is?s computationally simpler. Like Mantel Haenszel, it requires a 2*2 table for computation of summary effect. It cannot control for confounding.

**16. **Variance-Based Methods Download Calculators For:
Rate Difference
For only Relative Risk and 95% CI
Strengths and Limitations
Good For Rate Differences
Computationally Intensive Two separate calculators are provided for estimating the summary effects using the variance based methods. The first one can be used in the situation where difference in the rates of even occurrence is used as an effect measure. The second calculator can be used to calculate a summary relative risk estimate when, for all the studies listed, only relative risk estimate (e.g. odds? ratio or relative risk) is provided with 95% confidence interval. Two separate calculators are provided for estimating the summary effects using the variance based methods. The first one can be used in the situation where difference in the rates of even occurrence is used as an effect measure. The second calculator can be used to calculate a summary relative risk estimate when, for all the studies listed, only relative risk estimate (e.g. odds? ratio or relative risk) is provided with 95% confidence interval.

**17. **Tests of Homogeneity Establish Null Hypothesis that Effect Sizes Are Equal in All of the Studies [FAIL TO REJECT NULL]
Tested By Using Q-statistic
Q-statistic is distributed as chi-square distribution with degree of freedom = n-1 where n = number of studies
A statistic that is particularly useful for assessing whether the studies are all homogenous is the Q-statistic. The hypotheses we test here are:
The null hypothesis states that the studies are homogeneous
The purpose of this test is to see if the null hypothesis holds. For each model, Q-statistic is calculated. The Q-statistic has a chi-square distribution with degree of freedom N where N = (Number of Studies - 1).
For homogeneous studies, there is essentially no difference between a fixed effects model and the random-effects model.A statistic that is particularly useful for assessing whether the studies are all homogenous is the Q-statistic. The hypotheses we test here are:
The null hypothesis states that the studies are homogeneous
The purpose of this test is to see if the null hypothesis holds. For each model, Q-statistic is calculated. The Q-statistic has a chi-square distribution with degree of freedom N where N = (Number of Studies - 1).
For homogeneous studies, there is essentially no difference between a fixed effects model and the random-effects model.

**18. **Random Effects Model Download The Calculator!
Strengths and Limitations:
Can Generalize the Conclusions
Computationally Intensive Random effects model is based on the assumption that the studies are part of a large universe of similar studies. The relative risk estimate can be used to answer the question that will the intervention or the exposure be associated with the outcome studied (compare this with fixed effects model where the question was: did the studies show on average that the exposure or treatment associated with the outcome of interest?). However, a random effects model is computationally more intensive than a fixed effects model.Random effects model is based on the assumption that the studies are part of a large universe of similar studies. The relative risk estimate can be used to answer the question that will the intervention or the exposure be associated with the outcome studied (compare this with fixed effects model where the question was: did the studies show on average that the exposure or treatment associated with the outcome of interest?). However, a random effects model is computationally more intensive than a fixed effects model.

**19. **Continuous Outcomes Measurement Scale: Continuous
Outcome Measured in Same Scale
Download Spreadsheet Calculator!
Essentially Extension of ANOVA
Useful For Integrating Social Science Research Data For several measures, for example blood pressure, or biochemical estimates or scale scores, a binary value for the outcome (present/absent, or yes/no) is not applicable. It helps if one can measure the effects in a continuous scale. This is particularly useful for several different kinds of medical and social science data. Use this spreadsheet to calculate the summary effects. You may note that in cases where outcome measures of all the studies are measured in the same unit, finding the summary effect is more like a one-way analysis of variance procedure with studies as independent variable.For several measures, for example blood pressure, or biochemical estimates or scale scores, a binary value for the outcome (present/absent, or yes/no) is not applicable. It helps if one can measure the effects in a continuous scale. This is particularly useful for several different kinds of medical and social science data. Use this spreadsheet to calculate the summary effects. You may note that in cases where outcome measures of all the studies are measured in the same unit, finding the summary effect is more like a one-way analysis of variance procedure with studies as independent variable.

**20. **Sensitivity and Publication Bias Conduct Sensitivity Analysis
Stratify studies by their quality rating
Compare Fixed and Random Effects
Deal With Publication Bias
Construct A Funnel Plot
Based on the quality of the studies you selected, divide them into two or more groups. Then, conduct separate analyses for each group. Also, compare the results of fixed and random effects model for the studies. A funnel plot will help you to identify presence of publication bias. In the absence of publication bias, if a large number of studies are compared, the distribution of the studies in the funnel plot resembles a funnel looked at sideways with the wider end down. In the presence of publication bias, either end of the base of the funnel will appear truncated. Based on the quality of the studies you selected, divide them into two or more groups. Then, conduct separate analyses for each group. Also, compare the results of fixed and random effects model for the studies. A funnel plot will help you to identify presence of publication bias. In the absence of publication bias, if a large number of studies are compared, the distribution of the studies in the funnel plot resembles a funnel looked at sideways with the wider end down. In the presence of publication bias, either end of the base of the funnel will appear truncated.

**21. **Conclusion Presentation of Meta-analysis or Literature Synthesis
Download The Graph Maker
More Resources... The results of meta-analysis is presented in the form of graphs (graphs are plotted with studies against relative risk estimates). In this tutorial, a small spreadsheet application is presented to construct graphs for the presentation of the results of the meta-analysis. Download the graph maker.
This tutorial presents a general introduction to the process of meta-analysis. The web is replete with software and tutorials on meta-analysis. Some of them are listed in the following page. The results of meta-analysis is presented in the form of graphs (graphs are plotted with studies against relative risk estimates). In this tutorial, a small spreadsheet application is presented to construct graphs for the presentation of the results of the meta-analysis. Download the graph maker.
This tutorial presents a general introduction to the process of meta-analysis. The web is replete with software and tutorials on meta-analysis. Some of them are listed in the following page.