1 / 12

Meta-Analysis Basics

Meta-Analysis Basics. Concepts and History. What is Meta-analysis?. Summary of quantitative results of empirical studies – data analytic part of a systematic review – data analysis where studies are data

inmanm
Download Presentation

Meta-Analysis Basics

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Meta-Analysis Basics Concepts and History

  2. What is Meta-analysis? • Summary of quantitative results of empirical studies – data analytic part of a systematic review – data analysis where studies are data • Can be used like any other statistical technique (e.g., to test a theory or moderator rather than to summarize) • Effect size – quantitative association of IV with DV • Model – an equation used to represent data • M-A Considers a distribution of effect sizes • Sort of an empirical sampling distribution • Mean of the distribution • Variance of the distribution • Accounting for variance in the distribution

  3. Common Objectives • Show (and interpret) the distribution • Find the mean • Find the variance • Explain the variance • Look for problems • Bias • Outliers • Sensitivity to decisions about data

  4. Goals for You • By the end of the class, you should be able to complete your own publishable meta-analysis. • Some students have published papers that came from this class: Allen, T.D., French K.A., Dumani, S., & Shockley, K.M. (2015). Meta-analysis of work-family conflict mean differences: Does national context matter? Journal of Vocational Behavior, 90, 90-100.

  5. Literature Review • Narrative – traditional, but less common now • Meta-analytic • Some do not distinguish meta-analysis from quantitative review. But meta-analysis is really just a kind of statistical analysis like regression. Meta-analysis uses ES, but regression uses individual data as observations. Most statisticians consider meta-analysis just data analysis where the studies have different magnitudes of errors.

  6. Appropriateness of M-A • Empirical vs. theoretical • Quantitative vs. qualitative • Study outcome (level 1) vs individual data (level 0) • Single IV and DV for ES (congeneric measures – corr b/t measures of same IV or same DV should approach 1.0) • Same or similar design (e.g., do not combine experiments and correlations)

  7. Common Effect Sizes • Correlation coefficient: r • Odds ratio (converted to logit): ln(OR) • Standardized mean difference: d

  8. Main Steps • Find studies (Effect Sizes) • Code ES (convert if necessary) • Code study characteristics (e.g., date of publication) • Calculate distribution of ES (garden variety meta-analysis) • Examine ‘moderators’ • Sensitivity analyses (data and model) • Draw conclusions • Share (present and publish)

  9. M-A Pros & Cons • Structured, open to replication • Superior to global impression & vote counting • Quantitative relations between outcome & study characteristics • Handles large k • Time & effort • Expertise in analyses • Missing data & bias • Exchangeability (apples & oranges) • Methodological rigor (all studies vs. ‘good’ studies)

  10. Example in R • This course is not about R, it is about meta-analysis • However, we will use R to do meta-analysis • R is free • R is fairly easy to use • R can save a cluster of related commands – repetition • The ‘metafor’ package in R is really good • Many different meta-analysis algorithms • Many different publication-quality graphs • Many diagnostics just for meta-analysis

  11. Dataset McDaniel • Metafor package contains several datasets. McDaniel1994 is a collection of studies in which employee interview scores were correlated with job performance measures. • The effect size is the correlation, r. N is the sample size. • There are also moderators for kind of interview • type = j, s, p – job related, situational, psychological • Structure = u, s – unstructured, structured • Download from Canvas & run in R

  12. Easy, isn’t it? • The example I just ran is your basic meta-analysis. Simple to do, but you must understand what it means and whether the analysis is a good representation of the data. • That is why you should complete this course.

More Related