Combining effect sizes
1 / 9

Combining Effect Sizes - PowerPoint PPT Presentation

  • Updated On :

Combining Effect Sizes. What to Combine? How to do it?. What to Combine? (1). Whether effect sizes can/should be combined is controversial Safe if studies are replications (identical IV, DV, design, only difference is observations)

Related searches for Combining Effect Sizes

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation

PowerPoint Slideshow about 'Combining Effect Sizes' - lazar

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Combining effect sizes l.jpg

Combining Effect Sizes

What to Combine?

How to do it?

What to combine 1 l.jpg
What to Combine? (1)

  • Whether effect sizes can/should be combined is controversial

  • Safe if studies are replications

    • (identical IV, DV, design, only difference is observations)

    • Random samples from same population – results in a sampling distribution

What to combine 2 l.jpg
What to Combine (2)

  • Not safe or reasonable to combine if

    • IV or DV measures different constructs across studies (e.g., DV in study 1 is taxes paid, and DV in study 2 is subjective well being)

    • Different study designs ( study 1 has random assignment to condition; study 2 allows participants to choose conditions)

What to combine 3 l.jpg
What to Combine (3)

  • OK to combine if

    • Measures are same across studies (e.g., all studies use GRE) (can use unstandardized ES in analysis)

    • Measures across all studies are parallel (can use unstandardized ES)

    • Measures are essentially tau equivalent (same reliability, different mean and SD); use Standardized ES

    • Measures are congeneric (diff reliability, different M and SD) use Standardized ES, may adjust model for reliability.

What to combine 4 l.jpg
What to Combine (4)

  • Most social science studies are hard to justify as proper for a meta-analysis

  • Glass (any and all therapy, any and all outcomes)

  • Gaugler (assessment centers)

  • When is a study a replication?

  • Compare to medicine (e.g., aspirin and heart attack)

How to combine 1 l.jpg
How to Combine (1)

  • Take the simple mean (add all ES, divide by number of ES)


M = 1.8/3


Unbiased, consistent, but not efficient estimator.

How to combine 2 l.jpg
How to Combine (2)

  • Take a weighted average




(cf .6 w/ unit wt)

(Unit weights are special case where w=1.)

How to combine 3 l.jpg
How to Combine (3)

  • Choice of Weights (all are consistent, will give good estimates as the number of studies and sample size of studies increases)

    • Unit

      • Unbiased, inefficient

    • Sample size

      • Unbiased (maybe), efficient relative to unit

    • Inverse variance – endorsed by PMA

      • Reciprocal of sampling variance

      • Biased (if parameter figures in sampling variance), most efficient

    • Other – special weights depend on model, e.g., adjust for reliability (Schmidt & Hunter)

How to combine 4 l.jpg
How to Combine (4)

  • Inverse Variance Weights are a function of the sample size, and sometimes also a parameter.

  • For the mean:

  • For r:

  • For r transformed to z:

Note that for two of three of these, the parameter is not part of the weight. For r, however, larger observed values will get more weight. Mean can be biased.