Combining Effect Sizes

1 / 9

# Combining Effect Sizes - PowerPoint PPT Presentation

Combining Effect Sizes. What to Combine? How to do it?. What to Combine? (1). Whether effect sizes can/should be combined is controversial Safe if studies are replications (identical IV, DV, design, only difference is observations)

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.

## PowerPoint Slideshow about 'Combining Effect Sizes' - lazar

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript

### Combining Effect Sizes

What to Combine?

How to do it?

What to Combine? (1)
• Whether effect sizes can/should be combined is controversial
• Safe if studies are replications
• (identical IV, DV, design, only difference is observations)
• Random samples from same population – results in a sampling distribution
What to Combine (2)
• Not safe or reasonable to combine if
• IV or DV measures different constructs across studies (e.g., DV in study 1 is taxes paid, and DV in study 2 is subjective well being)
• Different study designs ( study 1 has random assignment to condition; study 2 allows participants to choose conditions)
What to Combine (3)
• OK to combine if
• Measures are same across studies (e.g., all studies use GRE) (can use unstandardized ES in analysis)
• Measures across all studies are parallel (can use unstandardized ES)
• Measures are essentially tau equivalent (same reliability, different mean and SD); use Standardized ES
• Measures are congeneric (diff reliability, different M and SD) use Standardized ES, may adjust model for reliability.
What to Combine (4)
• Most social science studies are hard to justify as proper for a meta-analysis
• Glass (any and all therapy, any and all outcomes)
• Gaugler (assessment centers)
• When is a study a replication?
• Compare to medicine (e.g., aspirin and heart attack)
How to Combine (1)
• Take the simple mean (add all ES, divide by number of ES)

M=(1+.5+.3)/3

M = 1.8/3

M=.6

Unbiased, consistent, but not efficient estimator.

How to Combine (2)
• Take a weighted average

M=(1+1+.9)/(1+2+3)

M=(2.9)/6

M=.48

(cf .6 w/ unit wt)

(Unit weights are special case where w=1.)

How to Combine (3)
• Choice of Weights (all are consistent, will give good estimates as the number of studies and sample size of studies increases)
• Unit
• Unbiased, inefficient
• Sample size
• Unbiased (maybe), efficient relative to unit
• Inverse variance – endorsed by PMA
• Reciprocal of sampling variance
• Biased (if parameter figures in sampling variance), most efficient
• Other – special weights depend on model, e.g., adjust for reliability (Schmidt & Hunter)
How to Combine (4)
• Inverse Variance Weights are a function of the sample size, and sometimes also a parameter.
• For the mean:
• For r:
• For r transformed to z:

Note that for two of three of these, the parameter is not part of the weight. For r, however, larger observed values will get more weight. Mean can be biased.