Publication availability bias
Download
1 / 10

Publication/Availability Bias - PowerPoint PPT Presentation


  • 125 Views
  • Updated On :

Publication/Availability Bias. Problems with bad data. Problem of missing studies. Missing at random is okay Nonrandom is a problem Sources of nonrandom samples of studies Publication bias – significant, small N Grad student work; other filedrawer

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Publication/Availability Bias' - claudette


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Publication availability bias l.jpg

Publication/Availability Bias

Problems with bad data


Problem of missing studies l.jpg
Problem of missing studies

  • Missing at random is okay

  • Nonrandom is a problem

  • Sources of nonrandom samples of studies

    • Publication bias – significant, small N

    • Grad student work; other filedrawer

    • Deliberate misrepresentation for financial reasons


Forest plot l.jpg
Forest Plot

Drift to the right by precision?

Source: Borenstein, Hedges, Higgins & Rothstein, 2009, p. 282


Funnel plot l.jpg
Funnel plot

Assymetry?

Esp lower right.

These appear biased and heterogeneous to me.

Source: Sutton (2009). In Cooper, Hedges, & Valentine (Eds) Handbook fo Research Synthesis Methods p. 501


Fail safe n l.jpg
Fail-safe N

  • How many studies do we need to make the result n.s.? (Rosenthal)

  • How many studies do we need to make the summary effect small enough to ignore? (Orwin)

kfs = failsafe studies

kobt = studies in meta

dobt = summary ES

dc = desired lower boutnd

dfs= studies with this (eg 0) size needed to lower ES

Corwin, R. G. (1983). A fail-safe N for effect size in meta-analysis. Journal of Educational Statistics, 8, 157-159.


Trim fill l.jpg
Trim & Fill

Creates symmetry;

Adjusts summary ES

Source: Borenstein, Hedges, Higgins & Rothstein, 2009, p. 287


Cumulative forest l.jpg
Cumulative Forest

Source: Borenstein, Hedges, Higgins & Rothstein, 2009, p. 288


Egger s regression l.jpg
Egger’s Regression

Ti = effect size; vi=sampling variance of ES

Should be flat (centered) if no bias. This shows small studies have higher values.

Source: Sutton (2009). In Cooper, Hedges, & Valentine (Eds) Handbook fo Research Synthesis Methods p. 441


Sensitivity analysis l.jpg
Sensitivity Analysis

Outliers. Run twice. Pray.

Source: Greenhouse & Iyengar (2009). In Cooper, Hedges, & Valentine (Eds) Handbook fo Research Synthesis Methods p. 422


Sensitivity analysis10 l.jpg
Sensitivity Analysis

Varying levels of tau-squared

Source: Louis & Zelterman (1994). In Cooper, Hedges (Eds) Handbook of Research Synthesis p. 418


ad