1 / 14

Advanced Modeling Issues

Advanced Modeling Issues. Lecture 9 EPSY 642 Meta Analysis Fall 2009 Victor L. Willson, Instructor. Current Issues. Multi-level models: Raudenbush & Bryk analysis in HLM6 Structural equation modeling in meta-analysis Clustering of effects: cluster analysis vs. latent class modeling

brendon-mawe
Download Presentation

Advanced Modeling Issues

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Advanced Modeling Issues Lecture 9 EPSY 642 Meta Analysis Fall 2009 Victor L. Willson, Instructor

  2. Current Issues • Multi-level models: Raudenbush & Bryk analysis in HLM6 • Structural equation modeling in meta-analysis • Clustering of effects: cluster analysis vs. latent class modeling • Multiple studies by same authors- how to treat (beyond ignoring follow-on studies), the study dependence problem • Multiple meta-analyses: consecutive, overlapping • Multiple outcomes per study

  3. Multilevel Models • Raudenbush & Bryk HLM 6 • One effect per study • Two level model, mediators and moderators at the second level • Known variance for first level (wi) • Mixed model analysis: requires 30+ studies for reasonable estimation, per power analysis • Maximum likelihood estimation of effects

  4. Multilevel Models • Model: Level 1: gi = gi + ei where there is one effect g per study i Level 2: gi = 0 + 1W + ui where W is a study-level predictor such as design in our earlier example Assumption: the variance of gi is known = wi

  5. Structural Equation Modeling in SEM • New area- early work in progress: • Cheung & Chan (2005, Psych Methods), (2009, Struc Eqn Modeling)- 2-step approach using correlation matrices (variables with different scales) or covariance matrices (variables measured on the same scale/scaling) • Stage 1: create pooled correlation (covariance) matrix • Stage 2: fit SEM model to Stage 1 result

  6. Structural Equation Modeling in SEM • Pooling correlation matrices: • Get average r: rmean(jk) = wi riij/ wijk I i where j and k are the subscripts for the correlation between variables j and k, where i is the ith data set being pooled Cheung & Chan propose transforming all r’s to Fisher Z-statistics and computing above in Z If using Z, then the SE for Zi is (1-r2)/n½ and

  7. Structural Equation Modeling in SEM • Pooling correlation matrices: for each study, COVg(rij, rkl) = [ .5rij rkl (r2ik + r2il + r2jk + r2jl) + rik*rjl + ril*rjk – (rij*rik*ril + rji*rjk*rjl + rki*rkj*rkl + rli*rlj*rlk)]/n Let i = covariance matrix for study i, G = {0,1} matrix that selects a particular correlation for examination, Then G = [ |G1|’ G2 |’…| Gk |’]’ and  = diag [1, 2, … k]

  8. Structural Equation Modeling in SEM Beretvas & Furlow (2006) recommended transformations of the variances and covariances: SDrtrans = log(s) + 1/(2(n-1) COV(ri,rj)trans = r2ij/(2(n-1)) The transformed covariance matrices for each study are then stacked as earlier

  9. Clustering of effects: cluster analysis vs. latent class modeling • Suppose Q is significant. This implies some subset of effects is not equal to some other subset • Cluster analysis uses study-level variables to empirically cluster the effects into either overlapping or non-overlapping subsets • Latent class analysis uses mixture modeling to group into a specified # of classes • Neither is fully theoretically developed- existing theory is used, not clear how well they work

  10. Multiple studies by same authors- how to treat (beyond ignoring follow-on studies), the study dependence problem • Example: in storybook telling literature, Zevenberge, Whitehurst, & Zevenbergen (2003) was a subset of Whitehurst, Zevenbergen, Crone, Schultz, Velging, & Fischel (1999), which was a subset of Whitehurst, Arnold, Epstein, Angell, Smith, & Fischel (1994) • Should 1999 and 2003 be excluded, or included with adjustments to 1994? • Problem is similar to ANOVA: omnibus vs. contrasts • Currently, most people exclude later subset articles

  11. Multiple meta-analyses: consecutive, overlapping • The problem of consecutive meta-analyses is now arising: • Follow-ons typically time-limited (after last m-a) • Some m-a’s partially overlap others: how should they be compared/integrated/evaluated? • Are there statistical methods, such as the correlational approach detailed above, that might include partial dependence? • Can time-relatedness be a predictor? Willson (1985)

  12. Multiple Outcomes per study • Multilevel Approach to study dependency: • Requires assumption of homogeneous error variance across studies • Variation within “cluster” (study) is the same for all studies • If above is reasonable, MLM may be a reasonable model • Weight function acts as “sample size” equivalent for second level analysis- use weighted average W1/2 or N for the effects within the study (unclear which will be more appropriate here) • If all effects within-study have the same sample size(s) the W for all are equal to each other

  13. Multiple Outcomes per study IndepVar Effect  W1/2 Study effect e

  14. CONCLUSIONS • Meta-analysis continues to evolve • Focus in future on complex modeling of outcomes (SEM, for example) • More work on integration of qualitative studies with meta-analysis findings

More Related