1 / 32

Meta-Analysis using HLM 6.0

Meta-Analysis using HLM 6.0. Yaacov Petscher Florida Center for Reading Research. Why use HLM?. Nested structure Necessity of special models Variation at both subject and study levels. “Special Case” of HLM.

afya
Download Presentation

Meta-Analysis using HLM 6.0

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Meta-Analysis using HLM 6.0 Yaacov Petscher Florida Center for Reading Research

  2. Why use HLM? • Nested structure • Necessity of special models • Variation at both subject and study levels

  3. “Special Case” of HLM • If ES are based on n ≥ 30, we assume approximate normal distribution with sampling variance assumed to be known • V-known models run in Interactive Mode • Time to brush up on your DOS command code!

  4. Standardized Mean Difference • No raw data for us • Must rely on stats to be converted to single metric • Many types of statistics that may be used Z t M/SD χ² F p-value r r²

  5. Level-1 (Within-Studies) Model = any standardized effect measure from study j = the corresponding population parameter = sample error associated with d

  6. Level-2 (Between-Studies) Model = grand mean effect size = regression coefficients = study characteristics (moderators) = level 2 random error

  7. Combined Model

  8. Estimation Empirical Bayes Estimator where

  9. EB Estimates • Level 1 • May be used as a shrinkage estimator to identify potential outliers • Shrinkage in the direction of the grand mean • Level 2 • Supplying the grand mean provides an estimate of the conditional shrinkage • Shrinkage towards a value that is conditional on the amount of prior contacts (WEEKS)

  10. The following example will be run using data from Raudenbush & Bryk Chapter 7 data (pg 211) The Effect of Teacher Expectancy on Pupil IQ

  11. Some Calculations • Need the Conditional Variances • Since d in this model is Fisher’s r to Z transformation, the formula is Since we’re not given n we need to calculate another way…..ideas?

  12. YAY! • Simply square your standard error

  13. Data File Prep Considerations • Since meta-analysis in HLM is a V-known model, only one data file is used

  14. Data File Prep Considerations, cont • Four key features to data prep (assume using SPSS) • Column 1 = ID in character format • Column 2 = ES estimates • Column 3 = Variance estimates • Column 4-n = Potential level-2 predictors

  15. Formatting • Variable View • Column 1 • String, width = 2, decimal = 0 • Columns 2-n • Numeric, width = 12, decimal = 3 • Save as a Fixed ASCII (.dat) file • Hold onto your output, you’re gonna need it! • You should now have a list of the variables and associated Format statements

  16. HLM in Batch Mode • Bring up your computer’s Command Prompt • Typically found in “Accessories” • By default, you should see C:\> • If not, type c: and hit enter • At this point you want to locate HLM • Type dir, hit enter • cd program files, enter • cd HLM6, enter • hlm2, enter • We’re now ready to begin!

  17. HLM2- MDM File Creation C:\Program Files\HLM6\hlm2 Will you be starting with raw data? y Is the input file a v-known file? y How many level-1 statistics are there? 1 How many level-2 predictors are there? 1 Enter 8 character name for level-1 variable number 1: Zes Enter 8 character name for level-2 variable number 1: weeks Input format of raw data file (the first field must be the character ID) format: (a2, 3f12.3) What file contains the data: e:\test.dat Enter name of MDM file: e:\test.mdm 19 groups have been processed C:\Program Files\HLM6>

  18. Specifying UC HLM Model C:\HLM6> hlm2 e:\test.mdm SPECIFYING AN HLM MODEL Level-1 predictor variable specification Which level-1 predictors do you wish to use? The choices are: For ZES enter 1 Level-1 predictors? (Enter 0 to end) 1 Level-2 predictor variable specification Which level-2 variables do you wish to use? The choices are: For WEEKS enter 1 Which level-2 predictors to model ZES? Level-2 predictor? (Enter 0 to end) 0

  19. ADDITIONAL PROGRAM FEATURES Select the level-2 variables that you might consider for Inclusion as predictors in subsequent models. The choices are: For WEEKS enter 1 Which level-2 variables to model ZES? Level-2 variable? (Enter 0 to end) 0 Do you wish to use any of the optional hypothesis testing procedures? n OUTPUT SPECIFICATION Do you want a level-2 residual file? n How many iterations do you want to do? 10000 Do you want to see OLS estimates for all of the level-2 units? n Enter a problem title: lvl1 Enter name of output file: e:\lvl1.lis

  20. Results for UC Model ----------------------------------------------------------------------------------------- Standard Approx. Fixed Effect Coefficient Error T-ratio d.f. ----------------------------------------------------------------------------------------- For ZES, B1 INTRCPT2, G10 0.084376 0.052039 1.621 18 ----------------------------------------------------------------------------------------- ----------------------------------------------------------------------------------------------- Random Effect Standard Variance df Chi-square P-value Deviation Component ------------------------------------------------------------------------------------------------ ZES, U1 0.13896 0.01931 18 36.25115 0.007 ------------------------------------------------------------------------------------------------ Significant variability exists in true-effect sizes

  21. EB Estimation Level 1 Since there are no predictors at Level 1, the last term is omitted, leaving us with

  22. Using Excel to Calculate EB Using the variance component from lvl 1 Model, create Lambda using the formula function

  23. Supplying the grand mean ES into the Excel formula function allows us get the EB estimates See the difference in results?

  24. Specifying CL2 HLM Model C:\HLM6> hlm2 e:\test.mdm SPECIFYING AN HLM MODEL Level-1 predictor variable specification Which level-1 predictors do you wish to use? The choices are: For ZES enter 1 Level-1 predictors? (Enter 0 to end) 1 Level-2 predictor variable specification Which level-2 variables do you wish to use? The choices are: For WEEKS enter 1 Which level-2 predictors to model ZES? Level-2 predictor? (Enter 0 to end) 1

  25. ADDITIONAL PROGRAM FEATURES Select the level-2 variables that you might consider for Inclusion as predictors in subsequent models. The choices are: For WEEKS enter 1 Which level-2 variables to model ZES? Level-2 variable? (Enter 0 to end) 0 Do you wish to use any of the optional hypothesis testing procedures? n OUTPUT SPECIFICATION Do you want a level-2 residual file? n How many iterations do you want to do? 10000 Do you want to see OLS estimates for all of the level-2 units? n Enter a problem title: lvl2 Enter name of output file: e:\lvl2.lis

  26. Results for CL2 Model ------------------------------------------------------------------------------------------------- Standard Approx. Fixed Effect Coefficient Error T-ratio d.f. P-value ------------------------------------------------------------------------------------------------- For ZES, B1 INTRCPT2, G10 0.408572 0.087146 4.688 17 0.000 WEEKS, G11 -0.157963 0.035943 -4.395 17 0.000 ------------------------------------------------------------------------------------------------- ------------------------------------------------------------------------------------------------ Random Effect Standard Variance df Chi-square P-value Deviation Component ------------------------------------------------------------------------------------------------ ZES, U1 0.00283 0.00001 17 16.53614 >.500 ------------------------------------------------------------------------------------------------

  27. EB Estimation Level 2 Since then = 0 and we’re left with Using G10 and G11, we can calculate the EB estimates for Level 2

  28. EB Estimation Level 2 • Since our Level-2 predictor takes on one of four different values, the shrinkage is towards one of the four points.

  29. End (for now)

More Related