1 / 31

Example of Models for the Study of Change

Example of Models for the Study of Change. David A. Kenny. Example Data. An Honors Thesis done by Allison Gillum of Skidmore College supervised by John Berman on the effects of an semester-long class on the Environment on Environmental Responsible Behaviors. Pretest-Posttest Design

Download Presentation

Example of Models for the Study of Change

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Example of Models for the Study of Change David A. Kenny

  2. Example Data An Honors Thesis done by Allison Gillum of Skidmore College supervised by John Berman on the effects of an semester-long class on the Environment on Environmental Responsible Behaviors. Pretest-Posttest Design 41 Treated and 199 Controls 2 Treated classes and 8 Control classes. No clustering effect due to class. Outcome: Environmentally Responsible Behaviors (ERB), a 12 item scale ranging for 1 to 7. For latent variable analyses, 3 parcels of 4 items were created.

  3. Models • Models • Controlling for Baseline • Simple • Allowing for Unreliability at Time 1 • Change Score Analysis • Raykov • LCS • Kenny-Judd • Standardized Change Analysis • Types • Univariate (average of 12 items) • Latent Variable (3 parcels of 4 items)

  4. Latent Variable Measurement Models • Unconstrained • c2(9) = 13.22, p = .153 • RMSEA = 0.044; TLI = .992 • Equal Loadings • c2(11) = 18.63, p = .068 • RMSEA = 0.054; TLI = .988 • The equal loading model has reasonable fit.

  5. Pretest Difference • Mean for Controls: 4.79 • Mean for Treateds: 5.21 • A mean difference of 0.42 • t(238) = 3.191, p = .002 • d = 0.64, a moderate effect size • There is a difference at the pretest! • The mean difference on latent variable at time 1 is 0.45.

  6. More on the Pretest Difference • Likely more environmentally conscious students more likely to take an environmental course. • Would you expect the difference to persist (CSA) or narrow (CfB)?

  7. Controlling for Baseline: Univariate • A beneficial effect of the course on the outcome: 0.3049. • Z = 3.992, p < .001 • b = 0.776 (expect a narrowing of the gap) • We shall see that this is the largest estimate of the treatment effect.

  8. CfB: Measurement Error in the Pretest Coefficient alpha of .872 for pretest Lord-Porter Correction Convert (.872 - .2112)/( 1 - .2112) = .866 Adjusted pretest score (MT is the mean for the Treated and MC is the mean for the controls): r(X1 – MT) + MT r(X1 – MC) + MC b = 0.2569, Z = 3.3308, p < .001

  9. Williams & Hazer Method Set X1 = X1T + E1 Fix the variance of E1 to (1 - r)sY12 or (1 - .872)(0.614) = 0.079. b = 0.2543, Z = 3.233, p = .001 (with b = .897) Same estimates of b and b as Lord-Porter (standard error a bit different)!

  10. Controlling for Baseline: Latent Variables • b = 0.3256, Z = 4.124, p < .001 • b = 0.816 (surprisingly relatively low) • Cannot directly compare estimates to the univariate analysis.

  11. Change Score Analysis: Univariate • All 3 methods (see next 3 slides) show a beneficial effect: • 0.2105, Z = 2.619, p = .009 • Smallest effect of any analysis, • Note that the Treateds improve (0.0874), and the Controls decline (-0.1231).

  12. Raykov

  13. LCS

  14. Kenny-Judd

  15. Change Score Analysis: Latent Variables • All 3 methods show a beneficial effect 0.2435, Z = 2.964, p < .003 • Again, you cannot directly compare the univariate and latent variable results. • Smallest effect of any latent variable analysis.

  16. Raykov

  17. LCS

  18. KJ

  19. Standardized Change Score Analysis: Univariate Analysis Residual variance decreases slightly over time (but not significantly, p = .31) • Time 1: 0.59 • Time 2: 0.52 Effect: .3078, Z = 2.775 , p = .006 Recalibrated to units of Time 2: 0.2266 , Z = 2.826 , p = .005.

  20. SCSA

  21. SCSA-Y2

  22. Standardized Change Score Analysis: Latent Variables Residual variance decreases slightly over time (but not significantly, p = .30) • Time 1: 0.56 • Time 2: 0.52 b = 0.3653, Z = 3.116 , p = .002 Units of Time 2: b = 0.2627, Z = 3.194 , p = .001

  23. SCSA

  24. SCSA-Y2

  25. Summary of Univariate Effects CfB: 0.3049 CfB with Reliability Correction: 0.2543 CSA: 0.2105 SCSA: 0.2266

  26. Summary of Latent Variable Effects CfB: 0.3256 CSA: 0.2435 SCSA: 0.2627

  27. What Estimate Would I Report? CSA Latent Variable: 0.2435 (Z = 2.964, p < .003) No reason to think that the factors that created the Time 1 difference to change. Note too the variance does not change. Others might respectfully disagree.

More Related