Modeling Achievement Trajectories When Attrition is Informative

1. Modeling Achievement Trajectories When Attrition is Informative Betsy J. Feldman & Sophia Rabe-Hesketh

2. Dropout and missing data in longitudinal educational data; • Traditional approaches (listwise deletion, mean imputation) • Mean imputation: variance, covariance and standard error will be underestimated • (a) imputed data are constant at a given time, hence, the variance will be underestimated; • (b) regression coefficients for covariates (other than time) will be biased toward zero • (c) treating imputed data as real ignore the variability of response, SE is underestimated (multiple imputation)

3. Missingness Mechanisms & Modeling Techniques • 1. Models for Growth • Level 1: time specific response (within-person) • Level 2: between-person variability in growth trajectories

4. Missing completely at random (MCAR): missingness does not depend on any other observed or unobserved variables • Xgi is atime-varying and time-invariate covariate matrix • ygi is observation for individual i in group g at time t • ugiis a vector of intercept and slope residuals for individual i in group g

5. Covariate dependent missingness • Missing at random (MAR) • MCAR or MAR are referred to as ignorable or noninformative.

7. Not missing at random (NMAR) • (a) missingness depend on missing values • (b) missingness depend on random coefficient

9. Survival-Process NMAR model • Single-indicator NMAR model

10. Two basic approach of modeling nonignorable missingness: • (a) selection model: assuming the missing either depend on outcome-dependent missingness or random-coefficient-dependent missing; • (b) pattern mixture model

11. Muthén-Roy Pattern-Mixture Model

12. Simulation • Five analysis • (a) a traditional growth model (HLM) • (b) a dual-process single-indicator model in which a single-indicator variable for dropping out at any time after the first time point • (c) a survival-process model (true model) • (d) listwise deletion • (e) mean imputation

13. Three manipulate variable • (a) sample sizes (n=300, 1000) • (b) percentage of missing data (10%, 40%) • (c) levels of dependence (weak and strong) of the drop-out process weak: -0.1, -0.2 strong: -0.5, -1.4

14. Results If missingness was NMAR but treated as ignorable, the slope means, slope variance, and covariance were biased only when missing percentage and the dependence were both high. The fit statistics is not likely to indicate the incorrect treatment. Standard error were found not to be affected much but the coverage were poor.

15. The slop parameters were biased when the missing percentage and the dependence were high, but better than growth model (MAR). SE were not affected. The parameters and the coverage were good for the survival-process model which was the true model. The results were not reported.

16. Listwise deletion resulted in upward bias for slop and intercept mean, but the variance were underestimated.

17. The slope means and variance were upward biased. ? The residual variance should be underestimated. Consequently, the standard error will be underestimated.

18. Empirical Data • National Education Longitudinal Study of 1988 (NELS: 88) • Analysis: • (a) linear growth model with and without the covariates • (b) dual-process survival-process NMAR model • (c) Single-indicator NMAR model

20. Ethical & Grade as dummy variables

24. Discussion & Conclusions • Treating NMAR as ignorable (depend on the random coefficients) can results in biased estimates, especially for the estimated variances and covariances. • The simulations showed what proportion of missingness and how strong the dependence will begin to result in serious bias.

25. Comments & Questions • regression coefficients are given in Figure 1. • Use another imputation other than mean imputation method •  The y was substituted by estimate of theta score, it may be risky to ignore the standard error of theta. • It will be better to use plausible values in the growth model or to use multilevel IRT approach to estimate growth.