Measurement Bias Detection Through Factor Analysis

1. Measurement Bias Detection Through Factor Analysis Barendse, M. T., Oort, F. J. Werner, C. S., Ligtvoet, R., Schermelleh-Engel, K.

2. Defining measurement bias • Violation of measurement invariance Where V is violator • If V is grouping variable, then MGFA is suitable • Intercepts – uniform bias • Factor loadings – non-uniform bias (vary with t)

3. Restricted Factor Analysis (RFA) • Advantages of RFA over MGFA: • V can be continuous or discrete, observed or latent • Investigate measurement bias with multiple Vs. • More precise parameter estimates and larger power • Disadvantage of RFA: • Not suited for nonuniform bias (interaction term)

4. Approaches for non-uniform bias • RFA with latent moderated structural equations (LMS) ---- Simulation (categorical V) showed at least as good as MGFA • RFA with random regression coefficients in structural equation modeling (RSP) ---- performance unknown

5. This paper… • Compared methods: • MGFA • RFA with LMS • RFA with RSP • Measurement bias • Uniform • Nonuniform • Violator • Dichotomous • Continous

6. Data generation (RFA) • True model: • Uniform bias: . Nonuniform bias: • T and v are bivariate standard normal distributed with correlation r • e is standard normal distributed • u is null vector

7. Simulation Design For continuous V: • Type of bias (only on item 1): • No bias (b=c=0), • uniform bias(b=0.3,c=0), • nonuniform bias (b=0,c=0.3), • mixed bias (b=c=0.3) • Relationship between T and V Independent (r=0), dependent (r=0.5)

8. Simulation Design For dichotomous V: • V=-1 for group 1 and v=1 for group 2 • Model can be rewritten into • Relationship between T and V: Correlation varies!

9. The MGFA method • When v is dichotomous, regular MGFA • When v is continuous, dichotomize x by V • Using chi-square difference test with df=2 • Uniform : intercepts • Nonuniform: loadings

10. The RFA/LMS method • V is modeled as latent variable: • Single indicator • Fix residual variance (0.01) • Fix factor loading • Three-factor model: T, V, T*V • Robust ML estimation • Chi-square test with S-B correction: : uniform bias : nonuniform bias

11. RFA/RSP method • Replacing with , where is a random slope. • Robust ML estimation • Chi-square test with S-B correction: : uniform bias : nonuniform bias

12. Single & iterative procedures • Single run procedure: test once for each item • Iterative procedure: • Locate the item with the largest chi-square difference • Free constrains on intercepts and factor loadings for this item and test others • Locate the item with the largest chi-sqaure difference • … • Stops when no significant results exist or half are detected as biased

13. Results of MGFA – single run • Shown in Table 2. • Conclusion: • better with dichotomous than with continuous V; • non-uniform bias is more difficult to detect than uniform bias; • Type I error inflated.

14. Results of MGFA – iterative run • Shown in Table 3. • Conclusion: • Iterative procedure produces close power as single run does. • Iterative procedure produces better controlled Type I error rate.

15. Results of RFA/LMS & RFA/RSP - single run • Shown in Table 4 and Table 5. • Conclusion: • LMS and RSP produce almost equivalent results. • larger power than MGFA with continuous V. • More severely inflated Type I error rates

16. Results of RFA/LMS & RFA/RSP - iterative run • Shown in Table 6. • Conclusion: • Power is close to the single run • Type I error rates are improved

17. Results of estimation bias - MGFA • Shown in Table 7. • Conclusion: • Bias in estimates is small • Bias in SD is non-ignorable • Smaller bias in estimates for dichotomous V (dependent T&V)

18. Results of estimation bias - RFA • Shown in Table 8 & 9 • Conclusion: • Similar results for LMS and RSP • Small bias in estimates • Non-ignorable bias in SD • Smaller SE than MGFA • Smaller bias in estimates than MGFA with dependent T&V, continuous V.

19. Discussion • Nonconvergence occurs with RFA/LMS

20. Non-convergence • Summary: