Modelling atypical students response patterns using multidimensional parametric models

1. Modelling atypical students response patterns using multidimensional parametric models Gilles Raîche, UQAM Sébastien Béland, UQAM David Magis, Université de Liège Jean-Guy Blais, Université de Montréal Pierre Brochu, CMEC Large-Scale Assessments: Policy, Research and Practice CSSE / CERA Montréal, 2010

2. SUMMARY • Introduction and Objectives • Unidimensional IRT Models • IRT Person Parameters Models • Person response Curve • Multidimensional Item Response Models • Estimation • An R Package: irtProb • Examples • Other Considerations • References and contacts

3. INTRODUCTION • Presentation • IRT Models of Interest • Unidimensional latent proficiency • Dichotomous response • Monotonic • Logistic Probability Distribution

4. OBJECTIVES • Simulation of Inappropriate Response Patterns • Person Misfit Detection Indices • Distributional Properties of Person Misfit Indices • Adjusted Proficiency Level Estimation in Presence of Person Misfit

5. UNIDIMENSIONAL IRT MODELS 3 Parameter Logistic (3PL) (Birnbaum, 1968) 4 Parameters Logistic (4PL) (McDonald, 1967) where if ai is considered as a standard deviation

6. PERSON RESPONSE CURVE (Trabin and Weiss, 1983)

7. MULTIDIMENSIONAL ITEM RESPONSE MODELS • Personal Variance (σ2) (Ferrando, 2004; Thurstone, 1927) • Personal Inattention (δ) • Personal Pseudo-Guessing (χ) (Strandmark and Linn, 1987)

8. MULTIDIMENSIONAL ITEM RESPONSE MODELS • Higher Order Models

9. ESTIMATION OF SUBJECT PARAMETERS • Package: irtProb • MAP Estimators • A Priori Probability Distribution • σ : U(0,4) • θ: U(-4,4) • X: U(0,1) • δ : U(0,1)

10. A R PACKAGE: irtProb • Available on R Cran Site • Functionnalities • Estimation of Person Parameters (MAP) • Likelihood Curves • Person Characteristic Curves • Probability, Density and Random Functions • Simulation of Response Patterns • Classical <-> IRT Item Parameters • Model Selection

11. EXAMPLES – 01 (X) 1 σ = 0, δ = 0, b = -5 to 5, c = 0, d = 0, 40 items, 100 simulated sujects Model 1: θ only Model 2: θ and Pseudo-Guessing Model 3 σ, θ, Pseudo-Guessing and δ

12. EXAMPLES – 01 (X)

13. EXAMPLES – 02 (X) 1 σ = 0, δ = 0, b = -5 to 5, c = 0, d = 0, 40 items, 100 simulated subjects Model 1: θ only Model 2: θ and Pseudo-Guessing Model 3 σ, θ, Pseudo-Guessing and δ

14. EXAMPLES – 02 (X)

15. EXAMPLES – 03 (σ) 1 X = 0, δ = 0, b = -5 to 5, c = 0, d = 0, 40 items, 100 simulated subjects Model 1: θ only Model 2: θ and σ Model 3 σ, θ, Pseudo-Guessing and δ

16. EXAMPLES – 03 (σ)

17. OTHER CONSIDERATIONS • Multidimensional EAP Estimation Very Computer Intensive • Warm Weighted Likelihood Estimator • Item Parameters Estimation • Confidence Interval For The Additionnal Person Parameters • Other Person Fit Indices: Pseudo-Guessing and Inattention

18. REFERENCES / 1 Barton, M. A. and Lord, F. M. (1981). An upper asymptote for the three-parameter logistic item-response model. Research bullelin 81-20. Princeton, NJ: Educational Testing Service. Birnbaum, A. (1968). Some latent trait models and their use in inferring an examinee’s ability. In F. M. Lord and M. Novick (Eds): Statistical theories of mental test scores. New York, NJ: Addison-Wesley. Ferrando, P. J. (2004). Person reliability in personality measurement: an item response theory analysis. Applied Psychological Measurement, 28(2), 126-140. Hulin, C. L., Drasgow, F., and Parsons, C. K. (1983). Item response theory. Homewood, IL: Irwin. Levine, M. V., and Drasgow, F. (1983). Appropriateness measurement: validating studies and variable ability models. In D. J. Weiss (Ed.): New horizons in testing. New York, NJ: Academic Press. Magis, D. (2007). Enhanced estimation methods in IRT. In D. Magis (Ed.): Influence, information and item response theory in discrete data analysis. Doctoral dissertation, Liège, Belgium: University de Liège.

19. REFERENCES / 2 McDonald, R. P. (1967). Nonlinear factor analysis. Psyhometric Monographs, 15. Raîche, G., and Blais, J.-G. (2003). Efficacité du dépistage des étudiants et des étudiants qui cherchent à obtenir un résultat faible au test de classement en anglais, langue seconde, au collégial. In J.-G. Blais, and G. Raîche (Ed.): Regards sur la modélisation de la mesure enen éducation et en sciences sociales. Ste-Foy, QC: Presses de l’Université Laval. Strandmark, N. L. and Linn, R. L. (1987). A generalized logistic item response model parameterizing test score inappropriateness. Applied Psychological Measurement, 11(4), 355-370. Thurstone, L. L. (1927). A law of comparative judgment. Psychological Review, 34, 273-286. Trabin, T. E., and Weiss, D. J. (1983). The person response curve : fit of individuals to item response theory models. In D. J. Weiss (Ed.): New horizons in testing. New York, NJ: Academic Press.

20. CONTACTS • Gilles Raîche • http://camri.uqam.ca • Sébastien Béland • sebastien.beland.1@hotmail.com • David Magis • david.magis@psy.kuleuven.be • Jean-Guy Blais • http://www.griemetic.ca • Pierre Brochu • p.broche@cmec.ca