1 / 18

Skills Diagnosis with Latent Variable Models

Skills Diagnosis with Latent Variable Models. Topic 1: A New Diagnostic Paradigm. Introduction. Assessments should aim to improve, and not merely ascertain the status of student learning

viveca
Download Presentation

Skills Diagnosis with Latent Variable Models

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Skills Diagnosis with Latent Variable Models

  2. Topic 1:A New Diagnostic Paradigm

  3. Introduction • Assessments should aim to improve, and not merely ascertain the status of student learning • For test scores to facilitate learning, they need to be interpretative, diagnostic, highly informative, and potentially prescriptive • Most large-scale assessments are based on traditional unidimensional IRT models that only provide single overall scores • These scores are useful primarily for ordering students along a continuum

  4. Alternative psychometric models that can provide inferences more relevant to instruction and learning currently exist • These models are called cognitive diagnosis models (CDMs) • Alternatively, they are referred to as diagnostic classification models (DCMs) • CDMs are multiplediscretelatent variable models • They are developed specifically for diagnosing the presence or absence of multiple fine-grained attributes (e.g. skills, cognitive processes or problem-solving strategies)

  5. Fundamental difference between IRT and CDM: A fraction subtraction example • IRT: performance is based on a unidimensional continuous latent trait • Students with higher latent traits have higher probability of answering the question correctly

  6. Fundamental difference between IRT and CDM: A fraction subtraction example • IRT: performance is based on a unidimensional continuous latent trait • Students with higher latent traits have higher probability of answering the question correctly • CDM: performance is based on binary latent attribute vector • Successful performance on the task requires a series of successful implementations of the attributes specified for the task

  7. Required attributes: (1) Borrowing from whole (2) Basic fraction subtraction (3) Reducing • Other attributes: (4) Separating whole from fraction (5) Converting whole to fraction

  8. Basic Elements and Notations of CDM • The response vector of examinee i will be denoted by , • The response vector contains J items, as in, • The attribute vector of examinee i will be denoted by • Each attribute vector or pattern defines a unique latent class • Thus, K attributes define latent classes

  9. Example: When , the total number of latent classes is • Although arbitrary, we can associate the following attribute vectors with the following latent classes:

  10. Basic CDM Input • Like IRT, CDM requires an binary response matrix as input • Unlike IRT, CDM in addition requires a binary matrix called the Q-matrix as input • The rows of the Q-matrix pertain to the items, whereas the columns the attributes • The 1s in the jth row of the Q-matrix identifies the attributes required for item j

  11. Examples of Attribute Specification

  12. Examples of Attribute Specification

  13. Basic CDM Output • The goal of CDM is to make inference about the attribute vector • The basic CDM output gives the (posterior) probability the examinee has mastered each of the attributes • That is, we get • For example, , indicates that we are quite certain that examinee has already mastered attribute 1

  14. Each examinee gets a vector of posterior probabilities • For reporting purposes, we may want to convert the probabilities into 0s and 1s • We can use different rules for this conversion • If ; Otherwise,

  15. Example:

  16. Each examinee gets a vector of posterior probabilities • For reporting purposes, we may want to convert the probabilities into 0s and 1s • We can use different rules for this conversion • If ; Otherwise, • If ; or If ; Otherwise,

  17. Example: ? – means we do not have sufficient evidence to conclude one way or the other

More Related