Loading in 2 Seconds...

Estimation of Ability Using Globally Optimal Scoring Weights

Loading in 2 Seconds...

- 122 Views
- Uploaded on

Download Presentation
## PowerPoint Slideshow about 'Estimation of Ability Using Globally Optimal Scoring Weights' - pascal

**An Image/Link below is provided (as is) to download presentation**

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript

Estimation of Ability Using Globally Optimal Scoring Weights

Shin-ichi Mayekawa

Graduate School of Decision Science and Technology

Tokyo Institute of Technology

Outline

- Review of existing methods
- Globally Optimal Weight: a set of weights that maximizes the Expected Test Information
- Intrinsic Category Weights
- Examples
- Conclusions

Background

- Estimation of IRT ability q on the basis of simple and weighted summed score X.
- Conditional distribution of X given qas the distribution of the weighted sum of the Scored Multinomial Distribution.
- Posterior Distribution of q given X.

h(q|x) @ f(x|q) h(q )

- Posterior Mean(EAP) of q given X.
- Posterior Standard Deiation(PSD)

Conditional distribution of X given q

- Binary items
- Conditional distribution of summed score X.
- Simple sum: Walsh(1955), Lord(1969)
- Weighted sum: Mayekawa(2003)
- Polytomous items
- Conditional distribution of summed score X.
- Simple sum: Hanson(1994), Thissen et.al.(1995)
- With Item weight and Category weight: Mayekawa & Arai(2007)

Example

- Eight Graded Response Model items 3 categories for each item.

Example (choosing weight)

- Example: Mayekawa and Arai (2008)
- small posterior variance good weight.
- Large Test Information (TI) good weight

Test Information Function

- Test Information Function is proportional to the slope of the conditional expectation of X given q, (TCC), and inversely proportional the squared width of the confidence interval (CI) of q given X.
- Width of CI
- Inversely proportional to the conditionalstandard deviation of X given q.

Maximization of the Test Informationwhen the category weights are known.

- Category weighted Item Scoreand the Item Response Function

Maximization of the Test Informationwhen the category weights are known.

Maximization of the Test Informationwhen the category weights are known.

- Test Information

Maximization of the Test Informationwhen the category weights are known.

- First Derivative

Maximization of the Test Informationwhen the category weights are known.

Globally Optimal Weight

- A set of weights that maximizethe Expected Test Informationwith some reference distribution of q .

It does NOT depend on q .

Example

NABCT A B1 B2 GO GOINT A AINT

Q1 1.0 -2.0 -1.0 7.144 7 8.333 8

Q2 1.0 -1.0 0.0 7.102 7 8.333 8

Q3 1.0 0.0 1.0 7.166 7 8.333 8

Q4 1.0 1.0 2.0 7.316 7 8.333 8

Q5 2.0 -2.0 -1.0 17.720 18 16.667 17

Q6 2.0 -1.0 0.0 17.619 18 16.667 17

Q7 2.0 0.0 1.0 17.773 18 16.667 17

Q8 2.0 1.0 2.0 18.160 18 16.667 17

LOx LO GO GOINT A AINT CONST

7.4743 7.2993 7.2928 7.2905 7.2210 7.2564 5.9795

Maximization of the Test Informationwith respect tothe category weights.

- Absorb the item weight in category weights.

Maximization of the Test Informationwith respect tothe category weights.

- Test Information
- Linear transformation of the categoryweights does NOT affect the information.

Maximization of the Test Informationwith respect tothe category weights.

- First Derivative

Maximization of the Test Informationwith respect tothe category weights.

- Locally Optimal Weight

Globally Optimal Weight

- Weights that maximizethe Expected Test Informationwith some reference distribution of q .

Intrinsic category weight

- A set of weights which maximizes:
- Since the category weights can belinearly transformed, we set v0=0, ….. vmax=maximum item score.

Example of Intrinsic Weights

- h(q)=N(-0.5, 1): v0=0, v1=*, v2=2

Example of Intrinsic Weights

- h(q)=N(0.5, 1): v0=0, v1=*, v2=2

Example of Intrinsic Weights

- h(q)=N(1, 1 ): v0=0, v1=*, v2=2

Summary of Intrinsic Weight

- It does NOT depend on q, butdepends on the reference distributionof q: h(q) as follows.
- For the 3 category GRM, we found that
- For those items with high discriminationparameter, the intrinsic weights tendto become equally spaced: v0=0, v1=1, v2=2
- The Globally Optimal Weight isnot identical to the Intrinsic Weights.

Summary of Intrinsic Weight

- For the 3 category GRM, we found that
- The mid-category weight v1 increases according to the location of the peak ofICRF. That is:

The more easy the category is,

the higher the weight .

- v1 is affected by the relative location ofother two category ICRFs.

Summary of Intrinsic Weight

- For the 3 category GRM, we found that
- The mid-category weight v1 decreases according to the location of the reference distribution of q: h(q).
- If the location of h(q) is high, the mostdifficult category gets relatively high weight,and vice versa.
- When the peak of the 2nd categorymatches the mean of h(q), we haveeqaully spaced category weights:

v0=0, v1=1, v2=2

Bayesian Estimation of q from X

(1/0.18)^2 = 30.864

Conclusions

- Polytomous item has the Intrinsic Weight.
- By maximizing the Expected Test Information with respect to either Item or Category weights, we can calculate the Globally Optimal Weights which do not depend on q.
- Use of the Globally Optimal Weights when evaluating the EAP of q given X reduces the posterior variance.

Download Presentation

Connecting to Server..