estimation of item difficulty index based on item response theory for computerized adaptive testing n.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
Authors : Shu -Chen Cheng, PowerPoint Presentation
Download Presentation
Authors : Shu -Chen Cheng,

Loading in 2 Seconds...

play fullscreen
1 / 25

Authors : Shu -Chen Cheng, - PowerPoint PPT Presentation


  • 109 Views
  • Uploaded on

Estimation of Item Difficulty Index Based on Item Response Theory for Computerized Adaptive Testing. Authors : Shu -Chen Cheng,. Guan-Yu Chen. Outline. 1. Introduction 2. Literature Reviews 3. Methods 4. Experiments and Results 5. Conclusions. 1. Introduction (1/2 ).

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Authors : Shu -Chen Cheng,' - komala


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
estimation of item difficulty index based on item response theory for computerized adaptive testing

Estimation of Item Difficulty Index Based on Item Response Theory for Computerized Adaptive Testing

Authors:Shu-Chen Cheng,

Guan-Yu Chen

slide2

Outline

1.Introduction

2.Literature Reviews

3. Methods

4. Experiments and Results

5. Conclusions

slide3

1.Introduction(1/2)

  • Computerized Adaptive Testing
    • Item Response Theory
    • Advantage: Personalized test, Shorter test length.
    • Shortcoming: The number of pre-test samples.
      • IRT-1PL: 20 items, 200 testees(Wright & Stone, 1979)
      • IRT-2PL: 30 items, 500 testees(Hulin et al., 1982)
      • IRT-3PL: 60 items, 1000 testees(Hulin et al., 1982)

( There are 1,513 items in our item bank!)

1 introduction 2 2
1.Introduction(2/2)
  • Test System = Item Bank + Item Selection
  • Item Difficulty Index Answers Abnormal Rate
  • Dynamic Item Selection Strategy  Particle Swarm Optimization
slide5

2.Literature Reviews

2.1Computerized Adaptive Testing

2.2 Item Difficulty Index

2.3 Item Response Theory

slide6

2.1 Computerized Adaptive Testing(1/2)

  • To select the item that its difficulty is most consistent with testee’s ability.
  • To assess testee’s ability immediately.
  • The difficulty of next item is affected by previous answer.
slide7

2.1 Computerized Adaptive Testing(2/2)

  • To test for different abilities through dynamitic item selection strategy.
    • High ability testee  No too easy items.
    • Low ability testee  No too difficult items.
  • A personalized test.
2 2 item difficulty index 1 2
2.2 Item Difficulty Index (1/2)
  • Method 1:

P : Item difficulty.

R : The number of correct answers.

N : The number of total testees.

2 2 item difficulty index 2 2
2.2 Item Difficulty Index (2/2)
  • Method 2:

P : Item difficulty.

PH : Correct rate of high score group.

PL : Correct rate of low score group.

(Generally take 25%, 27%, 33%, etc.)

slide10

2.3 Item Response Theory(1/2)

  • Item Response Theory (Lord, 1980)
    • To estimate testee’s ability, aptitude, or location of other continuous psychological interval by the information of their item responses.
    • Ability location  Item response (Psychometric theory)
    • In addition to the model of IRT, without any other information to describe the item responses.
slide11

2.3 Item Response Theory(2/2)

  • Three-Parameter Logistic Model(Birnbaum, 1968)

Pi(θ) : Correct probability of item i for ability θ.

ai: Discrimination parameter of item i.

bi: Difficulty parameter of item i.

ci: Guess parameter of item i.

slide12

3. Methods (1/4)

  • Answers
    • Testees’ ability>Item difficultyindex

 Most testees are supposed to answer correctly.

    • Testees’ ability<Item difficultyindex

 Most testees are supposed to answer wrong.

    • Testees’ ability=Item difficultyindex

 The correct answer rate is 50%.

slide13

3. Methods (2/4)

  • Answers Abnormal
    • Violations of any one of these above 3 assumptions among answers are answers abnormal.
      • 1st group with wrong answers.(Testee’s ability >Item difficulty)
      • 2nd group with correct answers. (Testee’s ability <Item difficulty)
      • 3rd group, correct answer rate ≠ 0.5. (Testee’s ability =Item difficulty)
slide14

3. Methods (3/4)

  • Answers Abnormal Rate

:Answers abnormal rate of item i with difficulty j.

  • h :1st group (Testee’s ability >Item difficulty).
  • l :2nd group(Testee’s ability <Item difficulty).
  • e :3rd group (Testee’s ability =Item difficulty).

T:The number of correct answers.

F:The number of wrong answers.

N :The number of total testees.

slide15

3. Methods (4/4)

  • Item Difficulty

Difficulty j, let

be the smallest.

: Item difficulty index of item i.

:

Answers abnormal rate of

item iwith difficulty j.

15

slide16

4. Experiments and Results

4.1 System Descriptions

4.2 Experiment Descriptions

4.3 Results and Discussions

slide17

4.1 System Descriptions (1/3)

http://ilearning.csie.stust.edu.tw/EST/Dedault.aspx

4 1 system descriptions 3 3
4.1 System Descriptions (3/3)

PSO Dynamic Item Selection Strategy

  • Item Difficulty
  • Knowledge Weights
  • Item Exposure Rate
slide20

4.2 Experiment Descriptions

  • Method: Online test
  • Item Bank:
    • Items: 1,513
    • Initial Difficulty: 0.5(9 levels, 0.1~0.9)
  • Participants:
    • Students: 51
    • Initial Ability: 0.2(9 levels, 0.1~0.9)
  • Periods: 6weeks
slide24

5. Conclusions

  • Each test item is treated as independent, and the item difficulty can be estimated individually. Therefore, the item bank can be expanded easily at any time.
  • The estimation based on the answers abnormal rate proposed in this study can estimate the item difficulty index quickly and reasonably without too many pre-test samples.
the end

The End ~

Thanks for your attention!