slide1
Download
Skip this Video
Download Presentation
The 6th Annual EALTA Conference, Turku 2009 – F. Kaftandjieva: “ Flaw & Order: World 1 and World 2 of Item Difficulty

Loading in 2 Seconds...

play fullscreen
1 / 17

The 6th Annual EALTA Conference, Turku 2009 – F. Kaftandjieva: “ Flaw & Order: World 1 and World 2 of Item Difficulty - PowerPoint PPT Presentation


  • 115 Views
  • Uploaded on

The 6th Annual EALTA Conference, Turku 2009 – F. Kaftandjieva: “ Flaw & Order: World 1 and World 2 of Item Difficulty – Synergies and Tensions”. B1. World 1 Observed Item Difficulty. World 3 Misplacement Index (MPI). World 2 Perceived Item Difficulty. C2 . C1 . B2 . B1 . A2 . A1 .

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'The 6th Annual EALTA Conference, Turku 2009 – F. Kaftandjieva: “ Flaw & Order: World 1 and World 2 of Item Difficulty ' - hamlin


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
slide2
The 6th Annual EALTA Conference, Turku 2009 – F. Kaftandjieva: “Flaw & Order: World 1 and World 2 of Item Difficulty – Synergies and Tensions”

Turku 2009 – F. Kaftandjieva

slide3
B1

World 1

Observed Item Difficulty

World 3

Misplacement Index (MPI)

World 2

Perceived Item Difficulty

Turku 2009 – F. Kaftandjieva

magic clouds intra judge consistency
C2

C1

B2

B1

A2

A1

Magic Clouds: Intra-judge Consistency

‘At what CEFR level

can a test taker already answer

the following item correctly?’

A2

B1

Turku 2009 – F. Kaftandjieva

snork maiden s art work judgment task
Snork Maiden’s Art Work: Judgment Task

8

1

4

6

9

A2

B1

A1

A2

A1

3

2

7

5

B1

A1

A2

A2

Turku 2009 – F. Kaftandjieva

snork s wish misplacement index mpi
C

C+D

8

3

1

5

7

9

4

2

6

A2

A1

A1

A1

A2

A2

B1

B1

A2

Snork’s wish: Misplacement Index (MPI)

MPI

0.40

0.83

0.83

0.83

1.00

1.00

0.86

0.86

0.60

3/5

6/7

6/7

5/5

5/5

5/6

5/6

5/6

2/5

A machine for finding things out: a machine that calculates whether things are right or wrong, good or bad.

0.81

Easy<

Observed Item Difficulty

>Difficult

Turku 2009 – F. Kaftandjieva

the muskrat s philosophical question how good is good enough
MPI =

The Probability of a Correct Estimation of Order

The Muskrat’s Philosophical Question:How Good Is Good Enough?

MPI = 0.50

MPI > 0.65

P-R-E > 50%

MPI > 0.75

Turku 2009 – F. Kaftandjieva

the hemulen who loved mpi indices the collection
Mean = 0.71

SD = 0.10

n = 381

0.0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1.0

MPIjudge

The Hemulen Who Loved MPI Indices:The Collection

39%

76%

0.65

0.75

Judges = 187

Sub-tests = 30

Items = 2089

Languages:

En – 232

Ge – 22

Sw – 92

Fi – 35

Skills:

R – 187

L – 77

W – 7

V – 29

G – 28

GV – 53

Turku 2009 – F. Kaftandjieva

the hemulen who loved mpi indices skill comparison
n = 77

G&V

R

R

L

n = 53

The Hemulen Who Loved MPI Indices:Skill Comparison

Turku 2009 – F. Kaftandjieva

the hemulen who loved mpi indices group comparison reading
The Hemulen Who Loved MPI Indices:Group Comparison (Reading)

n = 187

6 < ngr< 34

Turku 2009 – F. Kaftandjieva

the hemulen who loved mpi indices method comparison
Examinee

Centered

6J – 151E

Examinee

Centered

9J – 331E

0.88

0.87

Test

Centered

83J – 41I

Test

Centered

15J – 234I

0.79

0.71

The Hemulen Who Loved MPI Indices:Method Comparison

MPI

Case 1

Case 2

Turku 2009 – F. Kaftandjieva

the island of the hattifatteners item specific mpi indices
0.0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1.0

MPIitem

The Island of the Hattifatteners:Item Specific MPI Indices

Items = 2089

Tests = 30

Judges = 187

n = 25431

Mean = 0.72

SD = 0.21

71%

Turku 2009 – F. Kaftandjieva

moominpappa s notes
Item difficulty

and Item format tend to affect MPI.

More synergies

than tensions!

A program

for calculating MPI is available

on request.

MPI is

Group, Skill & Method dependent.

More research

is needed.

Moominpappa’s Notes

Turku 2009 – F. Kaftandjieva

slide15
Thanks!

Turku 2009 – F. Kaftandjieva

the hemulen who loved mpi indices a closer look
.97

.97

.98

1.0

The Hemulen Who Loved MPI Indices:A Closer Look

Turku 2009 – F. Kaftandjieva

ad