item analyses by disability category l.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
Item Analyses by Disability Category PowerPoint Presentation
Download Presentation
Item Analyses by Disability Category

Loading in 2 Seconds...

play fullscreen
1 / 18
rane

Item Analyses by Disability Category - PowerPoint PPT Presentation

97 Views
Download Presentation
Item Analyses by Disability Category
An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript

  1. Item Analyses by Disability Category Ross Moen, Martha Thurlow, Kentaro Kato October 11, 2007 PARA TAC Partnership for Accessible Reading Assessment (PARA): A collaboration between the University of Minnesota’sNational Center on Educational Outcomes and the Department of Curriculum & Instruction; CRESST, University of California, Davis; and Westat www.readingassessment.info

  2. Data– Item responses from 3rd and 5th graders (approx. 60,000 for each grade) in a state reading assessment (46 multiple-choice items for each grade) Comparison– Students without disabilities vs. students with SLI, LD, or EBD Analysis– Multistep multinomial logistic regression (DIF/DDF) and multistep binary logistic regression (differential missing response functioning); Items with p > .01 and R2 difference > .003 are flagged for potential item bias; Response characteristic curves (RCCs) were plotted Methodology

  3. Observation #1 • Statistically significant DIF/DDF was found for numerous items for all 3 groups of students with disabilities in both grades. • Large N can make even trivial differences statistically significant, so a more stringent criterion of R2> .003 was applied. • In both grades, only students with LD had items meeting the more stringent .003 criterion. For students with SLI and EBD, .003 is not even on the scale.

  4. Figure 1. DIF/DDF R2 difference between students without disabilities and students with LD for grade 3

  5. Figure 2. DIF/DDF R2 difference between students without disabilities and students with SLI for grade 3

  6. Figure 3. DIF/DDF R2 difference between students without disabilities and students with EBD for grade 3

  7. Observation #2 • Response characteristic curves suggest that not all DIF/DDF indicates bias against students with disabilities. • Examining RCCs for 4 third grade items where a distractor contributes nearly as much or more as the correct response to DIF/DDF for students with LD illustrates this.

  8. Grade 3 Items showing DIF/DDF for Students with LD

  9. RCCs for Item #6 A0 through D0 = students without disabilities; A1 through D1 = students with LD; the correct response is B

  10. RCCs for Item #25 A0 through D0 = students without disabilities; A1 through D1 = students with LD; the correct response is B

  11. RCCs for Item #42 A0 through D0 = students without disabilities; A1 through D1 = students with LD; the correct response is B

  12. RCCs for Item #48 A0 through D0 = students without disabilities; A1 through D1 = students with LD; the correct response is C

  13. Questions #1 • What interpretations might be made to account for these differences? • Does the test performance of students with LD look different from other students in ways besides DIF/DDF?

  14. Score Distribution of Grade 3 Students Without Disability

  15. Score Distribution of Grade 3 Students With SLI

  16. Score Distribution of Grade 3 Students With EBD

  17. Score Distribution of Grade 3 Students With LD

  18. Final Question • Do these observations – based on these data – merit further examination?