Impact of interruptions on test scores in indiana
Sponsored Links
This presentation is the property of its rightful owner.
1 / 21

Impact of Interruptions on Test Scores in Indiana PowerPoint PPT Presentation


  • 83 Views
  • Uploaded on
  • Presentation posted in: General

Impact of Interruptions on Test Scores in Indiana. Richard Hill June 25, 2014. Two Parts. Initial study prior to presentation to Legislative committee Quick Outline Details in paper: http://www.nciea.org/publication_PDFs/ISTEP%20RH072713.pdf Follow-up study

Download Presentation

Impact of Interruptions on Test Scores in Indiana

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Impact of Interruptions on Test Scores in Indiana

Richard Hill

June 25, 2014


Two Parts

  • Initial study prior to presentation to Legislative committee

    • Quick Outline

    • Details in paper: http://www.nciea.org/publication_PDFs/ISTEP%20RH072713.pdf

  • Follow-up study

    • http://www.nciea.org/publication_PDFs/GainsMadebyInterruptedStudents_RH092513.pdf


Three Confounding Factors

  • New policy for retention of students in Grade 3

  • Transition from paper and pencil to computer administration

  • Interruptions


Changes in Mean Scores over YearsELA


Changes in Mean Scores over YearsMathematics


Change from Paper and Pencil to Computer Administration

  • 2009 and 2010 – < 10%

  • 2011 – 36%

  • 2012 – 71%

  • 2013 – 95%


Change Not Even Across Grades

  • 2012

    • Grade 6 – 66%

    • Grade 7 – 86%

    • Grade 8 – 92%


Additional Center Analyses

  • School-by-school improvement at same grade between 2012 and 2013

  • School-by-school gain, following same cohort of students across grades

  • Student-level gain by students matched from 2012 to 2013


CTB Analyses

  • Group Analyses

    • Overall statewide averages

    • Interrupted vs. non-interrupted within 2013

    • Scores before interruption vs. scores after interruption

  • Individual Analyses

    • Before vs. after interruption

    • Performance predicted from previous tests


Presentation to Legislative Committee

  • Was overall finding of no change a function of two factors?

    • Some students adversely affected by interruption

    • Other students taking advantage of interruption to learn answers from outside sources, then changing answers when testing restarted


Model of Concern


Available Data

  • CTB could provide A+B and C+D

  • That is, they knew at which item student was interrupted, and they knew, by item, whether student had made a change

  • But they couldn’t tell (easily) exactly the time the change was made


Analysis of Table

  • Cell B is the event of interest—changes made after interruption to items presented before interruption

  • But C = 0, and A should equal D

  • So B = (A + B) – (C + D)


Example—Grade 3 Mathematics, Session 1


Results for Grades 3-5


Results for Grades 6-8


Conclusion

  • Concern raised by legislators was confirmed—data show that students changed answers from wrong to right more often after interruption than before

  • Impact on overall results was negligible—less than 0.2 scaled score points (on tests with standard deviations of 50-75)


% Making No Changes from Wrong to Right (Grades 3-5, Math Only)


Average % of Change from Wrong to Right (Grades 3-5, Math Only)


Conclusions from Second Analysis

  • Students reported by CTB as interrupted had higher rates of change from wrong to right

  • Again, estimated impact is less than 0.2 scaled score points

  • Students reported as interrupted by locals had lower rates of change from wrong to right


Consistent with first

Impact on overall results negligible--

Conclusion from Second Analysis


  • Login