Impact of interruptions on test scores in indiana
Download
1 / 21

Impact of Interruptions on Test Scores in Indiana - PowerPoint PPT Presentation


  • 93 Views
  • Uploaded on
  • Presentation posted in: General

Impact of Interruptions on Test Scores in Indiana. Richard Hill June 25, 2014. Two Parts. Initial study prior to presentation to Legislative committee Quick Outline Details in paper: http://www.nciea.org/publication_PDFs/ISTEP%20RH072713.pdf Follow-up study

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha

Download Presentation

Impact of Interruptions on Test Scores in Indiana

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Impact of Interruptions on Test Scores in Indiana

Richard Hill

June 25, 2014


Two Parts

  • Initial study prior to presentation to Legislative committee

    • Quick Outline

    • Details in paper: http://www.nciea.org/publication_PDFs/ISTEP%20RH072713.pdf

  • Follow-up study

    • http://www.nciea.org/publication_PDFs/GainsMadebyInterruptedStudents_RH092513.pdf


Three Confounding Factors

  • New policy for retention of students in Grade 3

  • Transition from paper and pencil to computer administration

  • Interruptions


Changes in Mean Scores over YearsELA


Changes in Mean Scores over YearsMathematics


Change from Paper and Pencil to Computer Administration

  • 2009 and 2010 – < 10%

  • 2011 – 36%

  • 2012 – 71%

  • 2013 – 95%


Change Not Even Across Grades

  • 2012

    • Grade 6 – 66%

    • Grade 7 – 86%

    • Grade 8 – 92%


Additional Center Analyses

  • School-by-school improvement at same grade between 2012 and 2013

  • School-by-school gain, following same cohort of students across grades

  • Student-level gain by students matched from 2012 to 2013


CTB Analyses

  • Group Analyses

    • Overall statewide averages

    • Interrupted vs. non-interrupted within 2013

    • Scores before interruption vs. scores after interruption

  • Individual Analyses

    • Before vs. after interruption

    • Performance predicted from previous tests


Presentation to Legislative Committee

  • Was overall finding of no change a function of two factors?

    • Some students adversely affected by interruption

    • Other students taking advantage of interruption to learn answers from outside sources, then changing answers when testing restarted


Model of Concern


Available Data

  • CTB could provide A+B and C+D

  • That is, they knew at which item student was interrupted, and they knew, by item, whether student had made a change

  • But they couldn’t tell (easily) exactly the time the change was made


Analysis of Table

  • Cell B is the event of interest—changes made after interruption to items presented before interruption

  • But C = 0, and A should equal D

  • So B = (A + B) – (C + D)


Example—Grade 3 Mathematics, Session 1


Results for Grades 3-5


Results for Grades 6-8


Conclusion

  • Concern raised by legislators was confirmed—data show that students changed answers from wrong to right more often after interruption than before

  • Impact on overall results was negligible—less than 0.2 scaled score points (on tests with standard deviations of 50-75)


% Making No Changes from Wrong to Right (Grades 3-5, Math Only)


Average % of Change from Wrong to Right (Grades 3-5, Math Only)


Conclusions from Second Analysis

  • Students reported by CTB as interrupted had higher rates of change from wrong to right

  • Again, estimated impact is less than 0.2 scaled score points

  • Students reported as interrupted by locals had lower rates of change from wrong to right


Consistent with first

Impact on overall results negligible--

Conclusion from Second Analysis


ad
  • Login