impact of interruptions on test scores in indiana n.
Download
Skip this Video
Download Presentation
Impact of Interruptions on Test Scores in Indiana

Loading in 2 Seconds...

play fullscreen
1 / 21

Impact of Interruptions on Test Scores in Indiana - PowerPoint PPT Presentation


  • 106 Views
  • Uploaded on

Impact of Interruptions on Test Scores in Indiana. Richard Hill June 25, 2014. Two Parts. Initial study prior to presentation to Legislative committee Quick Outline Details in paper: http://www.nciea.org/publication_PDFs/ISTEP%20RH072713.pdf Follow-up study

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Impact of Interruptions on Test Scores in Indiana' - konala


Download Now An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
two parts
Two Parts
  • Initial study prior to presentation to Legislative committee
    • Quick Outline
    • Details in paper: http://www.nciea.org/publication_PDFs/ISTEP%20RH072713.pdf
  • Follow-up study
    • http://www.nciea.org/publication_PDFs/GainsMadebyInterruptedStudents_RH092513.pdf
three confounding factors
Three Confounding Factors
  • New policy for retention of students in Grade 3
  • Transition from paper and pencil to computer administration
  • Interruptions
change from paper and pencil to computer administration
Change from Paper and Pencil to Computer Administration
  • 2009 and 2010 – < 10%
  • 2011 – 36%
  • 2012 – 71%
  • 2013 – 95%
change not even across grades
Change Not Even Across Grades
  • 2012
    • Grade 6 – 66%
    • Grade 7 – 86%
    • Grade 8 – 92%
additional center analyses
Additional Center Analyses
  • School-by-school improvement at same grade between 2012 and 2013
  • School-by-school gain, following same cohort of students across grades
  • Student-level gain by students matched from 2012 to 2013
ctb analyses
CTB Analyses
  • Group Analyses
    • Overall statewide averages
    • Interrupted vs. non-interrupted within 2013
    • Scores before interruption vs. scores after interruption
  • Individual Analyses
    • Before vs. after interruption
    • Performance predicted from previous tests
presentation to legislative committee
Presentation to Legislative Committee
  • Was overall finding of no change a function of two factors?
    • Some students adversely affected by interruption
    • Other students taking advantage of interruption to learn answers from outside sources, then changing answers when testing restarted
available data
Available Data
  • CTB could provide A+B and C+D
  • That is, they knew at which item student was interrupted, and they knew, by item, whether student had made a change
  • But they couldn’t tell (easily) exactly the time the change was made
analysis of table
Analysis of Table
  • Cell B is the event of interest—changes made after interruption to items presented before interruption
  • But C = 0, and A should equal D
  • So B = (A + B) – (C + D)
conclusion
Conclusion
  • Concern raised by legislators was confirmed—data show that students changed answers from wrong to right more often after interruption than before
  • Impact on overall results was negligible—less than 0.2 scaled score points (on tests with standard deviations of 50-75)
conclusions from second analysis
Conclusions from Second Analysis
  • Students reported by CTB as interrupted had higher rates of change from wrong to right
  • Again, estimated impact is less than 0.2 scaled score points
  • Students reported as interrupted by locals had lower rates of change from wrong to right
conclusion from second analysis
Consistent with first

Impact on overall results negligible--

Conclusion from Second Analysis