impact of interruptions on test scores in indiana
Download
Skip this Video
Download Presentation
Impact of Interruptions on Test Scores in Indiana

Loading in 2 Seconds...

play fullscreen
1 / 21

Impact of Interruptions on Test Scores in Indiana - PowerPoint PPT Presentation


  • 106 Views
  • Uploaded on

Impact of Interruptions on Test Scores in Indiana. Richard Hill June 25, 2014. Two Parts. Initial study prior to presentation to Legislative committee Quick Outline Details in paper: http://www.nciea.org/publication_PDFs/ISTEP%20RH072713.pdf Follow-up study

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about ' Impact of Interruptions on Test Scores in Indiana' - konala


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
two parts
Two Parts
  • Initial study prior to presentation to Legislative committee
    • Quick Outline
    • Details in paper: http://www.nciea.org/publication_PDFs/ISTEP%20RH072713.pdf
  • Follow-up study
    • http://www.nciea.org/publication_PDFs/GainsMadebyInterruptedStudents_RH092513.pdf
three confounding factors
Three Confounding Factors
  • New policy for retention of students in Grade 3
  • Transition from paper and pencil to computer administration
  • Interruptions
change from paper and pencil to computer administration
Change from Paper and Pencil to Computer Administration
  • 2009 and 2010 – < 10%
  • 2011 – 36%
  • 2012 – 71%
  • 2013 – 95%
change not even across grades
Change Not Even Across Grades
  • 2012
    • Grade 6 – 66%
    • Grade 7 – 86%
    • Grade 8 – 92%
additional center analyses
Additional Center Analyses
  • School-by-school improvement at same grade between 2012 and 2013
  • School-by-school gain, following same cohort of students across grades
  • Student-level gain by students matched from 2012 to 2013
ctb analyses
CTB Analyses
  • Group Analyses
    • Overall statewide averages
    • Interrupted vs. non-interrupted within 2013
    • Scores before interruption vs. scores after interruption
  • Individual Analyses
    • Before vs. after interruption
    • Performance predicted from previous tests
presentation to legislative committee
Presentation to Legislative Committee
  • Was overall finding of no change a function of two factors?
    • Some students adversely affected by interruption
    • Other students taking advantage of interruption to learn answers from outside sources, then changing answers when testing restarted
available data
Available Data
  • CTB could provide A+B and C+D
  • That is, they knew at which item student was interrupted, and they knew, by item, whether student had made a change
  • But they couldn’t tell (easily) exactly the time the change was made
analysis of table
Analysis of Table
  • Cell B is the event of interest—changes made after interruption to items presented before interruption
  • But C = 0, and A should equal D
  • So B = (A + B) – (C + D)
conclusion
Conclusion
  • Concern raised by legislators was confirmed—data show that students changed answers from wrong to right more often after interruption than before
  • Impact on overall results was negligible—less than 0.2 scaled score points (on tests with standard deviations of 50-75)
conclusions from second analysis
Conclusions from Second Analysis
  • Students reported by CTB as interrupted had higher rates of change from wrong to right
  • Again, estimated impact is less than 0.2 scaled score points
  • Students reported as interrupted by locals had lower rates of change from wrong to right
conclusion from second analysis
Consistent with first

Impact on overall results negligible--

Conclusion from Second Analysis
ad