sensitivity of teacher value added estimates to student and peer control variables n.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
Sensitivity of Teacher Value-Added Estimates to Student and Peer Control Variables PowerPoint Presentation
Download Presentation
Sensitivity of Teacher Value-Added Estimates to Student and Peer Control Variables

Loading in 2 Seconds...

play fullscreen
1 / 12

Sensitivity of Teacher Value-Added Estimates to Student and Peer Control Variables - PowerPoint PPT Presentation


  • 62 Views
  • Uploaded on

Sensitivity of Teacher Value-Added Estimates to Student and Peer Control Variables. March 2012 Presentation to the Association of Education Finance and Policy Conference Matt Johnson  Stephen Lipscomb  Brian Gill. VAMs Used Today Differ in Their Specifications. Research Questions.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Sensitivity of Teacher Value-Added Estimates to Student and Peer Control Variables' - lois


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
sensitivity of teacher value added estimates to student and peer control variables

Sensitivity of Teacher Value-Added Estimates to Student and Peer Control Variables

March 2012

Presentation to the Association of Education Finance and Policy Conference

Matt Johnson  Stephen Lipscomb Brian Gill

research questions
Research Questions
  • How sensitive are teacher value-added model (VAM) estimates to changes in the model specification?
    • Student characteristics
    • Classroom characteristics
    • Multiple years of prior scores
  • How sensitive are estimates to loss of students from sample due to missing prior scores?
preview of main results
Preview of Main Results
  • Teacher value-added estimates are not highly sensitive to inclusion of:
    • Student characteristics (correlation ≥ 0.990)
    • Multiple years of prior scores (correlation ≥ 0.987)
  • Estimates are more sensitive to inclusion of classroom characteristics (correlation = 0.915 to .955)
  • Estimates are not very sensitive to loss of students with missing prior test scores from sample (correlation = 0.992)
    • Precision increases when two prior scores are used but fewer teacher VAM estimates are produced
baseline model
Baseline Model
  • Explore sensitivity to several specifications:
    • Exclude score from two prior years (Yi,t-2)
    • Exclude student characteristics (Xi,t)
    • Include class average characteristics
  • Student data from a medium-sized urban district for 2008–2009 to 2010–2011 school years
  • All models run using the same set of student observations
  • Instrument using opposite subject prior score to control for measurement error
correlation of 6th grade teacher estimates relative to baseline vam specification
Correlation of 6th-Grade Teacher Estimates Relative to Baseline VAM Specification

Baseline: Student Characteristics and Prior Scores from t-1 and t-2

Findings are based on VAM estimates from 2008–2009 to 2010–2011 on the same sample of students.

percentage of 6th grade reading teachers in effectiveness quintiles by vam specification
Percentage of 6th-Grade Reading Teachers in Effectiveness Quintiles, by VAM Specification

Findings are based on VAM estimates for 99 reading teachers in grade 6 from 2008–2009 to 2010–2011 for a medium-sized, urban district. Correlation with baseline = 0.996.

percentage of 6th grade reading teachers in effectiveness quintiles by vam specification1
Percentage of 6th-Grade Reading Teachers in Effectiveness Quintiles, by VAM Specification

Findings are based on VAM estimates for 99 reading teachers in grade 6 from 2008–2009 to 2010–2011 for a medium-sized, urban district. Correlation with baseline = 0.915.

one or two years of prior scores
One or Two Years of Prior Scores?
  • Benefits of including two prior years:
    • More accurate measure of student ability
    • Increase in precision of estimates
  • Costs of using two prior years:
    • Students with missing prior scores dropped
    • Some teachers dropped from sample
  • Relative magnitude of costs/benefits?
one or two years of prior scores1
One or Two Years of Prior Scores?
  • Estimate two VAMs using one year of prior scores
    • First VAM includes all students
    • Second VAM restricts sample to students with nonmissing second prior year of scores
  • Correlation between teacher estimates: 0.992
  • Percentage of students dropped: 6.2
  • Percentage of teachers dropped: 3.9
  • Net increase in precision from using two prior years
    • Increase in average standard error of estimates: 2.3% when students with missing scores are dropped
    • Decrease in average standard error of estimates: 7.6% when second year of prior scores added
for more information
For More Information
  • Please contact
    • Matt Johnson
      • MJohnson@mathematica-mpr.com
    • Stephen Lipscomb
      • SLipscomb@mathematica-mpr.com
    • Brian Gill
      • BGill@mathematica-mpr.com