informing policy state longitudinal data systems
Download
Skip this Video
Download Presentation
Informing Policy: State Longitudinal Data Systems

Loading in 2 Seconds...

play fullscreen
1 / 51

Informing Policy: State Longitudinal Data Systems - PowerPoint PPT Presentation


  • 84 Views
  • Uploaded on

Informing Policy: State Longitudinal Data Systems. Jane Hannaway, Director The Urban Institute CALDER www.caldercenter.org. State of U.S. Education. ½ of minority students graduate from high school 4 grade level gap between white and minority students by 12 th grade

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about ' Informing Policy: State Longitudinal Data Systems' - christian-townsend


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
informing policy state longitudinal data systems
Informing Policy: State Longitudinal Data Systems

Jane Hannaway, Director

The Urban Institute

CALDER

www.caldercenter.org

state of u s education
State of U.S. Education
  • ½ of minority students graduate from high school
  • 4 grade level gap between white and minority students by 12th grade
  • 15% of minorities earn BAs w/in 9 years of 9th grade
the will and the way
The WILL and the WAY
  • The WILL
    • Left, Right, Center
    • Agreement on education crisis
    • Strange bedfellows
  • The WAY
    • Few, but growing, guideposts
finding the way with evidence a new day
Finding the WAY with Evidence-A New Day-
  • Who has the evidence?
    • States have the makings of the evidence
  • Where are the makings?
    • State administrative data systems
  • Why do states have it?
    • Important effect of NCLB
  • Why important?
    • Address questions never before possible
research background what we know
Research Background: What We Know
  • Teachers matter- single most important schooling contributor to student outcomes
  • Wide variation in teacher effectiveness. Some teachers are simply much better than others
  • Standard measures of teacher quality not much related to effectiveness, but directly related to spending.
research background what we don t know
Research Background:What We Don’t Know
  • What is it about teachers that matters?
3 research probes
3 Research Probes
  • Teacher Maldistribution
  • Teacher Selection
  • Teacher Mobility
teacher maldistribution 1
Teacher Maldistribution 1
  • Comparison of VA of teachers in high/ low poverty schools
  • North Carolina and Florida
  • Findings
    • Low poverty - higher va, but not much
    • High poverty – larger variation in school
slide11
Novice teachers are less effective than experienced teachers.
  • Returns to experience taper off 3-5 years.
distribution of value added of elementary math teachers in high poverty schools
Distribution of Value-Added of Elementary Math Teachers in High Poverty Schools

Solid line: Novice teachers

Dash line: Teachers with 1-2 years of experience

Dotted line: Teachers with 3-5 years of experience

distribution of value added of elementary math teachers in lower poverty schools
Distribution of Value-Added of Elementary Math Teachers in Lower Poverty Schools

Solid line: Novice teachers

Dash line: Teachers with 1-2 years of experience

Dotted line: Teachers with 3-5 years of experience

teacher maldistribution 2
Teacher Maldistribution 2
  • New York City
    • Phasing out of emergency certification
    • Introduction of alternative route teachers
can change predicted effectiveness by selection up front
Can change predicted effectiveness by selection up-front
  • Meaningful difference based only on attributes, though monitoring, development and selective retention also needed
teacher selection
Teacher Selection
  • Teach for America
    • North Carolina
    • Secondary school
    • Mainly math and science
tfa findings high school
TFA Findings – high school

Student FE, Math subjects

All TFA coefficients are significant at the .05 level.

teacher mobility
Teacher Mobility
  • Mobility highest at most challenging schools
  • The worst teachers are the first to leave
  • General tendency to move to more affluent schools
topic of the day performance incentives
Topic of the Day: Performance Incentives
  • Objective??
    • Recruitment/ selection
    • Retention/ deselection
    • Increase performance thru effort
issues
Issues
  • How good are the measures?
  • Individual vs school rewards?
  • Teachers without test scores?
va measures
VA Measures
  • Problems
    • Year to year variability
    • Measurement error
    • Sorting
  • How serious?
    • Less serious for policy research
    • More serious for individual stakes
predicting performance
Predicting Performance
  • Using first 2 yrs of performance – top to top/ bottom to bottom quintile
    • Goldhaber and Hansen (NC): 46%/ 44%
    • Koedel and Betts (SanDiego): 29%/ 35%
    • Sass (Florida): 22-32%/ 24-24%
policy implications
Policy Implications
  • Use VA freely for research
  • Use VA carefully for individual teacher judgments
    • Important information
    • Corrorboration
  • More years are better
    • Move tenure decision out!
research questions
Research Questions
  • Are teachers in high poverty schools more/ less effective (value-added) than teachers in lower poverty schools?
  • Do school factors affect differences in the value-added of high poverty and lower poverty teachers?
  • Do teacher qualifications affect differences in the value-added of high poverty and lower poverty teachers?
slide28
Data
  • Florida (2000/01- 2004/05)
    • Elementary
    • Student achievement – FCAT-SSS
      • Grades 3-10
    • Teacher links
      • Assignment, certification, experience, education
  • North Carolina (2000/1-2004/5)
    • Elementary
    • Student achievement
      • EOG – grades 3-8
      • EOC – secondary subjects
    • Teacher linked through proctor and verification
      • Assignment, certification, experience, education
definitions
Definitions
  • High poverty elementary schools (>70% FRL students)
  • Lower poverty elementary schools (<70% FRL students)
  • Very low poverty schools (<30% FRL students).
nc student teacher link
NC Student-Teacher Link

EOC student-level records

Aggregate to EOC

test classrooms by school,

year, subject, proctor id

Decision Rules

Match if teacher and proctor id identical and  fit statistic < 1.5.

sample restrictions
Sample Restrictions
  • Exclude charter schools
  • Exclude schools that switch high poverty to lower poverty status
  • Only classrooms w/ 10-40 students
  • Only self-contained elementary classrooms
analytic sample
Analytic Sample

Note: We focus on elementary schools, grades 3-5 where poverty information is most reliable. We exclude teachers from charter schools and we exclude classrooms with <10 students or >40 students in our samples.

methodological challenges
Methodological Challenges
  • Non-random sorting of teachers and students
  • Distinguishing teacher and school effects
  • Precision in Teacher Effects Estimates
  • Sources of Teacher Effectiveness Differentials
descriptive findings student performance
Descriptive Findings: Student Performance

* Differences between the given estimate and the corresponding estimates for schools with 70-100% FRL students significant at ≤ 5% and ** differences significant at ≤ 1%.

distribution of value added of elementary reading teachers in lower poverty schools
Distribution of Value-Added of Elementary Reading Teachers in Lower Poverty Schools

Solid line: Novice teachers

Dash line: Teachers with 1-2 years of experience

Dotted line: Teachers with 3-5 years of experience

distribution of value added of elementary reading teachers in high poverty schools
Distribution of Value-Added of Elementary Reading Teachers in High Poverty Schools

Solid line: Novice teachers

Dash line: Teachers with 1-2 years of experience

Dotted line: Teachers with 3-5 years of experience

slide47
Sources of Difference in Teacher Value-Added Between High-Poverty and Lower-Poverty Elementary Schools
sensitivity analysis
Sensitivity Analysis
  • School Effect
  • Empirical Bayes Adjustment
conclusions
Conclusions
  • Teachers in high poverty schools, on average, are less effective than teachers in lower poverty schools.
    • Changing schools (high poverty/lower poverty) does not affect teacher effectiveness
  • There is greater teacher variation within high poverty schools than within lower poverty schools.
conclusions con t
Conclusions (con’t)
  • Differences in teachers in High Poverty and Lower Poverty schools:
    • only weakly related to teacher qualifications
    • more strongly related to marginal effect of qualifications (experience)
    • not explained by school poverty level
study limitations
Study Limitations
  • Issues with value-added measures
    • separating current teacher contributions from other current contributions
      • E.g., current family circumstances

- dynamic sorting

      • sorting on time variant characteristics
    • Instability of measures
      • E.g., measurement error, motivation
ad