Informing policy state longitudinal data systems
Download
1 / 51

Informing Policy: State Longitudinal Data Systems - PowerPoint PPT Presentation


  • 83 Views
  • Uploaded on

Informing Policy: State Longitudinal Data Systems. Jane Hannaway, Director The Urban Institute CALDER www.caldercenter.org. State of U.S. Education. ½ of minority students graduate from high school 4 grade level gap between white and minority students by 12 th grade

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about ' Informing Policy: State Longitudinal Data Systems' - christian-townsend


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Informing policy state longitudinal data systems
Informing Policy: State Longitudinal Data Systems

Jane Hannaway, Director

The Urban Institute

CALDER

www.caldercenter.org


State of u s education
State of U.S. Education

  • ½ of minority students graduate from high school

  • 4 grade level gap between white and minority students by 12th grade

  • 15% of minorities earn BAs w/in 9 years of 9th grade


The will and the way
The WILL and the WAY

  • The WILL

    • Left, Right, Center

    • Agreement on education crisis

    • Strange bedfellows

  • The WAY

    • Few, but growing, guideposts


Finding the way with evidence a new day
Finding the WAY with Evidence-A New Day-

  • Who has the evidence?

    • States have the makings of the evidence

  • Where are the makings?

    • State administrative data systems

  • Why do states have it?

    • Important effect of NCLB

  • Why important?

    • Address questions never before possible


Research background what we know
Research Background: What We Know

  • Teachers matter- single most important schooling contributor to student outcomes

  • Wide variation in teacher effectiveness. Some teachers are simply much better than others

  • Standard measures of teacher quality not much related to effectiveness, but directly related to spending.


Research background what we don t know
Research Background:What We Don’t Know

  • What is it about teachers that matters?


3 research probes
3 Research Probes

  • Teacher Maldistribution

  • Teacher Selection

  • Teacher Mobility


Teacher maldistribution 1
Teacher Maldistribution 1

  • Comparison of VA of teachers in high/ low poverty schools

  • North Carolina and Florida

  • Findings

    • Low poverty - higher va, but not much

    • High poverty – larger variation in school





Distribution of value added of elementary math teachers in high poverty schools
Distribution of Value-Added of Elementary Math Teachers in High Poverty Schools

Solid line: Novice teachers

Dash line: Teachers with 1-2 years of experience

Dotted line: Teachers with 3-5 years of experience


Distribution of value added of elementary math teachers in lower poverty schools
Distribution of Value-Added of Elementary Math Teachers in Lower Poverty Schools

Solid line: Novice teachers

Dash line: Teachers with 1-2 years of experience

Dotted line: Teachers with 3-5 years of experience


Teacher maldistribution 2
Teacher Maldistribution 2 Lower Poverty Schools

  • New York City

    • Phasing out of emergency certification

    • Introduction of alternative route teachers





Can change predicted effectiveness by selection up front
Can change predicted effectiveness by selection up-front Schools

  • Meaningful difference based only on attributes, though monitoring, development and selective retention also needed


Teacher selection
Teacher Selection Schools

  • Teach for America

    • North Carolina

    • Secondary school

    • Mainly math and science


Tfa findings high school
TFA Findings – high school Schools

Student FE, Math subjects

All TFA coefficients are significant at the .05 level.


Teacher mobility
Teacher Mobility Schools

  • Mobility highest at most challenging schools

  • The worst teachers are the first to leave

  • General tendency to move to more affluent schools


Topic of the day performance incentives
Topic of the Day: SchoolsPerformance Incentives

  • Objective??

    • Recruitment/ selection

    • Retention/ deselection

    • Increase performance thru effort


Issues
Issues Schools

  • How good are the measures?

  • Individual vs school rewards?

  • Teachers without test scores?


Va measures
VA Measures Schools

  • Problems

    • Year to year variability

    • Measurement error

    • Sorting

  • How serious?

    • Less serious for policy research

    • More serious for individual stakes


Predicting performance
Predicting Performance Schools

  • Using first 2 yrs of performance – top to top/ bottom to bottom quintile

    • Goldhaber and Hansen (NC): 46%/ 44%

    • Koedel and Betts (SanDiego): 29%/ 35%

    • Sass (Florida): 22-32%/ 24-24%


Policy implications
Policy Implications Schools

  • Use VA freely for research

  • Use VA carefully for individual teacher judgments

    • Important information

    • Corrorboration

  • More years are better

    • Move tenure decision out!


Research questions
Research Questions Schools

  • Are teachers in high poverty schools more/ less effective (value-added) than teachers in lower poverty schools?

  • Do school factors affect differences in the value-added of high poverty and lower poverty teachers?

  • Do teacher qualifications affect differences in the value-added of high poverty and lower poverty teachers?


Data Schools

  • Florida (2000/01- 2004/05)

    • Elementary

    • Student achievement – FCAT-SSS

      • Grades 3-10

    • Teacher links

      • Assignment, certification, experience, education

  • North Carolina (2000/1-2004/5)

    • Elementary

    • Student achievement

      • EOG – grades 3-8

      • EOC – secondary subjects

    • Teacher linked through proctor and verification

      • Assignment, certification, experience, education


Definitions
Definitions Schools

  • High poverty elementary schools (>70% FRL students)

  • Lower poverty elementary schools (<70% FRL students)

  • Very low poverty schools (<30% FRL students).


Nc student teacher link
NC Student-Teacher Link Schools

EOC student-level records

Aggregate to EOC

test classrooms by school,

year, subject, proctor id

Decision Rules

Match if teacher and proctor id identical and  fit statistic < 1.5.


Sample restrictions
Sample Restrictions Schools

  • Exclude charter schools

  • Exclude schools that switch high poverty to lower poverty status

  • Only classrooms w/ 10-40 students

  • Only self-contained elementary classrooms


Analytic sample
Analytic Sample Schools

Note: We focus on elementary schools, grades 3-5 where poverty information is most reliable. We exclude teachers from charter schools and we exclude classrooms with <10 students or >40 students in our samples.


Methodological challenges
Methodological Challenges Schools

  • Non-random sorting of teachers and students

  • Distinguishing teacher and school effects

  • Precision in Teacher Effects Estimates

  • Sources of Teacher Effectiveness Differentials


Descriptive Findings: SchoolsElementary Student Demographics


Descriptive findings student performance
Descriptive Findings: SchoolsStudent Performance

* Differences between the given estimate and the corresponding estimates for schools with 70-100% FRL students significant at ≤ 5% and ** differences significant at ≤ 1%.


Descriptive findings teacher experience
Descriptive Findings: SchoolsTeacher Experience


Descriptive findings teacher qualifications
Descriptive Findings: SchoolsTeacher Qualifications


Distribution of value added of elementary reading teachers in lower poverty schools
Distribution of Value-Added of Elementary Reading Teachers in Lower Poverty Schools

Solid line: Novice teachers

Dash line: Teachers with 1-2 years of experience

Dotted line: Teachers with 3-5 years of experience


Distribution of value added of elementary reading teachers in high poverty schools
Distribution of Value-Added of Elementary Reading Teachers in High Poverty Schools

Solid line: Novice teachers

Dash line: Teachers with 1-2 years of experience

Dotted line: Teachers with 3-5 years of experience









Sources of Difference in Teacher Value-Added Between High-Poverty and Lower-Poverty Elementary Schools


Sensitivity analysis
Sensitivity Analysis High-Poverty and Lower-Poverty Elementary Schools

  • School Effect

  • Empirical Bayes Adjustment


Conclusions
Conclusions High-Poverty and Lower-Poverty Elementary Schools

  • Teachers in high poverty schools, on average, are less effective than teachers in lower poverty schools.

    • Changing schools (high poverty/lower poverty) does not affect teacher effectiveness

  • There is greater teacher variation within high poverty schools than within lower poverty schools.


Conclusions con t
Conclusions (con’t) High-Poverty and Lower-Poverty Elementary Schools

  • Differences in teachers in High Poverty and Lower Poverty schools:

    • only weakly related to teacher qualifications

    • more strongly related to marginal effect of qualifications (experience)

    • not explained by school poverty level


Study limitations
Study Limitations High-Poverty and Lower-Poverty Elementary Schools

  • Issues with value-added measures

    • separating current teacher contributions from other current contributions

      • E.g., current family circumstances

        - dynamic sorting

      • sorting on time variant characteristics

    • Instability of measures

      • E.g., measurement error, motivation


ad