Using state longitudinal data systems for education policy research the nc experience
1 / 19

Using State Longitudinal Data Systems for Education Policy Research : The NC Experience - PowerPoint PPT Presentation

  • Uploaded on

Using State Longitudinal Data Systems for Education Policy Research : The NC Experience. Helen F. Ladd CALDER and Duke University [email protected] Examples of research in NC by CALDER researchers. Charter schools (Bifulco and Ladd)

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation

PowerPoint Slideshow about ' Using State Longitudinal Data Systems for Education Policy Research : The NC Experience ' - charity-mccarty

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Using state longitudinal data systems for education policy research the nc experience

Using State Longitudinal Data Systems for Education Policy Research : The NC Experience

Helen F. Ladd

CALDER and Duke University

[email protected]

Examples of research in nc by calder researchers
Examples of research in NC by CALDER researchers Research : The NC Experience

  • Charter schools (Bifulco and Ladd)

    • How charter schools affect student achievement, achievement gaps and racial sorting.

  • Achievement gaps (Clotfelter, Ladd and Vigdor)

    • By student cohort grades 3-8; by race.

  • Distribution and movement of teachers (Clotfelter, Ladd Vigdor and Wheeler)

    • Descriptive with focus on high poverty schools

  • Public policies and the distribution of teachers (Clotfelter, Ladd, Vigdor and Glennie)

    • Salaries, alternative salaries and working conditions

    • $1800 bonus program program for teachers of math , and special education in low performing middle and high schools

    • The state’s accountability system

Nc research cont
NC research (cont) Research : The NC Experience

  • Teacher credentials and student achievement (Clotfelter, Ladd, and Vigdor)

    • Cross sectional analysis – 5th graders

      • (JHR, 2006)

    • Longitudinal analysis – grades 3-5.

      • (CALDER web page and shorter version forthcoming in EER)

    • Cross subject analysis – high school.

      • Work in progress

Strengths of nc data
Strengths of NC Data Research : The NC Experience

  • Student test scores

    • Based on tests that are linked to the state’s standard course of study

      • Grades 3-8 End-of-Grade (EOG) tests in math and reading

      • High School End-of-Course (EOC) tests in multiple subjects – e.g. algebra I, English I, biology,

    • Available since the mid 1990s.

    • Test scores can be linked by student over time

Strengths cont
Strengths (cont.) Research : The NC Experience

  • Other student data

    • Standard demographic data

      Race, gender, free lunch status, LEP

    • Education level of the parents (Some concerns)

    • Survey responses (connected to the test)

      • E.g. homework, use of computer, TV watching

    • Student addresses in some districts and in some years

Strengths cont1
Strengths (cont.) Research : The NC Experience

  • Teachers

    • Wide array of credentials

      • Teacher licensure test scores

      • Licensure – regular, lateral entry, other

      • National board certification (NC has high percentage)

      • Graduate degrees and when they got them

      • Undergraduate college

      • Certification by subject

  • Teacher salary data

  • Can follow teachers over time

Limitations of nc data
Limitations of NC data Research : The NC Experience

  • Linking of student to their specific teacher is possible but imperfect.

    • Grades 3-5. About 85 percent match

    • Grades 6-8. Poor match

    • High school- courses with EOC tests

      Good match – 75 percent

  • Incomplete home address data – relevant for choice and charter school studies

  • No link yet with higher education data

  • Basic value added model of student achievement grades 3 5
    Basic “value-added” model Research : The NC Experience of student achievement (grades 3-5)

    A student’s achievement in year t is a function of:

    • her achievement in the previous year

      (accounts for the cumulative nature of achievement)

    • teacher characteristics and credentials (e.g. gender, years of experience, teacher test score)

    • classroom characteristics

      (e.g. class size, profile of students in the class)

    • student characteristics

      (e.g. race, gender, poverty status)

    Achievement models 1 3 achievement levels a it or achievement gains a it a i t 1
    Achievement models 1-3 Research : The NC Experience Achievement levels (Ait) or achievement gains (Ait- Ai,t-1 )

    No fixed effects

    • Levels. Upward biased coefficients because of teacher- student sorting; potential bias from lagged achievement on RHS.

      With school fixed effects

      2. Levels. Better, but problem of sorting within schools remains and potential bias from lagged achievement; direction of bias unclear (see our JHR paper)

      3. Gains. Downward bias from misspecified persistence effect.

    Models 4 and 5 preferred
    Models 4 and 5 (preferred) Research : The NC Experience

    Full use of the longitudinal aspect of the data

    With student fixed effects

    4.Levels (but no lagged achievement). Lower bound estimates of effects of teacher credentials

    5. Gains. Upward bound estimates of effects of teacher credentials

    Interpretation of results
    Interpretation of results Research : The NC Experience

    All test scores are normalized to have a mean of 0 and a standard deviation of 1.

    => coefficients of interest will be small. E.g. 0.05

    Point of reference .

    Compare test score of a typical student whose parent has a high school degree but no college degree to the test score of a student whose parents are college educated.

    Estimated effect size = -0.11

    Question : are the effects of teacher credentials on student achievement large enough to counter this negative effect of relatively low parental education?

    Effects of credentials: Research : The NC Experience Teacher experienceCoefficients from preferred model (model 5)All are statistically significant

    Other teacher characteristics and credentials math only preferred model
    Other teacher characteristics Research : The NC Experience and credentials (math only, preferred model)

    • Teacher test score – linear form 0.015

    • Quality of undergraduate institution

      Competitive 0.010

      (Base = noncompetitive)

    • License

      Other license -0.059

      (Base = regular license)

      (Note: all coefficients are statistically significant)

    Teacher credentials master s degree math preferred model
    Teacher credentials Research : The NC Experience Master’s degree (math: preferred model)

    • Master’s degree -0.007 * unexpected

      Disaggregated specification

      MA before teaching -0.009

      MA 1-5 years into teaching -0.005

      MA 5+ years into teaching -0.010*

      * indicates that the coefficient is statistically significant at the 0.05 level.


    Teacher credentials national board certification math preferred model
    Teacher Research : The NC Experience credentials National Board Certification(math: preferred model

    NBCT-2 0.055

    NBCT-1 0.061

    NBCTcurr 0.046

    NBCTpost 0.041

    Are these effects large or small
    Are these effects large or small? Research : The NC Experience

    • Teachers have bundles of characteristics

      Comparison of teacher with average/strong credentials compared to one with weak credentials

      math + 0.15 to 0.20

      reading +0.08 to 0.12

    • Magnitudes are large relative to the estimated effects of reducing class size by 5 students.

    • Sufficient to offset much of the effect of weak parental education – particularly for math.

    Teacher credentials at the high school level work in progress
    Teacher credentials at the high school level – work in progress

    • Use NC EOC test scores – with students linked to their teachers by subject

    • Same problem of teacher and student sorting as at the elementary level

    • Additional challenge – students self select into courses

    Our approach cross subject model with student fixed effects
    Our approach – cross-subject model with student fixed effects

    • Start with all students in grade 10 in four different cohorts (2000-2003)

    • Focus on the subjects typically taken in 9th or 10th grade (algebra 1, English 1, biology, geometry, and ELP)– but include all test scores in whatever grade the course (and test) was taken

    • Define unit of observation as the student by subject

    • Include student fixed effects – to control for unmeasurable characteristics of students such as their ability

    Preliminary illustrative results teacher credentials matter
    Preliminary illustrative results: effects Teacher credentials matter

    • Teacher experience -- mainly first 2 years

    • Teacher licensure -- negative effect for lateral entry teachers

    • Certification by subject -- Teachers certified in subject > those in related subject> than other certified teachers

    • NBCT – more effective pre-NBCT; even more effective after certification.

    • Teacher test scores. Similar to elementary school.

      And more – to come. Stay tuned and check the CALDER web page later this summer.