1 / 24

MSRP Year 1 (Preliminary) Impact

MSRP Year 1 (Preliminary) Impact. Research for Better Schools RMC Corporation. Study Design READ 180: Evaluate student outcomes using an experimental design based on randomly assigning eligible students to treatment and control conditions within participating schools

beverly-leblanc
Download Presentation

MSRP Year 1 (Preliminary) Impact

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. MSRP Year 1 (Preliminary) Impact Research for Better Schools RMC Corporation

  2. Study Design READ 180: Evaluate student outcomes using an experimental design based on randomly assigning eligible students to treatment and control conditions within participating schools Student outcomes include reading achievement (ITBS) and state assessment (TCAP) results in core content areas Study Design MCLA: Evaluate teacher and student outcomes using an experimental design based on randomly assigning schools within matched pairs to treatment and control conditions Teacher outcomes include preparedness and frequency of literacy strategy use Analytic Approach READ 180: Cross-sectional ITT analyses of reading and core content area achievement Two-level models using spring ITBS and TCAP scores as a function of student and school variables Analytic Approach MCLA: Two-level models using spring indices as a function of teacher and school variables Two-level models using spring ITBS and TCAP scores as a function of student and school variables Overview: Study Design and Analytic Approach

  3. MCLA: Random Assignment of Schools

  4. Demographic Characteristics of Year 1 MCLA Student Sample

  5. Baseline Comparisons of Students in MCLA Treatment and Control Schools

  6. READ 180: Enrolled Students

  7. Statistical Model Used to Estimate READ 180 Impacts At the student level,

  8. Statistical Model Used to Estimate READ 180 Impacts

  9. Variables Included in READ 180 Impact Analytic Models (Year One):Dependent and Independent

  10. Variables Included in READ 180 Impact Analytic Models (Year One): Covariates

  11. Analytic Decisions • Missing Data • students missing pretest score(s) deleted from impact analysis on relevant measure(s) • Covariates • include all student- and school-level covariates in the model • run the model • eliminate the school covariate with the lowest significance level (highest p-value) not less than 0.2 • repeat steps 2 and 3 until the remaining covariates had p-values less than 0.2 • repeat steps 2-4 for the student covariates

  12. READ 180 Impacts on Students (Year One)

  13. Selected Characteristics of the Year 1 Teacher Sample for MCLA Impact Analyses

  14. Statistical Model Used to Estimate MCLA Teacher Impacts

  15. Statistical Model Used to Estimate MCLA Teacher Impacts

  16. All Variables Included in MCLA Impact Analytical Models for Year 1

  17. Analytic Decisions • Missing Data • teachers missing pretest score deleted from impact analysis on measure • Covariates • include all teacher and school-level covariates in the model • run the model • eliminate the school covariate with the lowest significance level (highest p-value) not less than 0.2 • repeat steps 2 and 3 until the remaining covariates had p-values less than 0.2 • repeat steps 2-4 for the teacher covariates

  18. Comparison of Teachers in MCLA Treatment and Control Schools on Year-End Indices for Preparedness and Frequency of Use

  19. Regression Models Employed to Test Year 1 MCLA Impact on Student Achievement

  20. Regression Models Employed to Test Year 1 MCLA Impact on Student Achievement

  21. Analytic Decisions • Missing Data • students missing pretest score(s) deleted from impact analysis on relevant measure(s) • Covariates • include all student- and school-level covariates in the model • run the model • eliminate the school covariate with the lowest significance level (highest p-value) not less than 0.2 • repeat steps 2 and 3 until the remaining covariates had p-values less than 0.2 • repeat steps 2-4 for the student covariates

  22. MCLA Impacts on Students (Year One)

  23. (Brief) Conclusions & Discussion READ 180: No significant Year One student impact • Late startup • (Most) students will receive two years of intervention Planned Future Analyses: • Three-level analyses planned to examine whether teacher characteristics exert a moderating effect on student outcomes • Exploratory analyses of relationships between amount of READ 180 instruction and effects on student outcomes

  24. (Brief) Conclusions & Discussion MCLA: • Significant (moderate) impact on teachers’ frequency and preparedness to use MCLA strategies • No significant impact on students’ achievement in reading or core content areas Discuss: • Subjectivity of measure (“Hawthorne Effect”) • Teacher findings support program logic model • Explore relationship between impact and participation in PD

More Related