Bridget Terry Long Angela Boatman Harvard Graduate School of Education IES Conference June 2010

1. Does Remediation Help All Students?How the Effects of Postsecondary Developmental Courses Vary by Background and Ability Bridget Terry Long Angela Boatman Harvard Graduate School of Education IES Conference June 2010

2. Remedial and Developmental Postsecondary Courses Florida Remediation Studies  Lack of academic prep is a significant barrier to success –  Nationally approx.40% of 1st year students are placed into college remediation (55-60% at CCs) How should this problem be addressed? • Developmental courses at the community colleges • All schools use the same placement instrument and have the same cutoff for placement • Florida Administrative Dataset: Sample: All first-time, degree-seeking CC students who began in Fall 1997, 1998, 1999, or 2000 (over 100,000 records) Juan Carlos Calcagno and B.T. Long – "The Impact of Postsecondary Remediation Using a Regression Discontinuity Approach: Addressing Endogenous Sorting & Noncompliance“ and “Does Remediation Help All Students? The Heterogeneous Effects of Postsecondary Developmental Courses"

3. Beth and Becky both take the College Placement Test (CPT) Beth scores just above the cut-off score Becky scores just below the cut-off score Beth to college-level courses Becky to remediation Compare the outcomes of Beth & Becky Regression Discontinuity: The Intuition Beth and Becky are observationally similar Crossover: Beth takes remediation anyway No Show: Becky never enrolls in remediation Endogenous Sorting: Beth retests to place out of remediation

4. Passing the First College-Level Course by Reading CPT Score at the Discontinuity Placed in Developmental Reading Placed in College Level Negative Effects on Passing First College-Level Course

5. Total Credits Earned by Reading CPT Score at the Discontinuity Placed in Developmental Reading Placed in College Level Positive Effects on Total Credits Earned

6. Florida Remediation Studies • Being assigned to a developmental course appears to increase total credits completed for students on the margin of passing out of the requirement • Does not increase the completion of college-level credits or eventual degree completion.  Remediation might promote early persistence but does not necessarily help students on the margin of passing the cutoff to make long-term progress • What about students of different backgrounds?The effects were: • More positive for female (compared to male), older (compared to younger), and LEP (compared to non-LEP) • More negative for low-income student (compared to non-Pell-eligible)

7. Does remediation work for students with far less preparation? • Past studies (FL, OH, TX) focus on students on the margin of needing remediation -- they do not investigate the effects of remediation on students who are extremely under-prepared (i.e. don’t have an appropriate control group) • The Tennessee Case • Multiple cutoffs and changes in placement policy over time  Investigate effects of different levels of remediation using multiple RDs • Sample: Fulltime students under 21 who began at a TN public college or university in the fall of 2000 “Does Remediation Work for All Students? How Remedial and Developmental Courses Affect Students With Different Levels of Academic Preparation” – A. Boatman and B.T. Long

8. Remediation in Tennessee Go to College-level courses ACT/SAT score determines which placement exam (if any) Take Developmental Algebra II Take COMPASS Algebra exam Take Developmental Algebra I Take COMPASS Arithmetic exam Take Remedial Arithmetic College Level Devel. Alg II Devel. Alg I Remed. Arith. COMPASS Algebra II test Score 50-100 Score 28-49 Score 0-27 COMPASS Arithmetic test Score 30-100 Score 0-29

9. RD #1:Developmental Algebra IIvs.College-level College Level Devel. Alg II Devel. Alg I Remed. Arith. COMPASS Algebra II test Score 50-100 Score 28-49 Score 0-27 Score 30-100 Score 0-29 COMPASS Arithmetic test

10. RD #1: Effects of Recommendation to Develop. Algebra II–vs–College-Level Math Credits Accumulated Over Three Years Not much Difference in Total Credits Diff in College Credits

11. RD #1: Effects of Recommendation toDevelop. Algebra II–vs–College-Level Math Negative Effects in Cumulative College Credits

12. RD #3:Remedial Arithmetic vs.Develop Algebra I College Level Devel. Alg II Devel. Alg I Remed. Arith. COMPASS Algebra II test Score 50-100 Score 28-49 Score 0-27 COMPASS Arithmetic test Score 30-99 Score 0-29

13. Math Developmental/Remedial Education Total College Credits Completed after 3 years Smaller in Magnitude

14. College-Level vs. Developmental vs. Remedial Writing College Composition Developmental Writing Remedial Writing COMPASS English test ACT English Score Score 68-100 Score 28-67 Score 0-27 RD #2: Develop. vs. Remedial Writing

15. Grade in first college-level course Not causal analysis, but interesting estimates…

16. Summary of Tennessee Results • Generally negative effects if placed in a lower math class, but the size of the effect is smaller farther down the test score distribution • Similar effects found for Developmental and Remedial Reading • In Writing, the effects are negative at the college/developmental threshold but positive at the developmental/remedial threshold • Positive effects on first college-level grade (though these results are not causal)

17. Implications and Remaining Questions • Remedial and Developmental courses do not affect all types of students equally (by background, ability level, or college level) • What is the best way to offer remediation? Characteristics of strong remedial programs? • What are the effects of limitations states impose on remedial course-taking (e.g., only at CCs, time limits)? • Early Placement Testing – a preventative measure?

18. Contact Information Prof. Bridget Terry Long, Ph.D. Harvard Graduate School of Education, NCPR, and NBER longbr@gse.harvard.edu Angela Boatman, Doctoral Candidate Harvard Graduate School of Education angela_boatman@mail.harvard.edu