Early Findings from the Implementation and Impact Study of Early College High School

1. Early Findings from the Implementation and Impact Study of Early College High School Larry Bernstein - RTIRyoko Yamaguchi and Fatih Unlu - Abt AssociatesJulie Edmunds - SERVE Elizabeth Glennie - RTI John Willse, UNC - Greensboro and Nina Arshavsky -SERVE SREE Conference 2010 March 4, 2010

2. Background/Context • Too many students not graduating from high school • Of those graduating, many not prepared for postsecondary education and world of work • Of every 100 students entering 9th grade in public HS in North Carolina, only 70 graduate within 5 years • Only 42 of them enroll in college • Only 19 of them complete a two-/four-year degree within 6 years of graduating from HS • NC response: extensive public-private effort to redesign HSs to make them more effective for all students

3. Model • National Early College High School (ECHS) Initiative – Gates and other funders • Goal to increase number of students graduating from HS prepared for college/work • Targeted at traditionally underrepresented populations of students • Study focuses on North Carolina’s effort, supported by North Carolina New Schools Project • NC has over 60 schools across the state • In NC, ECHSs are small schools, located on college campuses, grades 9-12 or 9-13

4. ECHS Design Principles Intermediate Outcomes Long Term Outcomes College Ready Articulated program of study, grades 9-12 or 13 leading to Associate’s degree College readiness activities Increased student attendance Powerful Teaching and Learning High-quality, rigorous, and relevant instruction Student collaboration and discussion Formative and multiple assessments Common standards Increased frequency of higher level courses Increased graduation rates Improved attitudes toward self and school Increased enrollment in college Personalization Academic and affective supports Supportive relationships Improved behavior Increased graduation from college Professionalism Ongoing professional development Collaboration among staff Collective responsibility and decision-making Increased aspirations toward college Improved student achievement Purposeful Design: Autonomous governance Located on college campus Small size Flexible use of time Integration with college

5. Study Overview • Partnership between SERVE, NCDPI, NC New Schools Project, Duke University, UNCG, Abt Associates, RTI International • Funded through federal IES grant • Four-year experimental study comparing students who applied to and were randomly accepted into ECHS with students who applied for and were randomly not accepted (attended regular HS) • Participating schools use lottery to select students out of eligible applicant pool; many lotteries stratified to place priority on target population

6. Study Goals Need to determine if and how ECHS is effective: • Examine impact of model on student outcomes including: attitudes, attendance, achievement, course-taking, and school leaving/dropout rates • Determine whether impact varies by student characteristics, such as: gender; race/ethnicity; poverty status; first generation college status; and prior achievement • Determine whether specific program components are associated with better student outcomes

7. Overview of Methodology • Experimental design • Schools determine eligible population • Lottery used to allocate spots • Lottery may be stratified if needed; analyses incorporate weights to reflect differential probabilities of selection • Outcome data collected on both treatment and control students • Implementation data collected primarily on enrolled students • Some information on control group students’ experiences also collected

8. Lottery Sample Total: 34 cohorts in 20 sites 3,244 students

9. Data Collected on both ECHS (Treatment) and Traditional (Control) Extant Data from School Records Original Student Opinion Survey Attitudes and engagement Behavior School experiences: Rigorous and relevant instruction; assessment Affective and academic support Relationships Expectations • Test scores • Course-taking • Attendance • Dropouts • Discipline

10. Data Collected only on ECHS Staff and Students • Implementation Surveys—All Staff and Students • Students—same content as Student Opinion Survey • Staff—questions on implementation of the design principles • Completed annually • Site visits • Interviews with staff and students • Observations • Interviews with college faculty • Occurs once

11. Samples School extant data: 718 9th grade students in 8 cohorts in 6 sites Student Opinion Survey: 575 9th Graders in 10 sites Implementation data: Surveys from 937 students and 95 staff members in 11 schools Site visits to 9 schools; 11 more planned

12. Background Characteristics of Current Ninth Grade Sample Sample: 718 students in 8 cohorts in 6 schools * significant at p ≤ .05; will need to account for differences in analyses.

13. Ninth Grade Results Impacts on: • College preparatory course-taking and progression • Students’ behavior and attitudes • Students’ high school experiences

14. Why focus on course-taking? • Big difference between ECHS and Traditional: expectation that everyone goes to college • Students who don’t take Algebra I by end of 9th grade will find it much harder to complete college-preparatory course of study • Higher level math courses not required for graduation but test scores included in school accountability → traditional schools may steer students away from those courses • Therefore math is particularly sensitive indicator • Look at course-taking patterns in other subjects as well

15. Course-taking Findings • By end of 9th grade, significantly more ECHS students taking and successfully progressing through college preparatory courses • ECHS overall reducing performance gaps due to SES characteristics → leveling of playing field

16. Impact on Algebra I *Significant at p≤.05

17. Impact on Geometry

18. Impact on Algebra II *Significant at p≤.05

19. Impact on College Prep Mathematics Course-taking *Significant at p≤.05

20. Impact on Civics and Economics *Significant at p≤.05

21. Impact on English I

22. Minimizing Gaps in Course Progression Algebra I Progression Rates for Minority and Non-Minority Students

23. Gaps in Course Progression by Group Min. vs. Non-Min. 1st. Gen. vs. Non-1st Gen. FRL vs. Non-FRL Alg. I. Eng. I Alg. I Eng. I Alg. I Eng. I

24. Impact on Students’ Behavior, Attitudes and Experiences • ECHS students missed significantly fewer days of school (unexcused) • ECHS students were less likely to be suspended • ECHS students reported significantly higher levels of academic engagement and greater self-efficacy in math • Overall, ECHS students reported significantly more positive experiences than control students

25. Impact on Student Attendance *Significant at p≤.05

26. Impact on Student Behavior *Significant at p≤.05

27. Impact on Student Attitudes • Any overall impacts in favor of ECHS • Impacts vary by school • ECHS students report higher levels of academic engagement than students in control group (effect size = .3) • Positive impact on self-efficacy in math (effect size = .2) • No impact on other outcomes: • Self-efficacy in English • Persistence (e.g. continuing to do work, etc.)

28. Looking at Implementation • Collecting data on Design Principles: • College Ready • Powerful Teaching and Learning • Personalization • Professionalism • Purposeful Design • Data from Student Opinion Survey • Data from Implementation Survey and site visits

29. Implementation Findings • Overall, ECHS students experience more positive school environments than students in control group • Level of implementation overall high but varies by school • No apparent pattern in level of implementation by design principles

30. Students’ Experiences Source: Student Opinion Survey, administered to treatment and control

31. Conclusions • North Carolina's ECHS model is creating a more positive school environment • Students in this environment have fewer absences and fewer suspensions • ECHS students more likely to be on-track for college (defined as taking and succeeding in the required courses) • Results suggest that ECHS model also most effective for target populations of students traditionally underrepresented in college  

32. For More Information Larry Bernstein lbernstein@rti.org 781-434-1725