Unintended Consequences of an Algebra-for-all Policy on High-skill Students:

March 2010 Unintended Consequences of an Algebra-for-all Policy on High-skill Students: The Effects on Instructional Organization and Students’ Academic Outcomes Takako Nomi Consortium on Chicago School Research at the University of Chicago Funded by the Institute for Education Sciences

Movement for More Rigorous Course Requirements National Context • Recognition that high schools are not sufficiently preparing students for college or workforce • National movements calling for increasing graduation requirements • College-prep curriculum is in place in 20 states, and more states and districts are raising graduation requirements in near future Chicago Context • Chicago increased graduation requirements in 1997 • Requiring college-prep curriculum for all students • Ending remedial coursework • All students required to take Algebra in 9th grade, followed by geometry and algebra II

Prior Research Research suggests that rigorous coursework improves students’ academic outcomes Constrained academic curriculum (private vs. public schools) Tracking High school coursework (e.g., AP courses) tied to college outcomes Limitations in prior research Not examining the effects of a policy mandating college-prep curriculum Selection bias among schools & among students within schools Not examining unintended consequences of such a policy Our recent work addressed some of these limitations (Allensworth, et al. 2010)

Policy effects in Chicago The effects of mandating a college-prep curriculum for the policy target students--average and lower ability students who would take remedial math in the absence of the policy When schools increased algebra enrollment: • More students took algebra and earned credits • More students failed in math • Students were more likely fail in Algebra than in remedial Math • Test scores unchanged Limitations: Unintended consequences on non-target students

Unintended Consequences Non-target students • High-skill students who would take Algebra regardless of the policy • The policy would not affect their course enrollment, but might affect their academic outcomes Instructional Organization: How classrooms are organized for instructional purposes • Student diversities in academic backgrounds • How to organize instruction to meet the need of all students with varying skills • Tracking/ability grouping—Tailored instruction Current policy discussions do not address how universalizing a college-prep curriculum may affect instructional organization • Classroom academic composition affects students’ outcomes (e.g., content difficulty and instructional pace)

Research Questions • To what extent did a policy that required algebra for all students in ninth grade affect classroom academic composition? • Schools might maintain the existing practice • Classroom academic composition remain the same • Schools might reorganize classrooms, incorporating low-skill students who would otherwise take remedial math into Algebra classes • Classrooms become more mixed-ability • Peer ability levels would decline for high-skill students 2) For high-skill students who were not targeted by the policy, how did the policy affect their academic outcomes? • If peer ability levels declined, were their outcomes negatively affected by the policy?

Data The population of first-time 9th grade cohorts in Chicago public high schools from 1994-95 to 1999-2000 (6 cohorts) • Schools in existence pre- and post-policy • Total N = 104,000 in 57 schools • 20,215 high-skill students CCSR Archive • Semester course transcripts • Complete administrative records with demographic information • Elementary and high school achievement test scores

Variables Student Outcomes 9th-grade • Math test scores Covariates Student Level • 8th grade math ability • Demographics (Gender, Age, poverty, social status, Race/ethnicity, Mobility prior to high school) • Special education status • School Level • Academic composition (average ability levels) in the base year • Cohort Level • Year-to-year changes in academic composition

Designs Two comparisons • Short interrupted time series (Abrupt changes in Algebra enrollment) • Changes in outcomes between pre-policy and post-policy cohorts • Problem with interrupted time series—other changes unrelated to the policy may affect outcome changes • Within-cohort comparisons • Comparing outcome changes between schools affected by the policy and schools unaffected by the policy • Only schools that offered remedial math pre-policy were affected by this policy, while all schools were affected by other changes • Some schools offered algebra for all students pre-policy, including very low ability students (control schools)

Assumptions No systematic differences between schools affected and unaffected by the policy in a way that are related to outcome trends • Tested in a number of ways… • After controlling for students’ incoming abilities, measured school characteristics did not explain why some schools had higher pre-policy algebra enrollment • Trends in students’ academic composition were similar between schools affected and unaffected by the policy • Pre-policy outcome trends were similar between schools affected and unaffected by the policy though their outcomes differed in the base year

Statistical Models Policy effect on students outcomes Level 1 (Students) Yijk= π0jk + π1jk(math ability)ijk + π2jk(math ability2)ijk + Ʃπik(X)ijk + eijk Level 2 (Cohorts) π0jk = β0k + Ʃβk(cohort year)jk +β6k(cohort ability)jk + rjk Level 3 (Schools) For intercept (pre-policy average outcomes) , β0k = γ00 + γ01(Affected)k + u0k For each cohort-year slope (Β1k through Β5k ), Β1k = γ10 + γ11(Affected)k+ u1k

Overall Changes in Classroom Academic Composition Unaffected Schools (full Algebra enrollment pre-policy) *Controlling for cohort average abilities

Overall Changes in Classroom Academic Composition Unaffected Schools (full Algebra enrollment pre-policy) Affected Schools (increased Algebra enrollment post-policy) *Controlling for cohort average abilities

Policy Effects on Classroom Academic Composition among high ability students *Controlling for cohort average abilities and student characteristics

Policy Effects on Test scores for High-ability Students

Trends in math scores Policy

Summary Curricular structure shapes how classrooms are organized • Schools that offered remedial math pre-policy used greater levels of sorting & more homogeneous grouping (tracking) • High-skill students attend classes with other high-skill students, and vise versa for low-skill students • In schools that did not offer remedial math, students attend more mixed-ability classrooms Algebra-for-all policy eliminated curriculum differentiation, but it also created more mixed ability classrooms • For high-skill students, even though the policy did not affect their course enrollment, it negatively affected their academic outcomes due to declines in peer ability levels • Instruction targeted at the middle students in the class

Implication/future research Policy Implication • Curricular policies need to consider multiple ways that affect students • Simply raising graduation requirements is not enough • How can we raise standards/expand opportunities without comprising excellence ? • Chicago implemented double-period Algebra in ‘03 (Nomi & Allensworth, 2009) • This intensified tracking, but provided supports for struggling students • This approach was successful in raising test scores for all students (but not for Algebra grades) Future research • The Algebra-for-all policy might have benefited a subgroup of students • Low-ability students who would take remedial math vs. those who would take algebra in the absence of the policy

Pre-Policy Early Post-Policy Mid Post-Policy Late Post-Policy 1998 1994 1997 2000 2003 1999 1995 2001 2004 1996 2002 COHORTS Algebra enrollment increased immediately with the policy Changes occurred immediately following policy High-skill students took algebra regardless of the policy The policy did not affect their course enrollment

Pre-Policy Algebra Enrollment Not Determined by School Composition Schools’ pre-policy algebra enrollment by school mean math ability for low-ability students Each dot (+) represents a school Among both schools with high- or low-average abilities, some had almost 100% algebra enrollment pre-policy. These schools were not affected by the policy. Percent of low-ability students enrolled in Algebra High degree of variation in algebra enrollment pre-policy among schools with the same average ability School average math ability

Unintended Consequences of an Algebra-for-all Policy on High-skill Students: