Impact of the California High School Exit Exam (CAHSEE): A Snapshot of Effects Wade Buckland, Sheila Schultz, and Becky Dvorak June 21, 2010 Presentation to The Council of Chief State School Officers National Council on Student Assessment
Background • CAHSEE Background • Exit exam assesses English language arts (ELA) and mathematics • HumRRO has been the independent evaluator since 1999 • A requirement to graduate since 2006 • Purpose of instruction study • Assess changes in curriculum and instruction associated with CAHSEE requirement • Identify effective intervention/remedial programs and practices that help struggling students pass CAHSEE requirements • Online surveys administered end of 2008-09 school year • Principals • ELA and mathematics department heads • ELA and mathematics teachers • Teachers of English learners (EL) • Teachers of students with disabilities (SWD)
Survey Sample Survey Sample by Position • Targeted 400 schools • At least one faculty member responded from 271 different schools
Presentation Focus • Teacher Quality: How do teacher experience and teaching credentials relate to student performance on the CAHSEE? • Standards Integration: How have California’s content standards been integrated? • Student Remediation: Have students had access to remediation? If so, what impact has the remediation had on student performance? • Impact on Instruction: How has the CAHSEE requirement influenced the California school system?
Teacher Quality: Years Teaching Teachers with Five or More Years of Experience • Teachers were relatively experienced. - The majority of Department heads reported that most of their teachers had at least 5 years of experience for ELA and Math -More common that most teachers teaching primary supplemental courses had at least 5 years experience than teachers of intervention programs
Teacher Quality: Student CAHSEE Performance • ELA: Years of teaching experience was positively related to student CAHSEE performance for primary and intervention courses. • Math: No significant relationships. • Further analyses found no significant relationship for ELA or math between teaching experience and gain in CAHSEE score from Grade 10 to Grade 11 (p>.05), regardless of course type. Relationship of Teaching Experience to CAHSEE Performance of 2009 10th Grade Students
Teacher Quality: Credentials Full Teaching Credential Prevalence • ELA classes tend to have more teachers working with an appropriate credential than math classes. • Intervention programs are slightly more likely to be staffed by teachers not holding an appropriate credential than a typical ELA or math class.
Teacher Quality: Student CAHSEE Performance Relationship of Teaching Credential to CAHSEE Performance of 2009 10th Grade Students • ELA: Having appropriate teaching credential was positively related to student CAHSEE performance for primary and intervention courses. • Math: Having appropriate credential was positively related to performance for primary courses; no significant relationship for intervention courses. • Further analysis found no significant relationship between teaching credential and gain in CAHSEE score from Grade 10 to Grade 11.
Standards Integration: CAHSEE Blueprints Principals Reporting Completion of School’s Integration of California Content Standards Contained in Blueprints Adopted for CAHSEE by Year • In 2005 and 2009, reported integration was mostly complete or complete. • In 2009, a slightly higher percent reported integration was complete compared to 2005.
Student Remediation: Courses Offered • Almost half of responding principals indicated their schools offer remediation courses targeted to juniors and seniors who have struggled to pass the CAHSEE requirements. • Approximately 11% of responding principals indicated their schools do not offer remediation courses. CAHSEE Intervention/Remediation Courses Offered, According to Principals
Student Remediation: Impact on CAHSEE Scores Effectiveness of District in Improving CAHSEE Scores for At-risk Students, According to Principals • About two-thirds of responding principals indicated their schools’ remediation efforts have been at least minimally effective in improving students’ CAHSEE scores.
Student Remediation: Impact on Drop-out Impact of CAHSEE Intervention/Remediation Courses on Preventing Students from Dropping out of School, According to Principals • Of responding principals whose schools offer remediation, almost 90% said those courses have had a positive impact on preventing student drop-out.
Impact on Instruction: Student Preparation Extent Increased Focus of Using CAHSEE-like Tasks, Matching Content of Instruction and Content of CAHSEE, and Covering Test-Taking Skills to Prepare Students for CAHSEE • While increased focus on using CAHSEE-like tasks, matching instruction to CAHSEE content, and covering test-taking skills to prepare students was fairly consistent, ELA teachers tended to increase their focus slightly more than math teachers, especially in matching instruction to CAHSEE content.
Impact on Instruction: Instigate Change Teacher’s Use of Assessment Results • CAHSEE assessment results were used at a high frequency to provide feedback to students, modify lesson plans, and practice differential instruction. • Less than half of respondents indicated that CAHSEE assessment results were used to assign students to supplemental/remedial classes and provide feedback to districts.
Impact on Instruction: Instigate Change Ways in Which ELA and Math Teachers Collaborated With Other Teachers • Most teachers reported they collaborate with other teachers in some way, especially by sharing ideas. • At least half of responding teachers reported collaborating by aligning instruction across courses, assessing individual student needs, and planning coverage of CAHSEE standards.
Impact on Instruction: Instigate Change Extent to Which CAHSEE Is Used to Change School’s Instruction, Assessment, and Overall Improvement • Most teachers reported CAHSEE is used to a slight or moderate extent to change their school’s instruction, assessment, and overall improvement. • CAHSEE was used slightly more to make overall improvements to the school than to change instruction or assessments.
Summary • Teaching experience was sometimes related to student CAHSEE performance • Years of experience for ELA teachers significantly related to ELA student CAHSEE performance • CAHSEE content standards have commonly been integrated into course instruction • Remediation/Intervention programs commonly present in schools • Although most believe these programs at least somewhat improve performance and decrease drop out rates, there is room for improvement • CAHSEE found to impact classroom instruction • Teachers reported increasing their focus on using CAHSEE-like tasks and matching the content of instruction with that of the CAHSEE
Further Reference • Full Reports: http://www.cde.ca.gov/ta/tg/hs/evaluations.asp • Specific questions: firstname.lastname@example.org