Independent Study of State End-of-Course Assessments Presentation of Findings

1. Independent Study of State End-of-Course Assessments Presentation of Findings House Education Committee February 1, 2008

2. Introduction • The Washington State Board of Education (SBE) contracted with Education First Consulting to conduct an independent study of state end-of-course assessments. • Our research team includes: • Jennifer Vranek, President of Education First Consulting • Jessica de Barros, Education Policy Consultant • Richard Brown, Psychometrician and Assistant Professor in the USC Rossier School of Education • Betheny Gross, Senior Research Analyst in the University of Washington’s Center on Reinventing Public Education • Christopher Mazzeo, Education Policy Consultant, Consortium for Chicago School Research

3. Our Approach Our team researched the following: • What lessons can Washington State learn from the literature on end of course tests (EOCs), comprehensive assessments and exit exams? • What have been the experiences of other states implementing EOCs? • What alternatives measure Washington’s standards? • What are the policy implications for Washington’s high school assessment system based on lessons learned from other states with EOCs?

4. Methodology • First, the team conducted a thorough review of the primary and secondary literature on EOCs and comprehensive tests, as well as the use of these formats for exit exams and school accountability. • Next, we researched and summarized nine states’ EOC programs. • CA, IN, MD, NJ, NY, SC, TN, TX and VA • We simultaneously conducted 30 interviews of education, government and business leaders in seven states. • CA, IN, NJ, SC, TN, TX and VA • Finally, we used our research findings to consider policy implications for Washington State’s high school assessment system.

5. What are EOCs? What is the WASL? End-of-course assessments are tests that: • Are given at the end of a particular course and measure the content found only in the standards for that course. • Are taken only by students taking a particular course, so not all students are assessed. The 10th grade Washington Assessment of Student Learning in reading, writing, mathematics and science is a comprehensive assessment because: • It is given at a common point in time (to all 10th graders) and • It measures content and skills from standards that are taught over several years, rather than in a particular course.

6. Purposes Of Assessment There are many ways to describe the purposes of statewide high school assessment. We summarize these as four main purposes: • Supporting student learning; • Holding students and/or schools accountable; • Determining readiness for postsecondary education and training; and, • Ensuring high-quality and efficient operations.

7. Assessments:Means to an End • On balance, both comprehensive and EOC assessments can meet these four purposes equally well. • However, each approach has different strengths and meets the purposes in different ways. • The advantages and disadvantages of either test format dependonthe state’s purposes for assessment.

8. Comparing Use Of Assessments • Comprehensive assessments are more common today than EOCs… • But more states are shifting to EOCs. • 16 states currently use EOCs • 11 additional states are planning to use EOCs

9. States with EOCs (In Place or Planned)

10. States with EOCs as Exit Exams or for School Accountability(In Place or Planned)

11. Key Findings:Comprehensive Assessments Overall, we found that comprehensive testing systems: • Usually focus on 10th grade or lower standards; • Assess a sliceof high school standards, rather than deep knowledge of particular subjects; • Can potentially narrow the delivered curriculum to what is tested; • Provide a “snapshot” of system performance; • Often take up less testing time overall and cost less; • Take a more straightforward approach to exit exams and school accountability; and • Rarely provide information on students’ readiness for postsecondary education coursework and training.

12. Key Findings:EOC Assessments Overall, we found that EOC testing systems: • Vary widely with respect to number and kinds of courses assessed; • Will measure a broader and deeper range of standards, but only if there are a sufficient number of EOCs in each subject; • Do not assess all students against common standards, unless states require all students to take certain courses and/or require all students to take certain EOCs; • Can promote more consistency of teaching and provide more timely information on learning and course quality; • Motivate students to learn through exit exams as well as other forms of lesser student stakes (e.g. counting results in course grades; • Make it more complicated to hold schools accountable, yet offer the potential to produce more validity and reliability; and • Can be better-suited for placing students in postsecondary education courses than comprehensive tests given by states in the 10th grade.

13. Policy Implications for Washington • Our research revealed that both comprehensive and EOC assessments have strengths and limitations, depending on the state’s purpose of assessment. • We believe Washington policymakers must first determine what their policy priorities are for student assessment in order to choose the most appropriate testing format.

14. A Decision Framework For Policy Priorities • To help policymakers compare how each test format meets various policy priorities, we developed a “decision framework” analyzing each type of assessment according to the four purposes of assessment: • Supporting Student Learning; • Holding Students and Schools Accountable; • Determining readiness for postsecondary education and training; and • Ensuring High-Quality and Efficient Operations. • Within each issue area, we analyzed the relative advantages and disadvantages of comprehensive assessments and EOCs according to multiple criteria.

15. Supporting Student Learning

16. Holding Students & Schools Accountable

17. Determining Readinessfor Postsecondary Education

18. Ensuring Quality & Efficient Operations

19. Sequencing Policy Decisions • Assessments are a means to the end of understanding how well students, schools and districts are meeting state standards. • It is important to consider the assessment policy within the larger context of the state’s K-12 education system and efforts to improve learning and teaching. • Over the next several months, the statewide system of standards, curriculum, instruction, assessment and accountability is being reshaped. • State policymakers should firstaddress key questions about high school education policy and then determine which format for the high school assessments is most relevant.

20. RecommendedPolicy Questions • What skills and knowledge do students need to be successful after exiting Washington’s K-12 public education system? How important is measuring readiness for postsecondary education? • Is it important for the state to assess academic standards close to the time when students learn content or at a common point in time? • When the new graduation requirements are adopted, what course credits will all students be expected to earn in English, math, science and social studies? • Should the main high school assessments also be used to measure readiness for postsecondary education? • Does the state want additional ways to hold students accountable, such as tying test results into course grades? • Are more statewide measures needed for schools or are the four current measures sufficient for the state accountability system?

21. EOC-SpecificRecommendations If, after addressing these questions, Washington policymakers consider transitioning to an EOC-based system, we recommend that policymakers: • Minimize cost and development time by working in collaboration with other states to implement standards-based, criterion-referenced assessments; • Require all students to earn a common set of course credits, and require all students to take the corresponding EOCs in these subjects, to ensure equity. • Maintain the 10th grade WASL in reading and writing, rather than creating EOCs in these subjects.

22. Discussion

23. Back-Up Slides

24. Detailed Findings • Well-designed comprehensive test systems and EOC systems share many characteristics of high-quality testing systems, such as • Measuring state standards with reliability and validity. • Assessing standards with a variety of formats (multiple-choice, constructed-response). • Turning around test results in a timely manner. • Scoring costs may increase with EOCs, but this depends on number of tests/test forms, types of item formats and scoring turnaround needs.

25. Detailed Findings • State EOC systems vary widely on the number and kinds of courses that are assessed. • EOC systems can assess a broader and deeper range of content and skills—if there are multiple assessments in a subject area, assessments for advanced-level courses and/or multiple subject areas. • Comprehensive tests typically focus on 10th grade standards and assess all students at a common point in time. • Most EOC systems still measure English with a comprehensive test. • Unless states require all students to pass a certain series of courses and/or require all students to take certain EOCs—different groups of students will be tested on different content. • E.g.: 700,000 California students took the Algebra I EOC (a course required for graduation) in 2006, but only 300,000 took the Geometry EOC.

26. Detailed Findings • While both types measure standards, EOCs are typically chosen by states to promote more consistency of teaching. • EOC exams are designed to measure the impact of a specific course on student learning. • Comprehensive exams are designed to measure the impact of instruction over several years on student learning. • CA, IN, TN and VA education leaders told us they administer EOCs because the standards, content, instruction, curriculum and assessment are more closely aligned than in comprehensive systems.

27. Detailed Findings 4. Changing test formats will not necessarily improve student learning of state standards or increase student performance. • Performance on EOCs and comprehensive tests varies widely. • Student results on the high-stakes California and South Carolina 10th grade exit exams are much higher than on those states’ EOCs. • It not possible to conclude that test format type has any influence on the number or proportion of students meeting standards. Factors that impact test scores include: • Content measured; • Rigor and complexity of test questions; • Performance standards (“cut scores”); • Alignment of standards and local curriculum with the assessment; and • Students’ opportunity to learn the material tested. • Most exit exam states—regardless of test format—have high levels of student performance overall (and many have closed performance gaps), especially when results from retakes are added.

28. Detailed Findings • Both EOCs and comprehensive assessments can be used as exit exams. In addition, EOCs can be used as part of student course grades. • A handful of states—including CA, IN, LA and UT—currently attach no stakes for students to their EOCs. • Several states—including GA, NC, SC and soon TN and TX—require districts to count EOC results as a portion of final course grades (usually 15-25%). • 13 states currently or will require students to pass at least one EOC in order to graduate. 10 of these states use only EOCs, while MA, NJ and SC use comprehensive and EOCs. This includes IN and TX, which plan to transition away entirely from their comprehensive tests.

29. Detailed Findings • It can be more complicatedto hold schools accountable with EOC tests. • It is more straightforward to use comprehensive tests for school accountability. • EOCs can paint a more complex—and more reliable—picture of high school performance. More EOC tests will yield more data and have the potential for greater validity and reliability. • States can use EOCs to meet No Child Left Behind requirements in reading, math and science. • 10 states currently use EOCs to determine AYP. • Two additional states are planning to use EOCs for AYP.

30. Detailed Findings • EOCs are better-suited to determine student readiness for postsecondary education and training than comprehensive tests given in 10th grade. • Most states planning to use high school tests for readiness and placement purposes are moving towards EOCs, in large part because EOCs given in advanced subjects can be used more easily to assess readiness than comprehensive tests given in the 10th grade. • E.g. CA, LA, NJ, NY, TX and TN do or plan to use EOCs to signal readiness for postsecondary education. • States also are adapting the SAT, ACT, Compass/Accuplacer to administer to all 11th-graders.