Oregon Standards Evaluation Project, Contract Amendment Phase: Summary of Preliminary Findings

1. Oregon Standards Evaluation Project, Contract Amendment Phase: Summary of Preliminary Findings Dr. Stanley Rabinowitz WestEd December 6, 2007

2. Contract Amendment Scope of Work Three key tasks: • Analysis of the degree to which Oregon’s existing content standards support the Essential Skills* • Analysis of the degree to which Oregon’s current test item pool (TESA/OAKS), work samples, and writing assessments assess proficiency in the Essential Skills* • Identification of Core Standards and recommendations for addressing gaps in coverage of core concepts* *Content Areas: ELA, math, science, and social sciences

3. Analyses are grounded in and informed by • Findings from WestEd’s evaluation of Oregon’s standards and assessments • Responses to Oregon’s Critical Questions • Review of research literature • Effective practices in other states • WestEd staff expertise in the content areas and in standards and assessment development, curriculum and instruction, measurement, and cognitive development • Input solicited from state stakeholder groups • Department of Education staff • Essential Skills Task Force

4. Task 1:Evidence of Support for Essential Skills Among Existing Standards • Based on draft 3.0 Essential Skills (ES) • Examined CIM standards in ELA, mathematics, science, and social sciences • “Backward mapped” support from CIM through benchmark grades 3, 5, and 8 • Presented preliminary findings to ES Task Force to inform ongoing work related to refinement of Essential Skills

5. Draft 3.0 Essential Skills • Read and comprehend a variety of texts at different levels of difficulty. • Write clearly and accurately for a variety of purposes. • Listen actively, speak clearly, and present publicly. • Think critically. • Apply math to solve problems in a variety of settings. • Use technology to learn, live, and work. • Demonstrate civic and community engagement. • Demonstrate global literacy.

6. Task 1 Findings • What percentage of the CIM content standards in each of the four content areas supports the Essential Skills? *Based on primary relationships (i.e., each standard’s support was recorded for primary link only).

7. Task 1 Findings, continued • Overall, how is this support (N = 243)distributed across the Essential Skills? • n = number of CIM standards, across four content areas, based on primary relationships

8. Task 1 Findings, continued • Within each content area, how do the CIM standards support the Essential Skills?* *Based on primary relationships.

9. Task 1 Findings, continued • Of the CIM standards that support an Essential Skill, what percentage provides strong support? *Based on primary relationships.

10. Task 1 Findings, continued • Of the CIM standards that support an ES, what percentage shows a coherent progression across grades 3, 5, 8, and CIM? *Based on primary relationships.

11. Summary of Task 1 Findings • The distribution of standards’ primary support across Essential Skills varies by content area. • 67% of ELA standards provide primary support for ES 1 and 2. • 94% of math standards provide primary support for ES 5. • 100% of science standards provide primary support for ES 4. • Primary support in the social sciences standards is balanced across ES 4, 7, and 8. • Overall, only 2% of all standards’ primary support is associated with ES 6. • In all four content areas, the majority of CIM standards (87% overall) that support an ES show a coherent progression across grades.

12. Task 2: Degree to Which State Test Items Assess Proficiency in the Essential Skills • Based on draft 3.0 ES and multiple choice items in existing state pool (OAKS) • Analyses of work samples and writing assessments will be added in final report • Content areas and grades included in analyses: • ELA and mathematics / grades 3, 5, 8, CIM • Science and social sciences / grades 5, 8, and CIM

13. Task 2 Findings • Overall, what percentage of items assesses proficiency in an Essential Skill?

14. Task 2 Findings, continued • How are these items (N = 4912) distributed across the eight Essential Skills? • n = number of items, across the four content areas *Writing and listening are not assessed via OAKS multiple choice items.

15. Task 2 Findings, continued • At each grade, of those items that assess an Essential Skill, what percentage of items assesses each ES?

16. Task 2 Findings, continued • In each content area, of those items that assess an Essential Skill, what percentage of items assesses each ES?

17. Task 2 Findings, continued • Of those items that assess an Essential Skill, what percentage demonstrates a strong linkage to that Essential Skill?

18. Next Task • Step 3: Identification of Core Standards • Which existing content standards meet the criteria for “core standards”? • Which core concepts are not embodied in the existing standards (“gap” analyses)? • ELA grades K-8 and CIM • Math CIM only (state has completed K-8) • Science grades 3, 5, 8, and CIM • Social sciences grades 3, 5, 8, and CIM

19. For more information about the standards and assessment evaluation: www.ode.state.or.us/teachlearn/real/standards/Standards_Review.aspx For more information about WestEd: www.wested.org