Oregon Content Standards Evaluation Project, Contract Amendment Phase: Preliminary Findings

1. Oregon Content Standards Evaluation Project, Contract Amendment Phase: Preliminary Findings Dr. Stanley Rabinowitz WestEd November 6, 2007

2. One of several key projects funded by the Gates Foundation Independent evaluation by WestEd, a non-profit educational research, development, and service agency Background: Overview of Standards Evaluation Project

3. Oregon Content Standards and Assessment System Evaluation WestEd was contracted to • Review and evaluate the content standards (all grades, all academic content areas) • Evaluate alignment among the state assessments and the content standards • Make recommendations for improvement of Oregon’s comprehensive standards and assessment system

4. Contract Amendment Phase • Extended scope of work to assist the ODE in implementing key recommendations that emerged from the comprehensive evaluation • Builds upon current state initiatives • Time frame: work will extend from mid-October to December 31, 2007

5. Contract Amendment Scope of Work • Three key steps: • Analysis of the degree to which Oregon’s existing content standards support the draft Essential Skills • Analysis of the degree to which Oregon’s current test item pool (TESA/OAKS), work samples, and writing assessments assess proficiency in the Essential Skills • Identification of Core Standards and recommendations for addressing gaps in coverage of core concepts

6. Focus for Today: Step 1Support for Essential Skills in Existing Standards • Based on draft 3.0 Essential Skills (ES) • Evidence of degree to which current content standards in ELA, mathematics, science, and social sciences support the ES • Backward mapping of support from CIM through benchmark grades 3, 5, and 8 • Recommendations for improving/refining draft Essential Skills to ensure effective application across content areas

7. Step 1 analyses are grounded in and informed by • Findings from the evaluation of Oregon’s standards and assessments • Responses to Oregon’s Critical Questions • Review of research literature • Trends in practices across other states • WestEd staff expertise in the content areas and in standards and assessment development, curriculum and instruction, measurement, and cognitive development • Input solicited from state stakeholder groups • Department of Education staff • Essential Skills Task Force

8. Draft 3.0 Essential Skills • Read and comprehend a variety of texts at different levels of difficulty. • Write clearly and accurately for a variety of purposes. • Listen actively, speak clearly, and present publicly. • Think critically. • Apply math to solve problems in a variety of settings. • Use technology to learn, live, and work. • Demonstrate civic and community engagement. • Demonstrate global literacy.

9. Protocol for Step 1 • Determine whether each CIM standard supports a draft Essential Skill • Which one (if any) of the eight ES does it primarily support? • What is the strength of that relationship? • Strong- CIM standard supports a fundamental skill or concept as explicitly stated in the Essential Skill • Partial- CIM standard supports the Essential Skill in a superficial way or at a lower complexity level • Backward map from the CIM standard to identify evidence of support for the ES in grades 3, 5, and 8 • Do standards in grades 3, 5, 8, and CIM support the Essential Skill in a logical and coherent sequence or progression?

10. Guidelines Used During Analyses • Analysts focused on the bold category-level statements that defined each Essential Skill. • Where provided, the numbered indicator statements were used to clarify the focus of each Essential Skill. • A CIM standard could support an Essential Skill without specifically addressing one of the indicator statements provided.

11. Preliminary Findings • What percentage of the CIM content standards in four content areas support the Essential Skills? *Based on primary relationships (i.e., each standard’s support was recorded for primary link only).

12. More Preliminary Findings… • Overall, how was this support distributed across Essential Skills? • n = number of CIM standards, across four content areas, based on primary relationships

13. More Preliminary Findings… • Within each content area, how do the CIM standards support the Essential Skills?* *Based on primary relationships.

14. More Preliminary Findings… • Of those CIM standards that support the Essential Skills, what percentage was rated as providing strong support? *Based on primary relationships.

15. More Preliminary Findings… • What percentage of CIM standards that support the ES show a coherent progression across grades 3, 5, 8, and CIM? *Based on primary relationships.

16. Summary of Preliminary Findings • Overall, 98% of existing standards across the four content areas (ELA, math, science, and social sciences) primarily supported one of the draft Essential Skills. • Across content areas, the majority of relationships among supporting standards and the Essential Skills was strong. • ELA and math: >92% support was strong • Science: 89% support was strong • Social sciences: 68% support was strong

17. Summary of Preliminary Findings • The distribution of standards’ primary support across Essential Skills varied by content area. • Overall, less than 10% of standards’ primary support was associated with ES 6, 7, and 8 combined. • Primary support in the ELA standards was balanced across three ES (1, 2, 3). • 100% of math standards provided primary support for ES 5. • 100% of science standards provided primary support for ES 4. • Primary support in the social sciences standards was balanced across three ES (4, 7, and 8). • Across content areas, evidence of coherent progression across grades ranged from 83-91% (85% overall).

18. Next Steps • Analysis of the degree to which Oregon’s current test item pool (TESA/OAKS), work samples, and writing assessments assess proficiency in the Essential Skills • Identification of Core Standards and recommendations for addressing gaps in coverage of core concepts • Formal report with findings and recommendations for improvement/ refinement emerging from the Amendment Phase

19. For more information about the standards and assessment evaluation: www.ode.state.or.us/teachlearn/real/standards/Standards_Review.aspx For more information about WestEd: www.wested.org