1 / 30

Oregon Content Standards and Assessment System Evaluation

Oregon Content Standards and Assessment System Evaluation. Prepared for the Oregon Department of Education by WestEd Dr. Stanley Rabinowitz Dr. Edynn Sato June 22, 2007. One of several key projects funded by the Gates Foundation

king
Download Presentation

Oregon Content Standards and Assessment System Evaluation

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Oregon Content Standards and Assessment System Evaluation Prepared for the Oregon Department of Education by WestEd Dr. Stanley Rabinowitz Dr. Edynn Sato June 22, 2007

  2. One of several key projects funded by the Gates Foundation Independent evaluation by WestEd, a non-profit educational research, development, and service agency Background: Overview

  3. Content Standards and Assessment System Evaluation: What? WestEd will: • Review the content standards (all grades, academic content areas) • Evaluate the structure of the content standards • Evaluate alignment between the state assessments and the content standards • Make recommendations for improvement of structures and systems

  4. Key Deliverables • Preliminary report of an initial review of selected content standards and grades using initial protocol/criteria • Final report for the comprehensive review of the content standards • Final report for the alignment of assessments to content standards • Final report on the structure and quality of Oregon’s content standards and assessments

  5. Comprehensive Evaluation of Content Standards • English language arts • Mathematics • Science • Social sciences • Arts • Second languages • Physical education • Health education • Technology • English language proficiency

  6. Standards Evaluation Studies The analyses aimed to address the following key questions: • Do Oregon’s content standards adequately represent the knowledge and skills that all students should know and be able to do? • Do Oregon’s content standards reflect the appropriate breadth and depth of the content area? • Do Oregon’s content standards have the clarity and consistency needed to adequately guide instruction and assessment?

  7. External Referents

  8. External Referents (continued)

  9. External Referents (continued)

  10. External Referents (continued)

  11. Standards Evaluation Studies (continued) Overall Quality • Oregon’s English Language Arts (ELA) standards overall are of good quality and provide breadth and depth of coverage of Reading, Literature, Writing, Speaking, and Listening. The standards generally are clearly written, focus on important skills/concepts for instruction, and are assessable. • Oregon’s mathematics standards overall are of good quality and reflect a breadth and depth of content coverage. The standards generally are clearly written, focus on important skills/concepts for instruction, and are assessable. • Oregon’s science standards overall are of good quality in terms of clarity and importance for instruction. However, in all grades except for grade 8, analysts determined that the standards did not reflect a range of depth of knowledge. For all grades except for grade 1, analysts determined that the standards did not reflect a breadth of knowledge. With regard to consistency, analysts determined that standards were inconsistent (language, skills, knowledge) in grades 4, 8, and CIM. And, in all grades except for grades 3, 5, 8, and CIM, there were issues of assessability.

  12. Standards Evaluation Studies (continued) Overall Quality (continued) • Oregon’s socialsciences standards overall are of good quality in terms of depth, breadth, consistency, importance for instruction, and assessability. However, overall these standards lack clarity. • Oregon’s arts standards overall are of good quality and cover a breadth and depth of content. The standards generally are clearly written, focus on important skills/concepts for instruction and are assessable. • Oregon's physical education standards overall are of good quality in terms of breadth and depth of content coverage, consistency across benchmark grades, importance for instruction, and degree of assessability. However, analysts determined that these standards are not easily understood due to lack of clarity and precision of language.

  13. Standards Evaluation Studies (continued) Overall Quality (continued) • Oregon’s health standards overall are of good quality in terms of depth, consistency, importance for instruction, and assessability. For all benchmark levels except for Benchmark 2, the standards overall are of good quality in terms of breadth of content. And, overall these standards lack clarity. • Oregon’s educational technology standards overall are of good quality in terms of depth, breadth, consistency, importance for instruction, and assessability. However, overall these standards lack clarity. • Oregon’s second languages standards overall are of good quality in terms of depth, breadth, clarity, consistency, importance for instruction, and assessability.

  14. Standards Evaluation Studies (continued) Overall Quality (continued) • Generally, Oregon’s English language proficiency (ELP) forms and functions are presented in a clear and useful format. The proficiency level descriptors clearly describe and differentiate the language skills of students at each level. However, the definitions of language function and forms of language appear incomplete. All the language functions except one are appropriate—literary analysis is typically considered English-Language Arts content rather than a language function. All the language forms except one are appropriate—the language of propaganda is not a form consistent with other elements of form presented by the state; rather it appears to be a genre.

  15. Standards Evaluation Studies (continued) Alignment to External Referents • Oregon’s ELA standards generally aligned to the Indiana, NAEP and McREL standards in terms of overall content order/sequence and depth of content. However, while the breadth of the Oregon standards appeared comparable to the NAEP and McREL standards, they were not as comparable to the Indiana standards. • Oregon’s mathematics standards were compared to Indiana’s mathematics standards as well as to the NCTM Principles and Standards for School Mathematics (2000) and Curriculum Focal Points for K–8 Mathematics (2006). Oregon’s mathematics standards generally aligned to the NCTM standards and to the Indiana standards with regard to overall depth and breadth of content coverage. However, there was not overall congruence between Oregon and Indiana’s content ordering and sequencing.

  16. Standards Evaluation Studies (continued) Alignment to External Referents (continued) • Oregon’s science standards were compared to Indiana’s science standards and the AAAS benchmarks (2001). • Generally, the Earth and Space Science standards aligned in terms of overall depth of content; however, they did not align in terms of content order/sequence or overall breadth of content. • Generally, the Physical Science standards aligned to the Indiana standards in terms of content order/sequence, overall depth, and overall breadth of content. The Oregon standards also aligned to the AAAS standards in terms of overall depth and breadth of content; however, they did not align in terms of content order/sequence. • Generally, the Life Science standards aligned to the Indiana standards in terms of content order/sequence, overall depth, and overall breadth of content. However, they did not align to the AAAS standards in terms of content order/sequence, overall breadth, or overall depth of content.

  17. Standards Evaluation Studies (continued) Alignment to External Referents (continued) • Oregon’s socialsciences standards were compared to Washington’s social science standards and the NAEP frameworks. • Generally, the Geography standards aligned in terms of overall content order/sequence with Washington’s standards. However, they did not align in terms of content order/sequence with the NAEP framework or in terms of the overall depth or breadth of content with the Washington standards or the NAEP framework. • Generally, the History standards did not align to the NAEP framework in terms of content order/sequence, depth or breadth of content. Nor did these standards align in terms of overall breadth with the Washington U.S. History and World History standards. However, the Oregon standards aligned to the Washington standards in terms of content order/sequence and overall depth.

  18. Standards Evaluation Studies (continued) Alignment to External Referents (continued) • Oregon’s socialsciences standards (continued). • Generally, the Civics standards aligned in terms of overall content order/sequence with Washington’s standards and the NAEP framework. However, they did not align in terms of overall depth or breadth of content with either the Washington standards or the NAEP framework. • Generally, the Economics standards aligned to the Washington standards and the NAEP framework in terms of content order/sequence, depth and breadth of content. • At the broad level of the content standard, there was a high degree of alignment between Oregon’s arts standards and those of Washington and McREL. In terms of content order / sequence, Washington’s standards spiral content across different benchmarks in a manner similar to Oregon; many of McREL’s benchmark level standards are not spiraled across different benchmarks. Both external referents show similar depth of content to Oregon’s standards.

  19. Standards Evaluation Studies (continued) Alignment to External Referents (continued) • Generally, Oregon's physical education standards, the National Association of Sports and Physical Education Standards (NASPE), and the Indiana standards share a common purpose. In broad terms, the nature of the content knowledge/cognitive information and the types of skills students are expected to learn and be able to demonstrate are comparable across the three sets of standards. The Oregon differs in depth of content coverage from the NASPE and Indiana State standards. • Oregon’s health standards were compared to Indiana’s health standards as well as to the National Health Education Standards (NHES). Oregon’s standards generally aligned to the Indiana and NHES standards with regard to overall content order/sequence and breadth of content coverage. However, there was not overall congruence in terms of depth of content.

  20. Standards Evaluation Studies (continued) Alignment to External Referents (continued) • Oregon’s educational technology standards generally aligned to the Washington and NETS standards in terms of overall content order/sequence, depth of content and breadth of content. • While Oregon and Indiana both have the same number of proficiency levels, a comparison of the depth and breadth of the second languages standards across levels in both states shows that the Indiana standards start at a slightly higher level than those in Oregon, and reach a higher level. Oregon’s proficiency levels appear to be based on ACTFL levels Novice-Low through Intermediate-Mid. The standards statements differ in level of detail. Generally, Oregon standards are written more broadly, and Indiana’s are more detailed and specific. Oregon’s set of Functions Supporting Standards describe a similar level of detail as Indiana’s standards.

  21. Standards Evaluation Studies (continued) Alignment to External Referents (continued) • Oregon’s English language proficiency (ELP) forms and functions were compared to current national research and literature on forms and functions (e.g., TESOL, CALLA, Butler, et al.). All but one of Oregon’s language functions (literary analysis) are appropriate and relevant for ELD instruction. The external referents include functions that are currently not present among those in the Oregon ELP standards.

  22. Standards Evaluation StudiesGeneral Recommendations • Evaluate standards for redundancy within a grade. • Evaluate standards for level of detail—some standards have elaborate detail and others do not. • Differentiate between detail that describes the curriculum to be taught and the intended learning outcomes (content skills, knowledge). • Evaluate coherence across levels of specificity for a given standard—e.g., strand, common curriculum goals (CCGs), grade-level standards—in terms of appropriateness of progression of specificity (general/broad to specific) and clarity of relationships.

  23. General Recommendations(continued) • Verify that expansion/extension of content reflected in standards (e.g., references that include student’s knowledge of the community, knowledge of historical periods) is reasonable and intentional. • Evaluate the clarity and effectiveness of small variations in the wording/language of specific standards across the grade levels—verify that the differences are intentional and reflect clear, real differences in expectations of students vis-à-vis the content/skill.

  24. General Recommendations(continued) • Embed definitions of key content terminology, if interpretations of terminology can be varied AND the intent of the state is to promote consistency • Make sure the language clearly conveys expected skills and complexity • Evaluate coherence and consistency of content emphases across grades

  25. Core Standards • “Content Standards” define the knowledge and skills that a state’s students are supposed to learn • The definition of “Core Standards” includes/reflects: • Knowledge and skills central to a content area • Significant/essential content/skill “targets” in a content area • State-specific priorities in a content area • Learning expectations and performance goals for all students • Core standards subsume other standards and/or provide a super-structure for a coherent system of content standards • Some states identify core standards in order to ensure that these standards are addressed in each grade and developed across grades—local curricula are expected to align to these standards

  26. Core Standards (continued) Considerations: • Will the standard provide students with knowledge and skills that will be of value beyond a single test date? (What endures?) • Will the standard provide knowledge and skills that will be of value in multiple disciplines? (What has leverage?) • Will the standard provide students with essential knowledge and skills that are necessary for success in the next level of instruction? Beyond school? (What is really necessary for “success”?) [Based on Reeves, 2004]

  27. Essential Skills • “Essential Skills” refer to the core abilities/process skills that all students are expected to develop and apply across disciplines • Essential skills typically span content areas and facilitate the effective delivery of content knowledge across disciplines • Essential skills typically are expected to be demonstrated in a variety of ways and in different contexts • Essential skills reflect state’s priorities • Local curricula are expected to incorporate these skills—the skills are frequently embedded within existing learning standards, curricula, and assessments.

  28. Essential Skills (continued) Considerations: • Will the skills be of value beyond a single test date? (What endures?) • Will the skills be of value in multiple disciplines? (What has leverage?) • Will the skills be necessary for success in the next level of instruction? Beyond school? (What is really necessary for “success”?) [Adapted from Reeves, 2004]

  29. Next Steps • Additional alignment analyses and information • Additional information regarding assessments and standards (ODE questions) • Analyses of “core standards” • Analyses of “essential skills” • Continued consideration of feedback/information provided by key stakeholders through surveys, meetings, etc.

  30. For more information about the standards and assessment evaluation: ode.state.or.us/go/real For more information about WestEd: www.wested.org

More Related