Models for aligning assessments to standards
1 / 15

- PowerPoint PPT Presentation

  • Uploaded on

Models for Aligning Assessments to Standards. Consortia Conference Call August 23, 2005 Regie Stites SRI International. Assessment Limits. Some (more) Chinese sayings: “Viewing the heavens through a bamboo tube, Measuring the ocean with a spoon.” “Riding a horse, viewing flowers.”

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation

PowerPoint Slideshow about '' - Jimmy

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Models for aligning assessments to standards l.jpg

Models for Aligning Assessments to Standards

Consortia Conference Call

August 23, 2005

Regie Stites

SRI International

Assessment limits l.jpg
Assessment Limits

Some (more) Chinese sayings:

“Viewing the heavens through a bamboo tube, Measuring the ocean with a spoon.”

“Riding a horse, viewing flowers.”

“Frog sitting in a well, viewing the sky.”

Accountability demands l.jpg
Accountability Demands

An American saying:

“What gets tested, gets taught.”

When West meets East, we’ve got a world of trouble unless …

  • everyone understands the limits of assessment, and

  • we achieve reasonably good alignment of standards, assessment, and instruction.

Two directions for alignment l.jpg
Two Directions for Alignment

Horizontal – align content standards to assessments, match content and depth

Vertical – align accountability tests to educational systems (curricula, materials content, instruction, student outcomes, stakeholder opinions)

How to align tests and standards three methods l.jpg
How to Align Tests and Standards:Three Methods

  • Sequential – develop content standards and curriculum, then develop standards-based tests

  • Expert review – use expert judgments to evaluate alignment, then select/reject test or explicate alignment

  • Content analysis – use content criteria to guide formal analysis, then select/reject test or explicate alignment

How to analyze alignment four expert models l.jpg
How to Analyze Alignment: Four Expert Models

  • ‘Webb’ – procedures and criteria developed by Norman Webb

  • ‘SEC’ – Surveys of Enacted Curriculum

  • ‘Achieve’ – alignment analysis service provided by Achieve, Inc.

  • ‘CBE’ – technical assistance from the Council for Basic Education

    See ‘Dimensions of Comparison’ on page 6 of Models for Alignment Analysis and Assistance to States, CCSSO (2002)

How alignment analysis helps l.jpg
How Alignment Analysis Helps

  • Provides feedback for revision of assessments or standards

  • Provides information to help align instruction with assessments and standards

  • Helps everyone understand the proper limits for interpreting results of accountability tests

Metaphors for alignment goals l.jpg
Metaphors for Alignment Goals

Alignment as Congruence – assessment and standards content mirror each other

Alignment as a Set of Correspondences – assessment content samples from standards content

Alignment as a Bridge –curriculum content connects standards and assessments

Alignment as Gravitational Pull – standards, assessments, and instruction all reflect common elements of a reform agenda

[from Baker, 2004]

Achieve s alignment protocol four dimensions l.jpg
Achieve’s Alignment Protocol:Four Dimensions

  • Content centrality – test item match to content of related standard

  • Performance centrality – test item match to the type of performance (cognitive demand) of related standard

  • Challenge – source and level of challenge for sets of items

  • Balance and range – match between emphasis and coverage of sets of items and standards

Achieve s alignment protocol three stages l.jpg
Achieve’s Alignment Protocol:Three Stages

Stage 1: Item-by-item analysis

1(a) Confirming the test blueprint

1(b) Content centrality

1(c) Performance centrality

Stage 2: Challenge

2(a) Source of challenge (each item)

2(b) Level of challenge (sets of items mapped to each standard)

Stage 3: Balance and range (sets of items mapped to each standard)

3(a) Balance (emphasis)

3(b) Range (coverage)

Achieve s alignment protocol ratings l.jpg
Achieve’s Alignment Protocol:Ratings

Stage 1: Content and performance centrality ratings =

2 clearly consistent; 1A not specific enough;

1B somewhat consistent; 0 inconsistent

Stage 2: Source of challenge rating =

1 appropriate source(s); 0 inappropriate source(s)

Stage 2: Level of challenge rating = narrative evaluation

Stage 3: Balance rating = narrative evaluation

Stage 3: Range rating = fraction of total objectives (indicators) mapped to standard assessed by at least one item

[above .67 = good, between .50 and .66 = acceptable]

Achieve s alignment protocol results l.jpg
Achieve’s Alignment Protocol:Results

Elementary English Language Arts

Content centrality (% of items rated 2)

State A = 100; State B = 89; State C = 66; State D = 46; State E = 29

Performance centrality (% of items rated 2)

State A = 100; State B = 79; State C = 62; State D = 79; State E = 21

Source of challenge (% of items rated ‘appropriate’)

State A = 90; State B = 76; State C = 89; State D = 88; State E = 79

Average range scores

State A = .27; State B = .75; State C = .73; State D = .52; State E = .31

Some resources l.jpg
Some Resources

Achieve alignment procedures

Robert Rothman et al. (May 2002). Benchmarking and alignment of standards and testing.

Metaphors for alignment

Eva Baker. (December 2004). Aligning curriculum, standards, and assessments: Fulfilling the promise of school reform.

Test publishers see more alignment than teachers

Chad Buckendahl et al. (2000). Alignment of standardized achievement tests to state content standards: A comparison of publishers’ and teachers’ perspectives.

‘Nine requirements’ for a responsible state assessment system

Commission on Instructionally Supportive Assessment. (October 2001). Building tests to support instruction and accountability: A guide for policymakers.

Resources continued l.jpg
Resources Continued

  • Horizontal and Vertical AlignmentBetsy Chase and Sasha Zucker, Harcourt Assessments, Inc.

  • Rethinking Issues of Alignment Under No Child Left BehindSri Ananda, WestEd.

Discussion points l.jpg
Discussion Points

  • What are your goals for alignment? What do you want to learn from an alignment analysis? What will you do with the results of an alignment analysis?

    • help educators and stakeholders interpret results of accountability tests?

    • influence instructional content and practice?

    • shape policy?