The Use of Accommodations for a K-12 Standardized Assessment:
Download
1 / 91

The Use of Accommodations for a K-12 Standardized Assessment: Practical Considerations from a State Perspective 1999 thr - PowerPoint PPT Presentation


  • 146 Views
  • Uploaded on

The Use of Accommodations for a K-12 Standardized Assessment: Practical Considerations from a State Perspective 1999 through 2004. Patricia Almond, University of Oregon      Meagan Karvonen, Western Carolina University March 21, 2006 Savannah, GA.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'The Use of Accommodations for a K-12 Standardized Assessment: Practical Considerations from a State Perspective 1999 thr' - natasha


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Slide1 l.jpg

The Use of Accommodations for a K-12 Standardized Assessment:Practical Considerations from a State Perspective 1999 through 2004

Patricia Almond, University of Oregon     Meagan Karvonen, Western Carolina University

March 21, 2006

Savannah, GA

Almond, Karvonen Savannah, GA March 2006


Although swd often take state tests with accommodations l.jpg
Although SWD often take state tests with accommodations Assessment:

States often have a hard time determining which accommodations to allow . . .(Olson, 2004)

Almond, Karvonen Savannah, GA March 2006


Accommodations from a state perspective in the context of requirements and responsibilities l.jpg

Accommodations from a State Perspective Assessment:In the Context of Requirements and Responsibilities

IDEA/NCLB

State Legislation

State Policies (testing, promotion, graduation, etc.)

Training and Guidelines

Assessment Technical Adequacy

Almond, Karvonen Savannah, GA March 2006


Multiple purposes and full participation l.jpg

Multiple Purposes and Full Participation Assessment:

“To earn their keep, state assessment systems attempt to do it all, often by asking the same assessment to fulfill several purposes.”

(Almond, Lehr, Thurlow, Quenemoen, 2002, p. 346)

Almond, Karvonen Savannah, GA March 2006


Purposes that can be applied to large scale assessment l.jpg
Purposes that can be applied to large-scale assessment: Assessment:

  • Aiding in instructional decisions about individual students

  • Providing information about the status of the education system

  • Motivating change

  • Evaluating program

  • Holding schools accountable for student performance

  • Encouraging change in classroom instruction

  • Certifying individual student achievement and mastery

(Heubert and Hauser, 1999)

Almond, Karvonen Savannah, GA March 2006


Types of decisions based on results obtained from assessments taken with accommodations l.jpg

School and District Accountability Assessment:

Schools are effective or in need of improvement

High stakes rewards and sanctions will cause improved achievement

Individual Student Proficiency

Promotion to next grade

Graduation from high school

Certification

Types of decisions based on results obtained from assessments taken with accommodations:

Almond, Karvonen Savannah, GA March 2006


Standard 13 2 the standards l.jpg
Standard 13.2 the Assessment:Standards

In educational settings, when a test is designed or used to serve multiple purposes, evidence of the test’s technical quality should be provided for each purpose.

Almond, Karvonen Savannah, GA March 2006


Aligning the newly reauthorized idea with title 1 l.jpg
Aligning the Newly Reauthorized Assessment:IDEA with Title 1

. . . States have an affirmative obligation to determine what types of accommodations can be made to assessments while maintaining their reliability and validity

House Committee Report on IDEA reauthorization? (108-77, April 29, 2003)

Almond, Karvonen Savannah, GA March 2006


State responsibilities l.jpg
State Responsibilities Assessment:

  • Documentation for Peer Review

    • Written policies, lists of allowable accommodations, score reports, empirical studies

  • Training for educators on how to select appropriate accommodations that will not invalidate the scores

    - Peer Review Guidance (2004)

Almond, Karvonen Savannah, GA March 2006


State policies on accommodations l.jpg
State Policies on Accommodations Assessment:

  • Differentiating between accommodations (allowable, standard) and modifications (not allowable, nonstandard)

  • Variations in documentation policies, reporting practices, eligibility guidelines (IEP, 504, ELL, universal)

  • Some decision making criteria:

    • Use of instructional accommodations (sometimes used for period of time prior to assessment)

    • Accommodation does not provide unfair advantage/change validity

    • Prohibitions against decisions based on program setting, disability category

    • Thurlow, Lazarus, Thompson, & Morse, 2005

Almond, Karvonen Savannah, GA March 2006


Potential problems in decision making school level l.jpg
Potential problems in decision-making Assessment:(school level)

  • Offering more (or fewer) accommodations than might be needed

  • Mismatch between instructional and assessment accommodations

  • Lack of documentation about decisions made

  • Lack of knowledge about accommodation vs. modification

Almond, Karvonen Savannah, GA March 2006


Guidance from the standards l.jpg
Guidance from the Assessment:Standards

  • Those who make accommodation decisions should be aware of existing research on the effects of disabilities on performance (10.2).

  • Decision-makers need to have access to accurate and current information about potential accommodations and modifications (10.8).

  • Accommodations and their rationales need to be described in detail (in administration manual and technical documents), including possible limits on validity of inferences (10.4, 10.5)

Almond, Karvonen Savannah, GA March 2006


Progress and room for growth l.jpg
Progress, and room for growth… Assessment:

  • Increased use of accommodations in large-scale assessments (Ysseldyke et al., 2004)

  • States recently identified as current and emerging issues related to accommodations:

    • Determination of appropriate and allowable accommodations

    • Technical assistance on appropriateness of specific accommodations for both instruction and assessment

    • Professional development, especially for general education teachers

    • Degree of specificity required in state guidance

      -NCEO 2005 State Report

Almond, Karvonen Savannah, GA March 2006


In the opening session we received a report on the status of accommodations l.jpg
In the Opening Session we received a report on the status of accommodations:

Accommodations in State Policies-What a Wonderful World of Diversity: Issues and Implications     Martha Thurlow, NCEO

Almond, Karvonen Savannah, GA March 2006


Slide15 l.jpg

In this Session We Provide accommodations:A Case Study: One State’s Approach to Determining “Allowable” Testing Accommodations From 1999 through 2004

Almond, Karvonen Savannah, GA March 2006


Chronology l.jpg
Chronology accommodations:

Almond, Karvonen Savannah, GA March 2006


Comprehensive not alternate l.jpg
Comprehensive accommodations:notAlternate

  • Integral part of the whole assessment system

  • Inclusive of all students

Almond, Karvonen Savannah, GA March 2006


Oregon s comprehensive assessment system in 1999 2000 l.jpg

Knowledge & Skills accommodations:

Reading/Literature

Mathematics

Science

Perf. Assessments

Writing

Math Problem Solving

All with Adaptations

Accommodations

Modifications

Extended Perf. Measures

X-Reading

X-Math

X-Writing

X-CLRAS

Oregon’s Comprehensive Assessment System In 1999-2000

Almond, Karvonen Savannah, GA March 2006


Nceo description of accommodations l.jpg
NCEO Description of Accommodations accommodations:

“Accommodations are changes in testing materials or procedures that enable students to participate in assessments in a way that allows abilities to be assessed rather than disabilities. They are provided to ’level the playing field.’ Without accommodations, the assessment may not accurately measure the [individual] student’s knowledge and skills.”

Emphasis added.

Almond, Karvonen Savannah, GA March 2006


Accommodations and modifications tables in test administration manuals l.jpg
Accommodations and Modifications Tables in Test Administration Manuals

Almond, Karvonen Savannah, GA March 2006


The settlement feb 2001 l.jpg
The Settlement Feb. 2001 Administration Manuals

. . . Accommodations shall be considered allowable, valid, and scorable if they are used during instruction[1] . . . and are listed on a student’s IEP or Section 504 plan, unless [the department] can show that the accommodation invalidates the score interpretation.[2]

Almond, Karvonen Savannah, GA March 2006


The settlement feb 2001 continued l.jpg
The Settlement Feb. 2001 continued Administration Manuals

. . . Rather than consider all accommodations first invalid until proven to be valid, [the department] shall consider all accommodations valid unless [it] can show that the accommodation would invalidate the score interpretation.

Almond, Karvonen Savannah, GA March 2006


The settlement feb 2001 continued23 l.jpg
The Settlement Feb. 2001 continued Administration Manuals

Any list of approved or not approved accommodations published by [the department] as a guide for school districts shall not be deemed exclusive . . .

Almond, Karvonen Savannah, GA March 2006


The settlement feb 2001 continued24 l.jpg
The Settlement Feb. 2001 continued Administration Manuals

When . . . a particular accommodation would invalidate the score interpretation, [the department] must provide an alternate assessment and/or alternate scoring procedures . . . to afford . . . students an assessment option that leads to the Certificate of Initial Mastery.

Almond, Karvonen Savannah, GA March 2006


2000 2001 expanded menu of testing options for full participation l.jpg
2000-2001 Expanded Menu of Administration ManualsTesting Options for Full Participation

  • Regular Tests & Achievement Level Testing (Rdg/Lit & Math)*

  • Challenge up or down

  • Accommodations

  • Modifications

  • Extended Reading, Math, Writing

  • Ext. Career and Life Role Assessment

  • Juried Assessment

* Regular refers to tests administered with and without accommodations.

Almond, Karvonen Savannah, GA March 2006


Juried assessment l.jpg
Juried Assessment Administration Manuals

  • Juried CIM Assessment

  • Juried Modifications

Almond, Karvonen Savannah, GA March 2006


Pathways to certificate of initial mastery cim l.jpg
Pathways to Certificate of Initial Mastery (CIM) Administration Manuals

Collections

  • ODE Collection of Evidence

  • OR University System—Proficiency Based Admission Standards System (PASS)

  • Statewide Assessments

  • CIM level State Tests (w/ or w/o accommodations)

  • CIM level State Tests w/ individually approved modifications

Almond, Karvonen Savannah, GA March 2006


Juried modifications l.jpg
Juried Modifications Administration Manuals

  • Available at CIM Level

  • District brings request forward

  • Student must be able to meet state standards

  • Decision is student-by-student & test-by-test

Almond, Karvonen Savannah, GA March 2006


Three examples of such requests for a juried review of a modification follow l.jpg
“Three examples of such requests for a juried review of a modification follow:

  • A student literate in a language other than English, Spanish or Russian, wishes to take the mathematics test translated into his or her primary language. Such a translation would typically be considered a modification.

    2. A student with limited physical dexterity writes using a special word processing program that creates words or phrases from a few keystrokes and the student wishes to take the writing test using the software.

    3. A student with a significant learning disability such as dyslexia, affecting the student’s ability to read and write, uses auditory methods such as taped texts and readers as a primary learning mode.”

Almond, Karvonen Savannah, GA March 2006


Slide30 l.jpg

“The panel might approve this modification as an accommodation for the particular student after reviewing the student's case if:

The student is skilled in using the read aloud adaptations

The measure of comprehension reflects the student’s own knowledge and understanding

The student achieves the same standards for interpreting text required of all students”

“ . . . a significant learning disability who uses assistive technology, screen readers, and recorded text . . . understanding text and interpreting “meaning”.

Almond, Karvonen Savannah, GA March 2006


Slide31 l.jpg

“If approved, the student would be permitted to use the "read aloud" modification with the Reading/Literature Knowledge and Skills assessment and have the opportunity to "meet" (e.g. be determined "proficient" on the standard.)”

Almond, Karvonen Savannah, GA March 2006


Accommodations in oregon l.jpg
Accommodations in Oregon "read aloud" modification with the Reading/Literature Knowledge and Skills assessment and have the opportunity to "meet" (e.g. be determined "proficient" on the standard.)”

Almond, Karvonen Savannah, GA March 2006


The accommodations panel l.jpg

The Accommodations Panel "read aloud" modification with the Reading/Literature Knowledge and Skills assessment and have the opportunity to "meet" (e.g. be determined "proficient" on the standard.)”

Almond, Karvonen Savannah, GA March 2006


Purpose l.jpg
Purpose: "read aloud" modification with the Reading/Literature Knowledge and Skills assessment and have the opportunity to "meet" (e.g. be determined "proficient" on the standard.)”

Conduct review of proposed additions to the accommodations table for the test administration manual to assure that that the adaptation will not invalidate interpretations and judgments based on the resulting test score.

Almond, Karvonen Savannah, GA March 2006


Process l.jpg
Process "read aloud" modification with the Reading/Literature Knowledge and Skills assessment and have the opportunity to "meet" (e.g. be determined "proficient" on the standard.)”

  • Proposal to add adaptation to the Accommodations Table

  • Staff analysis of relevant research and legal decisions about the effects of an adaptation on score validity

  • Accommodations Panel review of evidence and deliberation

Almond, Karvonen Savannah, GA March 2006


Results l.jpg
Results "read aloud" modification with the Reading/Literature Knowledge and Skills assessment and have the opportunity to "meet" (e.g. be determined "proficient" on the standard.)”

  • A recommendation to include the adaptation either on the Accommodations Table or the Modifications Table

  • A decision by the Associate Superintendent of Assessment and Evaluation about the status of the adaptation

  • Publication in test administration manual and revised tables

Almond, Karvonen Savannah, GA March 2006


Accommodations panel membership l.jpg

Deaf Educator "read aloud" modification with the Reading/Literature Knowledge and Skills assessment and have the opportunity to "meet" (e.g. be determined "proficient" on the standard.)”

Math Content Specialist*

ESL Teacher

Vision Specialist

District Test Administrator

Special Ed. Director

English Language Arts Content Specialist*

Parent

Educational Measurement Expert

University Researcher

School Principal

Special Ed. Teacher

Accommodations Panel Membership

* Formerly assessment item writers and assessment content panel members.

Almond, Karvonen Savannah, GA March 2006


Panel members will l.jpg
Panel members will: "read aloud" modification with the Reading/Literature Knowledge and Skills assessment and have the opportunity to "meet" (e.g. be determined "proficient" on the standard.)”

  • Review adaptations, research reports, and assessment materials,

  • Participate in deliberations and decision-making based on review criteria, and

  • Advise the department regarding test accommodations and modifications

Almond, Karvonen Savannah, GA March 2006


Member responsibilities l.jpg
Member Responsibilities: "read aloud" modification with the Reading/Literature Knowledge and Skills assessment and have the opportunity to "meet" (e.g. be determined "proficient" on the standard.)”

  • Represent area of knowledge and expertise represented by position on panel

  • Be familiar with the State Assessments

  • Attend all meetings

  • Apply review criteria fairly and ethically

  • Work collaboratively with other members

Almond, Karvonen Savannah, GA March 2006


Panel member requirements l.jpg
Panel Member Requirements: "read aloud" modification with the Reading/Literature Knowledge and Skills assessment and have the opportunity to "meet" (e.g. be determined "proficient" on the standard.)”

  • Knowledge: Masters or analogous

  • Education Experience: 5 years or equivalent

  • Advisory Experience: Prior experience on advisory or content panel

  • Test Experience: Prior testing experience

  • Teamwork: Effectively work as a member of a group

Almond, Karvonen Savannah, GA March 2006


Terminology for testing accommodations l.jpg
Terminology for Testing Accommodations "read aloud" modification with the Reading/Literature Knowledge and Skills assessment and have the opportunity to "meet" (e.g. be determined "proficient" on the standard.)”

ADAPTATION

  • Change made to the environment, curriculum, instruction, and/or assessment practices in order for a student to be a successful learner. Adaptations include accommodations and modifications. Adaptations are based on an individual students’ strengths and needs, and may vary in intensity and degree (originally from the Colorado Department of Education, 1995).

Almond, Karvonen Savannah, GA March 2006


Terminology continued l.jpg
Terminology "read aloud" modification with the Reading/Literature Knowledge and Skills assessment and have the opportunity to "meet" (e.g. be determined "proficient" on the standard.)”continued

ACCOMMODATION

  • An alteration in how a test is presented to or responded to by the person tested; includes a variety of alterations in presentation format, response format, setting in which the test is taken, timing or scheduling. The alterations do not substantially change level, content or performance criteria. The changes are made in order to provide a student equal access to learning and equal opportunity to demonstrate what is known.

Almond, Karvonen Savannah, GA March 2006


Terminology continued43 l.jpg
Terminology "read aloud" modification with the Reading/Literature Knowledge and Skills assessment and have the opportunity to "meet" (e.g. be determined "proficient" on the standard.)”continued

MODIFICATION

  • Substantial changes in what a student is expected to learn and/or demonstrate. The changes are made to provide a student opportunities to participate meaningfully and productively in learning experiences and environments. They include changes in instructional level, content, and performance criteria.

Almond, Karvonen Savannah, GA March 2006


Review criteria l.jpg
Review Criteria "read aloud" modification with the Reading/Literature Knowledge and Skills assessment and have the opportunity to "meet" (e.g. be determined "proficient" on the standard.)”

  • Standards

  • Purpose

  • Function

  • Consequence

  • Social Consequence

  • Systems Change

Almond, Karvonen Savannah, GA March 2006


Review criteria continued 1 content and performance standards l.jpg
Review Criteria Continued: "read aloud" modification with the Reading/Literature Knowledge and Skills assessment and have the opportunity to "meet" (e.g. be determined "proficient" on the standard.)”1. Content and Performance Standards:

  • What is intended to be measured by the assessment?

  • Clarification for this question provided by department of education staff.

Almond, Karvonen Savannah, GA March 2006


Review criteria continued 2 purpose l.jpg
Review Criteria Continued: "read aloud" modification with the Reading/Literature Knowledge and Skills assessment and have the opportunity to "meet" (e.g. be determined "proficient" on the standard.)”2. Purpose:

  • What is the requestor’s intent?

  • For example, to enhance access to test or to increase number of students who can participate?

  • Why was the request made to add the adaptation?

Almond, Karvonen Savannah, GA March 2006


Review criteria continued 3 function l.jpg
Review Criteria Continued: "read aloud" modification with the Reading/Literature Knowledge and Skills assessment and have the opportunity to "meet" (e.g. be determined "proficient" on the standard.)”3. Function:

  • What is the effect of the adaptation?

  • How does it work?

  • How will the accommodation be used?

  • How does it need to be done? For example, scribe accommodation required protocols for training and implementation.

Almond, Karvonen Savannah, GA March 2006


Review criteria continued 4 consequence l.jpg
Review Criteria Continued: "read aloud" modification with the Reading/Literature Knowledge and Skills assessment and have the opportunity to "meet" (e.g. be determined "proficient" on the standard.)”4. Consequence:

  • What inferences are being made or jeopardized?

  • What mistakes will be made if the adaptation is used as an accommodation? Ripple effect?

Almond, Karvonen Savannah, GA March 2006


Review criteria continued 5 social consequences l.jpg
Review Criteria Continued: "read aloud" modification with the Reading/Literature Knowledge and Skills assessment and have the opportunity to "meet" (e.g. be determined "proficient" on the standard.)”5. Social Consequences:

What affect will the testing adaptation have on students across settings?

What is the impact on students of using the adaptation?

Also look more globally—Does adaptation influence general perceptions of the public about the validity of assessment?

Almond, Karvonen Savannah, GA March 2006


Review criteria continued 6 system change l.jpg
Review Criteria Continued: "read aloud" modification with the Reading/Literature Knowledge and Skills assessment and have the opportunity to "meet" (e.g. be determined "proficient" on the standard.)”6. System change:

  • Would proposed adaptation constitute a system change outside the Accommodation Panel’s sphere of responsibility?

  • Does it propose a change to the assessment system and/or to the content and performance standards?

Almond, Karvonen Savannah, GA March 2006


Context within which to consider criteria l.jpg
Context within which to consider criteria: "read aloud" modification with the Reading/Literature Knowledge and Skills assessment and have the opportunity to "meet" (e.g. be determined "proficient" on the standard.)”

  • Equity—accessibility of resources

  • Instructional alignment

  • Reality (employability)

  • Fairness (advantage or disadvantage)

Almond, Karvonen Savannah, GA March 2006


Sources of evidence l.jpg
Sources of Evidence: "read aloud" modification with the Reading/Literature Knowledge and Skills assessment and have the opportunity to "meet" (e.g. be determined "proficient" on the standard.)”

  • Research

  • Professional expertise and judgment

  • Court/Hearing Decisions

Almond, Karvonen Savannah, GA March 2006


How it worked l.jpg

How it worked! "read aloud" modification with the Reading/Literature Knowledge and Skills assessment and have the opportunity to "meet" (e.g. be determined "proficient" on the standard.)”

Almond, Karvonen Savannah, GA March 2006


Slide54 l.jpg

http://www.ode.state.or.us/opportunities/grants/sig/statewide-assessment/validated.pdfhttp://www.ode.state.or.us/opportunities/grants/sig/statewide-assessment/validated.pdf

Almond, Karvonen Savannah, GA March 2006


Slide55 l.jpg

http://www.ode.state.or.us/teachlearn/testing/manuals/tables/nominationaccomform.dochttp://www.ode.state.or.us/teachlearn/testing/manuals/tables/nominationaccomform.doc

Almond, Karvonen Savannah, GA March 2006


For example proposed adaptation feb 01 scribe l.jpg

For Example:http://www.ode.state.or.us/teachlearn/testing/manuals/tables/nominationaccomform.docProposed Adaptation (Feb ‘01): Scribe

Assessment: Mathematics Problem Solving

Almond, Karvonen Savannah, GA March 2006


Slide57 l.jpg

Point to or dictate multiple-choice responses to a scribe*http://www.ode.state.or.us/teachlearn/testing/manuals/tables/nominationaccomform.doc

Footnote:

* Test administrators, proctors, scribes, page-turners, educational assistants, and others supporting a student’s test taking must be neutral in responding to the student during test administration. Assistance in test administration must not give away the correct answer. The student’s response must accurately represent the student’s own choice.

Scribe as it appeared on accommodations table for multiple choice assessment in 2000 prior to Settlement

Almond, Karvonen Savannah, GA March 2006


Adaptation as proposed from the field l.jpg

Adaptation as proposed from the field:http://www.ode.state.or.us/teachlearn/testing/manuals/tables/nominationaccomform.doc

Almond, Karvonen Savannah, GA March 2006


Description student dictates mathematics problem solving response to a scribe l.jpg
Description:http://www.ode.state.or.us/teachlearn/testing/manuals/tables/nominationaccomform.doc Student dictates mathematics problem solving response to a scribe.

Student

  • Reads mathematics problem

  • Solves problem using manipulatives, straight edge, and stencils

  • Dictates written response to scribe

Almond, Karvonen Savannah, GA March 2006


Rationale from person submitting nomination l.jpg
Rationale from Person Submitting Nomination:http://www.ode.state.or.us/teachlearn/testing/manuals/tables/nominationaccomform.doc

The use of a scribe in MPS opens up the context to more accurately represent society norms.

Training scribe removes mantel of the “unfair” biases of using a scribe and is consistent with what others receive.

Other factors:

None specified.

Almond, Karvonen Savannah, GA March 2006


Research review l.jpg

Research Reviewhttp://www.ode.state.or.us/teachlearn/testing/manuals/tables/nominationaccomform.doc

Almond, Karvonen Savannah, GA March 2006


Dictation to a proctor or scribe conclusions tindal fuchs 1999 l.jpg
Dictation to a Proctor or Scribe Conclusions, Tindal & Fuchs (1999)

  • Dictation consistently boosts performance on both objective and performance assessments

  • True both for students with & without disabilities in elementary through college

  • Evidence was inconclusive either in support or in criticism of dictation as an accommodation

Almond, Karvonen Savannah, GA March 2006


Research on dictation to proctor scribe in math l.jpg

Limited focus in empirical studies, mixed results (1999)

Implausibly high scores for some groups and in some grades

Benefit in problem-solving CBM, but not concept & application (Fuchs et al., 2000)

Use of accommodations (including dictation) did not benefit SWD compared with those without disabilities on a state performance assessment (Elliott et al., 1999)

Conflicting results on 3 related studies re: impact of dictation on score (Koretz, 1997; Koretz & Hamilton, 1999; Koretz & Hamilton, 2000); scores ranged from lower than those without disabilities

Research on Dictation to Proctor/Scribe in Math

Almond, Karvonen Savannah, GA March 2006


Panel review l.jpg

Panel Review (1999)

Almond, Karvonen Savannah, GA March 2006


Panel deliberation l.jpg

Purpose: (1999)

Capture a response otherwise unavailable

Capture an optimal response (minimize interfering behavior or maximize ‘capable’ recording)

In MPS student interprets task, chooses strategies, shows path of thinking

Function of Allowing/Disallowing:

Student’s motor skills do not preclude participation

Allows unconfounded assessment of the MPS construct

Scribes may have limited knowledge and may alter the student’s response

Performance can be generalized

Panel Deliberation:

Almond, Karvonen Savannah, GA March 2006


Panel deliberation continued l.jpg

Social consequences: (1999)

Problem solving can be contextualized in multiple ways

Accommodation can be shaped and become time limited

Increases participation & may bring down scores

Scores document individual competence, may lead to improved instruction

Medium

Computer

Video

Scribe (unfamiliar/

trained)

Panel Deliberation continued:

Almond, Karvonen Savannah, GA March 2006


Panel recommendation l.jpg

Add Scribe to MPS Accommodations Table (1999)

Conditions

Must be a trained scribe

Student’s response must be transcribed verbatim. The role of the scribe is recorder.

Translating, interpreting, and clarifying student’s intention not permitted.

Panel Recommendation

Almond, Karvonen Savannah, GA March 2006


Associate superintendent response l.jpg
Associate Superintendent Response (1999)

Conditional approval pending:

  • Guidelines for use by scribes

  • Procedures for training scribes

Almond, Karvonen Savannah, GA March 2006


Panel s amended recommendation l.jpg
Panel’s Amended Recommendation (1999)

  • Literal transcription may ask student to repeat but not to explain

  • May read back and make changes when directed by student

  • Training should cover speed/rate, conventions of scribing, and format or layout

  • Training might be conducted through brief video sequence

Almond, Karvonen Savannah, GA March 2006


Wording as it appeared on the 2001 mps accommodations table l.jpg

Accommodation: (1999)

Student requiring a scribe dictates mathematics problem solving.

Footnote:

A scribe needs to provide a literal transcription of the students response without inserting any information not presented directly by the student. The following three clarifications . . . .

Wording as it appeared on the 2001 MPS accommodations table.

Almond, Karvonen Savannah, GA March 2006


Scribe accommodation footnote continued l.jpg
Scribe accommodation footnote continued (1999)

. . . should be followed and assume that the student has the scoring guide or knowledge of it. The purpose of these clarifications is to ensure the response is that of the student not the scribe.

1. When the scribe does not understand (e.g., misses or tracks) what the student has said, s/he may ask a clarifying question such as “could you repeat that?” Scribes may not ask any other questions.

2. Scribes may not ask questions to clarify information when they know what the student has said but do not understand what it means or why it was said.

3. If asked by the student, the scribe is allowed to read what the student has said. They are then allowed to make changes, insertions, or deletions that are initiated by the student.

Almond, Karvonen Savannah, GA March 2006


Scribe accommodation footnote continued72 l.jpg
Scribe accommodation footnote continued (1999)

At the end of the session the scribe should ask as a final question “Did I accurately record what you have said?” Students should review (look at) what they have had scribed to edit the total response.

Both the student and the scribe need to sign off at the bottom of the test protocol to reflect agreement that all recorded information is correct. Scribes need to be familiar with the needs of the student (and have worked with students and scribing before) and not be scribing for the first time with a student during the test.

Training of scribes needs to address three important issues to ensure accurate recording: (a) speed or rate of recording, (b) conventions for (en)coding symbols, and (c) format and layout of the response. Practice sessions should be held with students and scribes in advance of actual use so the scribe understands students’ articulation. During the administration of the test, school districts, students and/or parents have the option to request an audio or videotape of the session.

Almond, Karvonen Savannah, GA March 2006


Slide73 l.jpg

Example (1999)

A Nomination

For session participants to review and deliberate.

-----Original Message-----From: TiffanieSent: April 02To: PatSubject: Accomodations Panel

. . . the student’s math problem solving test has been marked “modified” because he received assistance with spelling and grammar in his writing. The IEP team requests . . . consider whether this modification may be considered an accommodation. The student performed all math-related tasks individually on the test.

Almond, Karvonen Savannah, GA March 2006


Alex s tiles grade 5 statistics and probability l.jpg

Example (1999)

Alex’s TilesGrade 5 – Statistics and Probability

Alex had a bag filled with colored tiles. It contained: 4 red tiles, half as many green tiles as red tiles, 8 purple tiles, twice as many blue tiles as purple tiles, and 3 times as many yellow tiles as green tiles. If Alex reached into his bag and pulled out 1 tile without looking, what is the probability that he will pull out a yellow tile?

Almond, Karvonen Savannah, GA March 2006


Mathematics problem solving student directions l.jpg

Please, show your answer and all of your work (first efforts and those that worked) in the spaces provided, so everything you do is clear to the readers.

You may use manipulatives or a calculator to work on your problem.

1. CONCEPTUAL UNDERSTANDING - the “what”

2. PROCESSES & STRATEGIES - the “how”

3. VERIFICATION - the “defense”

4. COMMUNICATION - the “connecting path”

5. ACCURACY - “correctness”

Example

Mathematics Problem SolvingStudent Directions

To receive the highest score in each of the five areas, you will want to be certain your work SHOWS each of these parts of a successful solution.

Almond, Karvonen Savannah, GA March 2006


Student response part 1 l.jpg

Example and those that worked) in the spaces provided, so everything you do is clear to the readers.

Student response part 1

Almond, Karvonen Savannah, GA March 2006


Student response part 2 l.jpg

Example and those that worked) in the spaces provided, so everything you do is clear to the readers.

Student response part 2

Almond, Karvonen Savannah, GA March 2006


Proposal form to add adaptation to accommodation table l.jpg

Example and those that worked) in the spaces provided, so everything you do is clear to the readers.

Proposal Form--To add Adaptation to Accommodation Table

Almond, Karvonen Savannah, GA March 2006


Review of relevant research l.jpg
Review of relevant research and those that worked) in the spaces provided, so everything you do is clear to the readers.

Almond, Karvonen Savannah, GA March 2006


Break into groups for discussion review criteria l.jpg

Example and those that worked) in the spaces provided, so everything you do is clear to the readers.

Break into Groups for DiscussionReview Criteria

  • Standards

  • Purpose

  • Function

  • Consequence

  • Social Consequence

  • Systems Change

Almond, Karvonen Savannah, GA March 2006


Recommendations for mps spelling and grammar assistance l.jpg
Recommendations for MPS Spelling and Grammar Assistance and those that worked) in the spaces provided, so everything you do is clear to the readers.

Session Participant Recommendations

  • -

  • -

  • -

Almond, Karvonen Savannah, GA March 2006


The panel recommendations l.jpg

ACC: Assistance with spelling (needs to be discrete) ◊ and those that worked) in the spaces provided, so everything you do is clear to the readers.

◊ Provide assistance one-on-one without disturbing or cuing other students. (Spelling is NOT a part of the assessment.)

MOD: Assistance with grammar.

The Panel Recommendations

Almond, Karvonen Savannah, GA March 2006


Overall the panel deliberations and decisions focused on l.jpg
Overall the Panel Deliberations and Decisions focused on: and those that worked) in the spaces provided, so everything you do is clear to the readers.

  • Affects on Individual Student

  • Maintaining the same content and difficulty for tests and items

  • Removing construct irrelevant variance

  • Instructing in “how to” through wording and footnotes in the tables

Almond, Karvonen Savannah, GA March 2006


Slide84 l.jpg

School and District Accountability and those that worked) in the spaces provided, so everything you do is clear to the readers.

Schools are effective or in need of improvement

High stakes rewards and sanctions will cause improved achievement

Individual Student Proficiency

Promotion to next grade

Graduation from high school

Certificate of Mastery

The Accommodations Panel tended to make their recommendations by considering how test results would affect the student.

Almond, Karvonen Savannah, GA March 2006


Conclusion one l.jpg
Conclusion One and those that worked) in the spaces provided, so everything you do is clear to the readers.

The state had tables listing accommodations and an accommodations panel in place before the legal action, however, the process became more structured and assertive about increasing the number of allowable accommodations and more carefully prescriptive about how-to administer testing accommodations.

Almond, Karvonen Savannah, GA March 2006


Conclusion two l.jpg
Conclusion Two and those that worked) in the spaces provided, so everything you do is clear to the readers.

Classifying adaptations as modifications triggered a demand for an alternate route or appeals process when decisions about certifying individual students were the stakes from the assessment results.

Almond, Karvonen Savannah, GA March 2006


Conclusion three l.jpg
Conclusion Three and those that worked) in the spaces provided, so everything you do is clear to the readers.

A State, as the test sponsor, carries the major weight for establishing policy and procedures around test accommodations and is dependent on research that informs judgments about appropriate and allowable accommodations.

Almond, Karvonen Savannah, GA March 2006


Conclusion four l.jpg
Conclusion Four and those that worked) in the spaces provided, so everything you do is clear to the readers.

Describing accommodations with both “is” and “is not” examples, that is, ways in which an accommodation can be provided that will and will not invalidate the assessment results is useful in communicating appropriate and allowable uses of testing accommodations.

Almond, Karvonen Savannah, GA March 2006


Is and is not examples help communicate to proctors and testing coordinators l.jpg

Accommodation Table (2002) and those that worked) in the spaces provided, so everything you do is clear to the readers.

“Read Mathematics and science, (not reading/literature), items and response choices aloud to the student.”

Modifications Table (2002)

“Read the reading/literature passages, items, and distractors aloud to the student.”

“Is” and “Is not” Examples help communicate to proctors and testing coordinators

Almond, Karvonen Savannah, GA March 2006


We need judges because the laws are rules that don t fit the exceptions l.jpg

we need judges because the laws are rules that don’t fit the exceptions

Grant Wiggins’ paraphrase of Aristotle

Almond, Karvonen Savannah, GA March 2006


Aristotle l.jpg

Law should be the final sovereign; and personal rule, whether it be exercised by a single person or a body of persons, should be sovereign in only those matters which law is unable, owing to the difficulty of framing general rules for all contingencies, to make an exact pronouncement.

- Aristotle

Almond, Karvonen Savannah, GA March 2006


ad