New England Common Assessment Program
This presentation is the property of its rightful owner.
Sponsored Links
1 / 24

New England Common Assessment Program PowerPoint PPT Presentation


  • 93 Views
  • Uploaded on
  • Presentation posted in: General

New England Common Assessment Program. Grades 3-8 & 11 Bias/Sensitivity Review- Part 2 March 26 and 27, 2006 Quechee, VT. What Is Item Bias?.

Download Presentation

New England Common Assessment Program

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


New england common assessment program

New England Common Assessment Program

Grades 3-8 & 11

Bias/Sensitivity Review- Part 2

March 26 and 27, 2006

Quechee, VT


New england common assessment program

What Is Item Bias?

  • Bias is the presence of some characteristic of an assessment item that results in the differential performance of two individuals of the same ability but from different student subgroups

  • Bias is not the same thing as stereotyping (but we don’t want stereotypes in NECAP either)

  • We need to ensure that ALL students have an equal opportunity to demonstrate their knowledge and skills


New england common assessment program

The “Big Idea” of Bias-Sensitivity Review

NECAP should measure the results of learning opportunities NOT life experiences


New england common assessment program

How Do We Prevent Item Bias?

  • Reading Passages Selected by Item Developers

  • Bias-Sensitivity Review of Reading Passages – Focus on Sensitivity Issues

  • Reading and Math Items are Developed

  • Bias-Sensitivity Review of Reading & Math Items – Focus on Item Bias

  • Item Review of Reading and Math Items

  • Field-Testing Feedback

  • Pilot-Testing Data Analysis (DIF)


New england common assessment program

Role of the Bias-Sensitivity Review Committee

  • Review reading passages, and then both math and reading items

  • Complete independent ratings of each passage or item and record comments

  • Be prepared to discuss “Mixed Reviews” and “Problem Items” with your review team; Finalize independent ratings

Please Note: The Bias-Sensitivity Review Committee plays an advisory role. Final decisions on which passages and items to include in NECAP will be made by the NH, RI and VT DOEs


New england common assessment program

Role of the Bias-Sensitivity Review Committee

The Bias-Sensitivity Review Committee DOES need to make recommendations that will help ensure…

  • Sensitivity to different cultures, religions, ethnic and socio-economic groups, and disabilities

  • Balanced gender roles that are free of gender stereotypes

  • Use of positive language, situations and images

  • Use of context, situations, images and examples that are universally understood across groups and cultures

  • Overall, that the test is free of items and text that may elicit strong emotions in specific groups of students, and as a result, prevent those groups of students from accurately demonstrating their skills and knowledge

  • Overall, that the test is free of items that measure life experiences instead of learning experiences


New england common assessment program

Role of the Bias-Sensitivity Review Committee

  • Reading Level

  • Grade Level Appropriateness

  • GE Alignment

  • Instructional Relevance

  • Language Structure and Complexity

  • Accessibility

  • Overall Item Design

The Bias-Sensitivity Review Committee DOESNOT need to make recommendations concerning…

The Item Review Committees Are Addressing These Issues and Factors


New england common assessment program

The Bias-Sensitivity Review Process

  • December 12th and 13th – Review All Reading Passages

  • March 26th and 27th – Review All Reading and Mathematics Test Items

    Process:

  • Today – (1) Training, (2) Practice, (3) Clarification of Guidelines and Procedures, (4) Break Into Two Groups for Independent Review of Item Sets – Group 1 will review reading and math for grades 3, 5 and 7 and math for grade 11, and Group 2 will review reading and math for grades 4, 6 and 8 and reading/writing for grade 11.

  • Tomorrow – (1) Break into Small Groups for Re-Evaluation and Discussion of “Problem Items” (2) Finalize Independent Ratings, (3) Evaluate the Review Process and Provide Feedback


New england common assessment program

General Guidelines

  • Avoid issues and topics considered sensitive by the general public

  • Keep in mind that some topics that are appropriate for classroom discussion may not be appropriate for assessment

  • Consider context and setting; Bias-Sensitivity Review is not a “literal” process

  • Think of “bias” as an assessment issue as well as a cultural issue, that is, a specific topic or feature of a passage or item that may disadvantage one of two otherwise equally matched groups of students


New england common assessment program

Abortion

Birth Control

Child Abuse/Neglect

Creationism

Divorce

Incest

Occult/Witchcraft

Questioning Parental Authority

Rape

Religious Doctrine

Sex/Sexuality

Sexual Orientation

Sexually Transmitted Diseases

Suicide

Weight

Topics to Avoid in Reading Passages & Test Items


New england common assessment program

Death

Drugs, Alcohol & Tobacco

Family Issues

Guns/ Gun Control

Handicapping Conditions

Holidays

Homelessness

Misuse of Animals/ Animal Rights

Murder

Pregnancy

Racism, Sexism & Ageism

Religion (Except in Historical Context)

Violence

Topics Which May or May Not Be Acceptable


New england common assessment program

Awareness Issues

  • Avoid economic, regional, cultural or gender bias in questions, prompts, identified situations, graphics or reading selections

  • Avoid profanity unless it has a clear literary or historical context

  • Keep in mind that the ability to answer a test question should reflect learning opportunities not cultural opportunities or life experience

  • Remember that we are proud of our states and region and expect to see that pride reflected in assessment passages and items


New england common assessment program

Item Review Rating Form

New England Common Assessment Program

Bias & Sensitivity Review Committee

Item Review Evaluation Form

Read the statement below and select the rating that best describes your evaluation of the

item for potential bias and sensitivity concerns.

“This item does not raise bias and/or sensitivity concerns that would interfere with the performance of a group of students.”

Comments

Please provide specific comments to explain your evaluation. Your comments should address your particular bias/sensitivity concerns, the subgroup(s) that you feel may be impacted, and the extent of the potential impact within the subgroup(s).


New england common assessment program

Bias-Sensitivity Examples and Non-Examples

Once, I got my head stuck in our porch railing. If that man with a cell phone hadn’t stopped and called the fire department, I’d probably still be there. Cell phones can save lives.

Everybody at school has a cell phone. Once, during an English test, somebody actually got a call and in a regular speaking voice, said, “Sorry, I can’t talk now because I’m taking an English test.”

“This passage does not raise bias and/or sensitivity concerns that would interfere with the performance of a group of students”

Rating Descriptions

4 Strongly Agree

3 Agree

2 Disagree

1 Strongly Disagree

Rating Descriptions

4 Strongly Agree

3 Agree

2 Disagree

1 Strongly Disagree


New england common assessment program

What’s the Point?

“Avoid ECONOMIC, regional, CULTURAL or gender bias in questions, prompts, identified situations…”

Okay because the writer’s

culture and economic status

are not factors in

the passage

Not okay because the passage

assumes a cultural and

economic status that won’t

be shared by all test-takers


New england common assessment program

Bias-Sensitivity Examples and Non-Examples

“This passage does not raise bias and/or sensitivity concerns that would interfere with the performance of a group of students”

Larry buys an iPod ® that has a regular price of $120. The iPod ® is on sale for 30% off. What is the sale price of the iPod ®?

 A. $150

 B. $84

 C. $36

 D. $90

Rating Descriptions

4 Strongly Agree

3 Agree

2 Disagree

1 Strongly Disagree

Larry buys a sweater that has a regular price of $40. The sweater is on sale for 30% off. What is the sale price of the sweater? Show your work or explain how you know.

Rating Descriptions

4 Strongly Agree

3 Agree

2 Disagree

1 Strongly Disagree


New england common assessment program

What’s the Point?

“Avoideconomic, regional, CULTURAL or gender bias in questions, prompts, identified situations…”

“(Use) context, situations, images and examples that are UNIVERSALLY UNDERSTOOD across groups and cultures”

Not okay because many

students can’t afford

luxury items such as

IPods

Okay because clothing is

a necessity and $40 is a

more reasonable

amount


New england common assessment program

Bias-Sensitivity Examples and Non-Examples

“This passage does not raise bias and/or sensitivity concerns that would interfere with the performance of a group of students”

  • Explain how the Model T differed from earlier attempts to build a “horseless carriage.” Use examples from the story.

  • (Related to a reading passage called “Henry’s Horseless Carriage.”)

Rating Descriptions

4 Strongly Agree

3 Agree

2 Disagree

1 Strongly Disagree

  • Explain how the antique cars described in the story are different than modern cars. Provide at least 3 examples.

  • (Related to a reading passage called “Henry’s Horseless Carriage.”)

Rating Descriptions

4 Strongly Agree

3 Agree

2 Disagree

1 Strongly Disagree


New england common assessment program

What’s the Point?

“Avoideconomic, regional, CULTURAL or gender bias in questions, prompts, identified situations…”

“(Use) context, situations, images and examples that are UNIVERSALLY UNDERSTOOD across groups and cultures”

Okay because student is

only required to use

information contained in

the passage.

Not okay because the student

needs both information

from the story AND information

from personal experience


New england common assessment program

Bias-Sensitivity Examples and Non-Examples

“This passage does not raise bias and/or sensitivity concerns that would interfere with the performance of a group of students”

Rating Descriptions

4 Strongly Agree

3 Agree

2 Disagree

1 Strongly Disagree

Rating Descriptions

4 Strongly Agree

3 Agree

2 Disagree

1 Strongly Disagree


New england common assessment program

What’s the Point?

“We need to ensure that ALL students have an equal opportunity to demonstrate their knowledge and skills”

“(Use) context, situations, images and examples that are UNIVERSALLY UNDERSTOOD across groups and cultures”

Okay because the context is

generally understood by

all students.

Not okay because some

students (particularly VT & NH

students) may have experience

using the trail guide formula


New england common assessment program

Bias-Sensitivity Examples and Non-Examples

“This passage does not raise bias and/or sensitivity concerns that would interfere with the performance of a group of students”

  • The word hear belongs in which sentence?

  •  A. People come ____ for vacation.

  •  B. I want to sit _____ by the window.

  •  C. Take cover when you _____ thunder.

  •  D. Please put my books over _____.

Rating Descriptions

?

4 Strongly Agree

3 Agree

2 Disagree

1 Strongly Disagree

  • The word hear belongs in which sentence?

  •  A. People come ____ for vacation.

  •  B. I want to sit _____ by the window.

  •  C. She can _____ the loud thunder.

  •  D. Please put my books over _____.

Rating Descriptions

4 Strongly Agree

?

3 Agree

2 Disagree

1 Strongly Disagree


New england common assessment program

What’s the Point?

“Avoideconomic, regional, CULTURAL or gender bias in questions, prompts, identified situations…”

“(Use) context, situations, images and examples that are UNIVERSALLY UNDERSTOOD across groups and cultures”

Is this NOT okay because

thunder may not be

“universally understood” by

persons who are deaf?

Is this better? Is it

okay because the sentence

refers to another person’s

hearing, not the reader’s


New england common assessment program

The “Big Idea” of Bias-Sensitivity Review

NECAP should measure the results of learning opportunities NOT life experiences


  • Login