Slide1 l.jpg
This presentation is the property of its rightful owner.
Sponsored Links
1 / 36

Validity, Effectiveness and Feasibility of Accommodations for English Language Learners With Disabilities (ELLWD) PowerPoint PPT Presentation


  • 127 Views
  • Uploaded on
  • Presentation posted in: General

Validity, Effectiveness and Feasibility of Accommodations for English Language Learners With Disabilities (ELLWD). Jamal Abedi University of California, Davis/CRESST. Accommodations for ELLWD.

Download Presentation

Validity, Effectiveness and Feasibility of Accommodations for English Language Learners With Disabilities (ELLWD)

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Slide1 l.jpg

Validity, Effectiveness and Feasibility of Accommodations for English Language Learners With Disabilities (ELLWD)

Jamal Abedi

University of California, Davis/CRESST


Accommodations for ellwd l.jpg

Accommodations for ELLWD

The process of provision of accommodations to ELLWD is more complex than the process used for either SD or ELL students.

Care must be taken to choose accommodations that are appropriate for this particular subgroup of students.

ELLWDs need accommodation addressing both their language needs and their disabilities.


Why should english language learners be accommodated l.jpg

Why Should English Language Learners Be Accommodated?

  • Their possible English-language deficiency may interfere with their content knowledge performance.

  • Assessment tools may be culturally and linguistically biased for these students.

  • Linguistic complexity of the assessment tools may be a source of measurement error.

  • Language factors may be a source of construct irrelevant variance.


Why should students with disabilities be accommodated l.jpg

Why Should Students With Disabilities be Accommodated?

  • Their disabilities put them at disadvantage.

  • Accommodations must be provided to level the playing field.


Slide5 l.jpg

Site 2 Grade 7 SAT 9 Subsection Scores


Slide6 l.jpg

Site 4 Grade 8 Descriptive Statistics for the SAT 9 Test Scores by Strands


Slide7 l.jpg

Normal Curve Equivalent Means & Standard Deviations for Students in Grades 10 and 11, Site 3 School District

Reading Science MathMSDMSDMSDGrade 10SWD only16.412.725.513.322.511.7LEP only24.016.432.915.336.816.0LEP & SWD16.311.224.8 9.323.6 9.8Non-LEP/SWD 38.016.042.617.239.616.9All students36.016.941.317.538.517.0Grade 11SWD Only14.913.221.512.324.313.2LEP Only22.516.128.414.445.518.2LEP & SWD15.512.726.120.125.113.0Non-LEP/SWD 38.418.339.618.845.221.1All Students36.219.038.218.944.021.2


Accommodations for ells and swds l.jpg

Accommodations for ELLs and SWDs

  • Can the same accommodations used for students with disabilities be used for ELLs?

  • Can the same accommodations used for ELLs be used for students with disabilities?


How are we doing in practice nationally l.jpg

How Are We Doing in Practice Nationally?

  • Are the states and districts across the nation cognizant of this important principle of using accommodations that are appropriate for a particular subgroup?

  • Are there any objective national criteria to help states to select appropriate accommodations for ELLWD students?

  • Or, is the assignment of accommodations to these students based on temporary and subjective decisions?


Slide10 l.jpg

Accommodation in Stanford 9 Testing Used by a District


Slide11 l.jpg

SY 2000-2001 Accommodations Designated for ELLs Cited in States’ Policies

There are 73 accommodations listed:

N:Not Related

R:Remotely Related

M:Moderately Related

H:Highly Related

From: Rivera (2003) State assessment policies for English language learners. Presented at the 2003 Large-Scale Assessment Conference


Slide12 l.jpg

SY 2000-2001 Accommodations Designated for ELLs Cited in States’ Policies

I. Timing/Scheduling (N = 5)

N1. Test time increased

N2. Breaks provided

N3. Test schedule extended

N4. Subtests flexibly scheduled

N5. Test administered at time of day most beneficial to test taker

N = not related; R = remotely related; M = moderately related; H = highly related


Slide13 l.jpg

SY 2000-2001 Accommodations Designated for ELLs Cited in States’ Policies

II. Setting (N = 17)

N1. Test individually administered

N2. Test administered in small group

N3. Test administered in location with minimal distraction

N4. Test administered in familiar room

N5. Test taker in separate location (or carrel)

N6. Test administered in ESL/Bilingual classroom

N7. Individual administration provided outside school (home, hospital, institution, etc.)

N8. Test taker provided preferential seating

N9. Increased or decreased opportunity for movement provided

N10. Teacher faces test taker

N11. Special/appropriate lighting provided

N12. Adaptive or special furniture provided

N13. Adaptive pencils provided

N14. Adaptive keyboards provided

N15. Person familiar with test taker administers test

N16. ESL/bilingual teacher administers test

N17. Additional one-to-one support provided during test administration in general education classroom (e.g. instructional assistant, special test administrator, LEP staff, etc.)

N = not related; R = remotely related; M = moderately related; H = highly related


Slide14 l.jpg

SY 2000-2001 Accommodations Designated for ELLs Cited in States’ Policies

III. Presentation (N = 32)

R1. Directions repeated in English

R2. Directions read aloud

R3. Audio-taped directions provided in English

N4. Key words or phrases highlighted

M5. Directions simplified

M6. Audio-taped directions provided in native language

M7. Directions translated into native language

N8. Cues provided to help test taker remain on task

M9. Directions explained/clarified in English

M10. Directions explained/clarified in native language

M11. Both oral and written directions in English provided

M12. Both oral and written directions in native language provided

M13. Test items read aloud in English

H14. Test items read aloud in simplified/sheltered English

N15. Audio-taped test items provided in English

H16. Test items read aloud in native language

H17. Audio-taped test items provided in native language

N = not related; R = remotely related; M = moderately related; H = highly related


Slide15 l.jpg

SY 2000-2001 Accommodations Designated for ELLs Cited in States’ Policies

III. Presentation (N = 32)

N18. Assistive listening devices, amplifications, noise buffers, appropriate acoustics provided

N19. Key words and phrases in test highlighted

H20. Words on test clarified (e.g. words defined, explained)

H21. Bilingual word lists, customized dictionaries (word-to-word translations) provided

N22. Enlarged print, magnifying equipment, Braille provided

N23. Memory aids, fact charts, list of formulas and/or research sheets provided

N24. Templates, masks or markers provided

N25. Cues (e.g. arrows and stop signs) provided on answer form

N26. Acetate shield for page provided

N27. Colored stickers or highlighters for visual cues provided

R28. Augmentive communication systems or strategies provided (e.g. letter boards, picture communication devices, voice output systems, electronic devices)

H29. Simplified/sheltered English version of test provided

H30. Side-by-side bilingual versions of test provided

H31. Translated version of the test provided

N32. Test interpreted for the deaf or hearing impaired/use of sign language provided

N = not related; R = remotely related; M = moderately related; H = highly related


Slide16 l.jpg

SY 2000-2001 Accommodations Designated for ELLs Cited in States’ Policies

IV. Response (N = 17)

N1. Test taker marks answers in test booklet

N2. Test administrator transfers test-taker’s answers

N3. Test taker’s transferred responses checked for accurate marking

N4. Copying assistance provided between drafts

N5. Test taker types or uses a machine to respond (e.g. typewriter/word processor/computer)

N6. Test taker indicates answers by pointing or other method

N7. Papers secured to work area with tape/magnets

N8. Mounting systems, slant boards, easels provided to change position of paper, alter test taker’s position

N.9. Physical assistance provided

N10. Enlarged answer sheets provided

R11. Alternative writing systems provided (including portable writing devices, computers and voice-activated technology)

R12. Test taker verifies understanding of directions

R13. Test taker dictates or uses a scribe to respond in English

N14. Test taker responds on audio tape in English

H15. Oral response in native language translated into English

H16. Written response in native language translated into English

H17. Spelling assistance, spelling dictionaries, spell/grammar checker provided

N = not related; R = remotely related; M = moderately related; H = highly related


Slide17 l.jpg

SY 2000-2001 Accommodations Designated for ELLs Cited in States’ Policies

V. Other (N = 2)

N1. Out-of-level testing provided

N2. Special test preparation provided

Preliminary Findings: State Assessment Policies for English Language Learners, SY 2000-2001

GW/CEEE, Large-Scale Assessment Conference 2003

[email protected]

N = not related; R = remotely related; M = moderately related; H = highly related


Slide18 l.jpg

There are 73 Accommodations Listed

47 or 64% are not related

7 or 10% are remotely related

8 or 11% are moderately related

11 or 15% are highly related


The most important issue is the concern over the validity of accommodation strategies l.jpg

The most important issue is the concern over the validity of accommodation strategies:

  • Research findings suggest that providing accommodations may increase performance of ELLs/SDs, while also benefiting non-ELLs/SDs.

  • There is not enough research support for many of the accommodations currently being used in the national and state assessments.

  • The only way to make judgments about the efficiency and validity of accommodations used by states is to use them in experimentally controlled studies with both ELL/SWD and non-ELL/SWD students.


Slide20 l.jpg

Concern over the validity of accommodation strategies:

A research example

Some forms of accommodation strategies, such as the use of a glossary with extra time, raised the performance of both ELL and non-ELL students (Abedi, Hofstetter, Lord, and Baker, 1998, 2000)

  • ELL students’ performance increased by 13% when they were tested under glossary with extra time accommodation.

  • While this looks promising, it does not present the entire picture.

  • Non-ELL students also benefited from this accommodation, with an increase of 16%.

  • English and bilingual dictionary recipients may be advantaged over those without access to dictionaries. This may jeopardize the validity of assessment.


Slide21 l.jpg

Concerns over the validity of accommodation strategies:

A research finding

There are, however, some accommodations that help ELL students with their English language needs without compromising the validity of assessment.

  • Linguistic modification of test items is among these accommodations.

  • This accommodation also helped students with learning disabilities.

  • Thus, an accommodation may have the potential to be effective and valid for both LEP and SD, consequently relevant for LEPWD.


Slide22 l.jpg

Concern over the validity of accommodation strategies

Validity: The goal of accommodations is to level the playing field for ELL/SWD, not to alter the construct under measurement.

Consequently, if an accommodation affects the performance of non-ELL/SWD, the validity of the accommodation could be questionable.

Feasibility: For an accommodation strategy to be useful, its implementation must be possible in large-scale assessments.

Strategies that are expensive, impractical, or logistically complicated are unlikely to be widely accepted.


How validity of accommodations can be examined l.jpg

How validity of accommodations can be examined?

Only through experimentally controlled research where:

  • ELL/SWD and non-ELL/SWD students are randomly assigned to experimental and control groups

  • Both ELL/SWD and non-ELL/SWD students are observed under accommodated and non-accommodated assessments


Slide24 l.jpg

Can existing data (from national and state assessments) be used for examining the effectiveness and validity of accommodations?

  • Effectiveness of accommodations can be examined only if ELL/SWD students are randomly assigned to the accommodated and non-accommodated conditions

  • Validity of accommodations can be examined only if non-ELL/SWD students are randomly assigned to the accommodated and non-accommodated conditions


How the validity of accommodations can be examined l.jpg

How the validity of accommodations can be examined?

  • Using existing data?

  • Through experimentally controlled field study?


Slide26 l.jpg

LEP Students Assessed With and Without Accommodations, 1998 NAEP Writing Assessment: National Sample, Public and Nonpublic Schools Combined


Research findings l.jpg

Research findings

  • Pitoniak, M., Lutkus, A., Cahalan-Laitusis, C., Cook, L. & Abedi, J. (2005). Are Inclusion Policies and Practices for State Assessment Systems and NAEP State Assessments Aligned?

  • Sireci, S. G., Li, S. & Scarpati, S. (2003). The effects of test accommodation on test performance: A review of the literature (Center for Educational Assessment Research Report No. 485). Amherst: University of Massachusetts.

  • Thompson, S., Blount, A., Thurlow, M. (2002). A summary of research on the effects of test accommodations: 1999 through 2001 (Technical Report 34). Minneapolis: University of Minnesota, National Center on Educational Outcomes.

  • Thurlow, M. L., McGrew, S., Tindal, G., Thompson, S. J., Ysseldyke, J. E., & Elliott, J. L. (2000). Assessment accommodations research: Considerations for design and analysis (NCEO Technical Report 26). Minneapolis: University of Minnesota, National Center on Educational Outcomes.

  • Tindal, G., & Fuchs, L. (2000). A summary of research on test changes: An empirical basis for defining accommodations. Lexington, KY: University of Kentucky, Mid-South Regional Resource Center.

  • Abedi, J., Hofstetter, C., & Lord, C. (2004). Assessment accommodations for English language learners: Implications for policy-based empirical research. Review of Educational Research, 74(1), 1-28


How the validity of accommodations can be tested in an experimentally controlled condition l.jpg

How the validity of accommodations can be tested in an experimentally controlled condition?


Conclusions and recommendation l.jpg

Conclusions and Recommendation

Accommodations:

Must be relevant to the subgroups of students

Must be effective in reducing the performance gap between accommodated and non-accommodated students

Must be valid, that is, accommodations should not alter the construct being measured

The results could be combined with the assessments under standard conditions

Must be feasible in the national and state assessments


Slide31 l.jpg

Conclusion

There is not enough research support for many of the accommodations that are currently used in national and state assessments.

The only way to make judgments about the efficiency and validity of these accommodations is to use them in experimentally controlled situations with both ELL/SWD and non-ELL /SWD students and examine their validity and effectiveness under a solid experimental condition.

The results of CRESST studies along with other studies nationwide have provided support for some of the accommodations used for ELL students.


Conclusion cont l.jpg

Conclusion cont.

Examples of research-supported accommodations:

  • Providing a customized dictionary is a viable alternative to providing traditional dictionaries.

  • The linguistic modification of test items that reduce unnecessary linguistic burdens on students is among the accommodations that help ELL students without affecting the validity of assessments.

  • Computer testing with added extra time and glossary was shown to be a very effective, yet valid accommodation (Abedi, Courtney, Leon and Goldberg, 2003)


Slide33 l.jpg

Conclusion cont.

It is thus imperative to examine different forms of accommodations before using them in state and/or national assessments.

Without information on important aspects of accommodations such as validity, it would be extremely difficult to make an informed decision on what accommodation to use and how to report the accommodated and non-accommodated results.


For more information please contact jamal abedi at l.jpg

For more information, please contact Jamal Abedi at:

(530) 754-9150

or

[email protected]


  • Login