Making Consistent Decisions About Accommodations for English Language Learners – Research Summit – Texas Comprehensive

1. Making Consistent Decisions About Accommodations for English Language Learners – Research Summit – Texas Comprehensive Center @ SEDL Austin, Texas March 16–17, 2009

2. Linguistic Accommodations on Assessments: Linguistic Modification Edynn Sato, Ph.D. Director, Special Populations for AACC WestEd

3. Content and Language What is the relationship? Content Language Content Language Content Language

4. What is Linguistic Modification? • A theory- and research-based approach to clarifying and simplifying the language of the assessment without simplifying the content of the assessment. • Linguistic modification is not simply “good editing” practice and does not result in “easier” test items.

5. What is Linguistic Modification? • Linguistic modification involves a process intended to increase English learners’ access to assessment content and subsequently increase the validity of the assessment results. • Access is the minimization or removal of conditions (i.e., sources of construct-irrelevant variance, such as aspects of presentation/format of test information, aspects of response requirements, sociocultural contexts or references that may disadvantage certain students, etc.) that may interfere with students’ ability to meaningfully engage with content or demonstrate their content knowledge and skills. • Appropriate access does not significantly change the targeted construct • Providing students the opportunity to learn, exposing them to grade-level content, or applying broadly the principles of Universal Design do not necessarily address the particular access needs English learners adequately. • Lack of appropriate access has consistently posed a threat to validity.

6. Why Linguistic Modification? • Relative to other strategies such as portfolios and native language translations, linguistic modification lends itself more readily to standardization and broader application across language groups.

7. Linguistic Modification: Key Points • Addresses the linguistic needs of ELLs • Increases the linguistic accessibility of the test item/task • Does not change the content/construct tested • Does not alter essential grade-level content-related language

8. Assessment of English Learners:General Considerations Critical and interdependent factors include: • Population • Characteristics (diverse) • Access needs (linguistic) • Content • Language skill vs. content skill (how interrelated are the language and content skills) • Construct • Language demands • Proficiency expectations • Context • Assessment vs. instruction • Design/Presentation

9. Linguistic Modification: Considerations • Item context • Item graphics • Item vocabulary/wording • Item sentence structure • Item format …and, the interactions among these dimensions

10. Linguistic Modification: Strategies Excerpt from Sato (2008)

11. Linguistic Modification: Strategies (continued) Excerpt from Sato (2008)

12. Study of Linguistic Modification: Overview • 4,617 seventh and eighth grade students from 13 schools in California • ELs (Spanish), low reading ability non-ELs, high reading ability non-ELs • Matched sets of original and linguistically modified versions of mathematics items (number sense, measurement) • Random assignment of students

13. Study of Linguistic Modification(continued) Preliminary findings: • Created a set of linguistically modified items that did not significantly alter the mathematics constructs assessed • expert judgment, cognitive interviews of EL and non-EL students, analysis of non-EL student performance on linguistically modified item and matched original items, analyses of non-EL student performance on statewide academic achievement tests and on the two versions of items developed for the study

14. Study of Linguistic Modification(continued) Preliminary findings: • Positive effect, though not significant, of linguistic modification of test items on the mean scores of EL students and on the mean scores of non-EL students with low-reading ability

15. Framework for understanding language demands on ELLs Language Demands • A language demand is categorized as either a Linguistic Skillor an Academic Language Function based on whether the demand is fundamental to the development and use of language or is a contextual application of language. From: Sato, Lagunoff, Worth, Bailey, and Butler (2005); Bailey, Butler, and Sato (2007)

16. Language Demands From Sato (2008, June)

17. Language Complexity—Density From Sato (2008, June)

18. Language Complexity—Construction From Sato (2008, June)

19. Types of “Expertise” Needed • Knowledge of language development and language acquisition • Knowledge of English language learners • Knowledge of content (English language arts, mathematics, science, social studies) • Knowledge of (large-scale) assessment • Knowledge of instruction

20. References • Bailey, A. L., Butler, F. A., & Sato E. (2007). Standards-to-standards linkage under Title III: Exploring common language demands in ELD and science standards. Applied Measurement in Education, 20(1),53–78. • Sato, E., Lagunoff, R., Worth, P., Bailey, A. L., & Butler, F. A. (2005). ELD standards linkage and test alignment under Title III: A pilot study of the CELDT and the California ELD and content standards (Final Report to the California Department of Education). San Francisco: WestEd. • Sato, E. (2008). Linguistic modification: Part II — A guide to linguistic modification: Increasing English language learner access to academic content. Washington, DC: LEP Partnership. • Sato, E. (2008, June). Language as Content and Language of Content. What's the Difference? — Defining the Language Construct: Strategies and Implications. Presentation at the annual conference of the Council of Chief State School Officers National Conference on Student Assessment. Orlando, FL.

21. For more information: Edynn Sato, Ph.D. Director, Special Populations Assessment and Accountability Comprehensive Center (AACC) esato@wested.org To learn more about the AACC: www.aacompcenter.org