Assessment Considerations for English Language Learners Race to the Top Assessment Program

1. Assessment Considerations for English Language Learners Race to the Top Assessment Program Charlene Rivera December 2, 2009

2. Key considerations 1. General design issues of the new assessment system 2. ELLs’ linguistic and sociocultural needs 3. Instructional context 4. Design of accommodated tests 5. State assessment policy

3. 1. General design advice Design the entire assessment system with all students in mind, including ELLs • Recognize that the language minority population is heterogeneous • Consider the entire continuum of ELLs from early stages of English language learning to former ELLs Recognize that the needs of students who are learning English are distinct from those of students with disabilities The Race to the Top Assessment Program is an attempt to allow consortia of states to design comprehensive and inclusive assessment systems – scope is dauntingly challenging!

4. General design advice continued Consider the composition of the consortium Establish priorities for implementation • Modify curriculum to reflect new standards • Provide professional development Scale up assessment system in stages • Establish reasonable timeline for development and implementation in each state • Establish clear expectations and approaches for bridging from current state standards and assessment systems to new system

5. General design advice continued Consider extending the ELA test for students at beginning levels of ELP and who have been in U.S. fewer than 3 years – or using ELP reading and writing assessment as a proxy for ELA until student can meaningfully take the ELA test

6. General design advice continued Establish a reporting system that provides timely, useful information • About individual student achievement, classrooms, districts, etc. • About ELLs ELP proficiency and proficiency in content • About Former ELLs proficiency in content Establish a system for monitoring • progress of ELLs and former ELLs

7. 2. ELLs’ linguistic & sociocultural needs ELL population has increased nearly 60% over past 10 years to over 5 million (NCELA, 2007) Great diversity of languages spoken, cultures, and educational background including: • Migrants from Puerto Rico who are educated in Spanish • Children from U.S. territories • Immigrants from all over the world • Refugees escaping persecution in their home countries • Children born in the U.S. to any of these groups

8. ELLs’ linguistic & sociocultural needs advice Design test specifications with involvement of specialists who are knowledgeable about the academic language demands of content for ELLs at different grade levels Involve stakeholders in each consortium who are knowledgeable about the continuum of needs of second language learners

9. ELLs’ linguistic & sociocultural needs advice continued Establish TACs to Review, provide feedback to designers of assessment system • Needed at consortium as well as SEA/LEA levels • Include stakeholder representation • Include members who can provide input on the extent to which assessment design can address the continuum of ELLs at different levels of ELP Each assessment must be valid and reliable • For specific purposes • For all subcategories of students, including ELLs at different stages of English language acquisition

10. 3. Instructional context advice Assessment is an integral part of instruction, not an add-on Teachers must be part of development teams, considering implications for teaching and learning Teachers should also be involved in providing professional development to their colleagues to support quality instruction that will enable valid assessment of all students, including ELLs

11. Instructional context advice continued Expect that leaders in SEAs and LEAs have introduced the new standards and that teachers feel comfortable teaching the subject matter to be tested Expect that leaders have the same high expectations for ELLs as for all other students Require SEAs/LEAs to conduct professional development on how to teach academic language needed for ELLs to acquire content, especially at middle and high school

12. Instructional context advice continued Require each state in a consortium to conduct professional development for teachers and others responsible for administering assessments – before test administration - about: • The new assessment • Expectations for student performance Require consortium members and test developers to consider how ELP outcomes can/should be used to inform the outcomes of content assessments

13. 4. Design of accommodated tests Accommodations involve changes to testing procedures, materials, or situation to allow students meaningful participation in an assessment Effective accommodations for ELLs address unique linguistic and sociocultural needs of the student without altering the test construct Accommodated scores should be sufficiently equivalent in scale to be pooled with unaccommodated scores

14. Design of accommodaed tests continued ELL-responsive accommodations provide direct or indirect linguistic support • Direct – adjust language of the test, in either English or another language • Indirect – adjust conditions under which test is taken, e.g., extra time to process test items

15. Design of accommodated tests continued Universal design is good – but not always sufficient to make test items accessible to ELLs if second language testing experts are not part of the universal design team Appropriate accommodation for ELLs should, at a minimum: • Be standardized • Be ELL-responsive • Include use of English and other languages, as appropriate

16. Design of accommodated tests continued Consider whether an accommodation is intended to help students at low, moderate, or advanced levels of ELP Research is needed to examine relationship between ELP level and effective use of an accommodation Technology has the potential to standardize delivery of accommodations.

17. Design of accommodated tests continued With native language accommodations it is important to consider whether a student has oral knowledge of the language and literacy skills to take advantage of these What is the student’s level of ELP? How long has the student been in U.S. schools? What is the student’s oral and literacy skills in the second/third language? Has, or is the student receiving instruction in the native language in the content tested?

18. Design of accommodations continued Most native language accommodations are designed for Spanish speakers • Spreading costs across a consortium may allow native language accommodations in multiple languages Need more research about native language accommodations, e.g., use of commercial word-to-word dual language dictionary, customized dual language glossary, pocket word-to-word dual language translator, commercial dual language dictionary

19. Design of accommodated tests continued Native language accommodations - e.g., dual language test booklets, translated and adapted tests – allow students to demonstrate knowledge in either English or the native language, thereby reducing construct-irrelevant variance Recorded audio translation is underutilized • Allows access to test content for students not literate - but with receptive capacity - in their native language • Avoids lack of standardization inherent in sight and oral translations done on the “fly”

20. Design of accommodated tests continued Accommodations to improve to test directions don’t help students access the content of test items Other “accommodations” are administrative supports rather than true accommodations because they do not address students’ linguistic needs

21. Design of accommodated tests continued Studies should track effectiveness of accommodations so that decisions can be made about their use and improvements can be made to assessments

22. Design of accommodated tests continued Need research about implementation of accommodations • Which accommodations do students use? • To what extent do specific accommodations support ELLs at different levels of ELP? Accommodations are not a panacea, but they are a support that should be used • New assessment system should build into its design methods for monitoring use of specific accommodations, assessing utility, feasibility, and quality of accommodations to support ELLs at different levels of ELP

23. 5. State assessment policy Lack of uniform implementation of assessments and accommodations for ELLs is due, in part, to: • Lack of clear state assessment policies • Lack of communication by SEA to districts and schools about how to implement policies Therefore, with the design of a new assessment system, each consortium should develop policy that can be adapted by each participating SEA, including a plan for communicating to LEA and school staff

24. Final Thoughts Hold ELLs to the same high expectations as all other students Consider the needs of ELLs from the beginning of developing assessment system Involve teachers, second language testers and content specialists in development of the assessment system Involve teachers in professional development to address curriculum and roll out of the assessment system

25. Final Thoughts, 2 Make sure that ELLs are instructed in the content tested Make sure accommodations differentiate ELLs at different levels of ELP Consider extending the continuum of English Language Arts assessments to include students at beginner levels of ELP and who have been in U.S. schools fewer than 3 years