Including Students with Learning Disabilities in Accountability Assessments

1. Options for including students with learning disabilities in assessments for school accountability Ken Warlick, University of Kentucky Rachel Quenemoen, University of Minnesota Sue Rigney, U.S. Department of Education

2. Increase awareness of implications of federal assessment and accountability requirements on students with learning disabilities Increase awareness regarding accommodations Increase awareness regarding alternate assessments Increase awareness on emerging trends in assessment Objective

3. “…to ensure that all children have a fair, equal, and significant opportunity to obtain a high-quality education and reach, at a minimum, proficiency on challenging State academic achievement standards and state academic assessments” Purpose of No Child Left Behind

4. Improve results for student with disabilities through improved teaching and learning Raise expectations for students with disabilities Increase access to the general curriculum Provide parents information about their child’s achievement in relationship to the performance of other children in their school Purpose of Assessment Requirements of IDEA

5. Students with disabilities previously exempted from assessment and accountability system Students with disabilities previously received instruction in separate curriculum Change from low to high expectations for students with disabilities State leadership in fostering school and district accountability Issues in both NCLB and IDEA

6. General assessment General assessment with accommodations (or modifications) Alternate assessment on grade level achievement standards Alternate assessment on alternate achievement standards Computer-based assessments Assessment Options

7. Accommodations: The National Picture • Accommodations use is on the rise. • About 50% of students with learning disabilities receive an accommodation during testing. • Evidence from experimental studies indicates that some accommodations boost performance. • Most common accommodations are • Small group administration • Read-aloud • Extended time

8. Definitional Issues Some accommodations are considered to change the construct assessed, and others are viewed as “ok” Okay Not Okay Accommodation Adaptation Standard administration Modification Modification Non-approved accommodation Non-allowed accommodation Non-standard administration Know what terms mean in your state and district!

9. What Makes An Assessment Accommodation OK? • It is aligned with instructional accommodations, but is not an excuse not to teach • Student needs it to demonstrate knowledge and skills – or to participate in assessment • The consequences of using the accommodation have been carefully considered • OK is not defined by what the test publisher says (unless has research evidence)

10. Closing the Achievement Gap Looks Like This:

11. Keep high incidence populations in school and in special education (LD 3 X as likely to drop out of school as non-disabled) LD comprises 50% of special education population LD is a life-long disability. Student’s needs change over time. They need learning and coping strategies but special education does not cure them Can states close achievement gap without paying attention to LD?

12. Research on Accommodations for LD Students – What Does It Tell Us? • Accommodations most frequently studied are oral presentation (13) and extended time (10) • Various research designs are used • Many limitations in conducted research are identified

13. Issues: • Tendency to allow too many accommodations, possibly reducing expectations for student learning. • Poor decision making about accommodations, reflecting lack of knowledge about instructional accommodations. • Use of accommodations as an excuse to exclude students scores on reports or accountability.

14. Promising Practices: • States keeping track of how many students use accommodations, which ones, and their impact on achievement. • Out-of-the-box thinking about the use of accommodated test scores. • Clear decision-making criteria (e.g., alignment to instructional accommodations) and training on how to make decisions. • Teaching high school students (at the latest) about the tests they take and about the accommodations that they need.

15. Must be familiar with the state or district content/learning standards Must be familiar with the curriculum Must be familiar with typical instructional activities in general education Implications for IEP teams

16. Adequate Yearly Progress

17. Aligned with the State’s content standards. Yield results separately in reading/language arts and math. Designed and implemented to support use of the results to determine AYP. Alternate Assessments

18. Clearly defined structure Guidelines for which students may participate Clearly defined scoring criteria and procedures Report format that clearly communicates student performance in terms of the academic achievement standards defined by the State Alternate Assessments should have…

19. Must meet the same requirements for high technical quality that apply to regular assessments under NCLB: Validity Reliability Accessibility Objectivity Consistent with nationally-recognized professional and technical standards. Alternate Assessments

20. Alternate assessment scored against grade-level standards Alternate assessment scored against alternate achievement standards States may use more than one alternate assessment

21. Could only be considered an alternate assessment based on alternate achievement standards if Alternate achievement standards defined through a documented and validated standards-setting process Proficient results included in the 1% cap Out-of-Level Assessments

22. 10 Recommendations: See Myth-Busters Handout Researching Out-of-Level Testing

23. Universally Designed Assesmsents: A Quick Definition Universally designed assessments are built from the beginning and continually refined to be accessible and valid for the greatest number of students!

24. Do we want to change the standard of performance? NO Can we forget about accommodations if we do this? NO Is this all figured out – for now and forever? NO Is this something that will benefit only students with disabilities? NO But, what does that really mean?

25. Measures What it Intends to Measure • Universally designed assessments reflect good measurement qualities: • Minimize skills required beyond those being measured • Reflects the intended content standard (reviewers have information about what is being measured)

26. An Example of Universal Design: Mathematics Tests The reading requirements of a math test often prevent students with marginal reading ability from demonstrating competency in math.

27. Ordering Pizza (Original Item) The cafeteria manager surveyed the students in a middle school to find out if they would buy Brand X pizza on Friday if the manager sold it. She made a circle graph to display the results of her survey. NO YES Based on the results of the survey, answer the following questions: • What fraction of students would buy Brand X pizza on Friday? • What percent of students would buy Brand X pizza on Friday? • There are 1200 students in this school. How many students will buy Brand X pizza on Friday if the manager’s survey is accurate?

28. Ordering Pizza (Revised Item) Maria surveyed the students in her school to find out if they liked pizza on Friday. She made a circle graph to display the results of her survey. NO YES • What fraction of students said “yes”? • What percent of students said “yes”? • There are 1200 students in Maria’s school. How many students said “yes”?

29. What could this item measure? Decode text? Comprehend extended passages? Extent of vocabulary? Understand the moral or point of the fable? Discuss the common elements of any fable? Compare and contrast fables with news reports? Articulate the relationship between the fable and the overall culture? Anything else? (National Center on Accessing the Curriculum, 2003) Suppose a test item requires a student to read an Aesop’s fable

30. Something to think about

31. The “door” we need to go through: What is meant by the construct of ‘reading’ What about ‘literacy?’ The answer(s) to these questions will determine the universal design or accommodations that can be used when certain content standards are taught and assessed!

32. Visual Tactile (feeling print) Auditory (listening to printed messages) Multi-modal (using any combination of the above modalities) Printed text; ASL Text in Braille and Nemeth Codes Listen to taped text Computer-based “assistive” reading/ viewing programs Modes of print interaction: with examples of accommodations

33. Individualized accommodation decisions should be linked to the standard, the construct assessed, the nature of instruction or the assessment, and the student’s characteristics Purposeful reading – reading to select and apply relevant information for a given task Does this allow different modes of print interaction? And, what are the implications of these different modes for accommodations?

34. Issues: • Universal design does not eliminate the need for accommodations. • Universal design requires the systematic application of good test development – more attention needs to be given at the same time that we need to develop more assessments. • Little research exists on universally designed assessments.

35. Promising Practices: • Item developers who know disability issues and their interaction with test formats and procedures. • “Bias” or “sensitivity” review committee with at least one member who knows disability issues. • Field testing that includes students with disabilities who use accommodations during the field test.

36. Does NOT mean adaptive testing DOES mean making use of technology for accommodations DOES mean grappling with very, very important issues What is computer-based assessment

37. Efficient administration Preferred by students Improved writing performance Built-in accommodations Immediate results Efficient item development Increased authenticity Opportunities

38. Use of technology cannot take the place of content mastery Issues of equity and skill in computer use Added challenges for some students Technological challenges Security of online data Lack of expertise in designing accessible Web pages Prohibitive development cost Challenges

39. Web-based, individualized assessment : Students with IEP or 504 Plan that specifies need for "reader" as an instructional and assessment accommodation; Students who require and routinely use text-reader or screen-reader technologies to access printed material in classroom instruction and assessment; Students who have accessed and used the CATS Online Practice Area. Based upon success of pilot studies, 16 districts, 31 Schools, & 204 students participated in “live” CATS Online in the spring of 2003 Example: Kentucky Online Assessment

40. CATS Online

41. Check us out! NCEO Resources Visit: education.umn.edu/nceo or Search for NCEO