Download
1 / 77
770 likes | 906 Views
Monitoring Access Tools and Accommodations in a New Assessment Context Laurene Christensen, Ph.D. National Conference on Student Assessment. What is NCEO?. Federally-funded technical assistance center Support SEAs and consortia on issues related to inclusive assessment
E N D
Monitoring Access Tools and Accommodations in a New Assessment ContextLaurene Christensen, Ph.D.National Conference on Student Assessment
What is NCEO? • Federally-funded technical assistance center • Support SEAs and consortia on issues related to inclusive assessment • Students with disabilities • ELLs • ELLs with disabilities • Website: nceo.info
Changing Assessment Context Smarter Balanced Assessment Common Core State Standards ASSETS PARCC Assessment DLM Assessment ELPA21 NCSC Assessment College and Career Readiness
From Accommodations to Accessibility Tools and Accommodations Accommodations are adjustments to the testing situation, test format, or test administration that provide equitable access during assessments for students with disabilities and students who are English learners.
Accommodations on DLM • Dynamic Alternate Assessment System • Universal design • Variety of response formats • Compatible with variety of assistive technologies • Allows for varying levels of teacher assistance
Accommodations on NCSC • Format of assessment (AA-AAS): • 4 complexity levels; 30 items per subject; can be delivered online • Most items are multiple choice • Some constructed response; can use picture symbols or graphic organizers • Most accommodations are embedded within items • Allowed accommodations are included in test administrators’ scripts
Accommodations on ELPA21 • Still being decided • Administrative Considerations • Universal Features • Designated Features • Accommodations
Accommodations on ASSETS • ELP assessments • Computer delivered • Accommodations and accessibility features built-in • Designed to be appropriate for a range of needs
First steps: The Need for Monitoring • Monitoring requirements of IDEA • Monitoring requirements from ESEA • Expectations from peer review (uncertain right now)
What is monitoring accommodations? • Monitoring accommodationsis a process of evaluating policies, procedures, and practices in order to ensure that a state’s assessment system is inclusive of students with disabilities and English language learners.
Monitoring is about compliance • The State has analyzed the use of specific accommodations for different groups of students with disabilities and has provided training to support sound decisions by IEP teams. • The State routinely monitors the extent to which test accommodations are consistent with those provided during instruction. But it is about more than compliance….
Monitoring is about improving outcomes • Systematic attention to the provision of accommodations can ensure that students are able to show what they know and can do
Monitoring Accommodations: Overview of Tool • Developed in collaboration with CCSSO • Reviewed extensively by ASES SCASS group • Based on a review of publicly available materials • 5 Step process • Each step includes examples, a checklist, additional resources • Appendices include additional examples from states • Extensive glossary at the end
5 Steps to Monitoring Accommodations • Know the rules and regulations for accommodations monitoring • Document decisions about access tools and accommodations • Document the use of access tools and accommodations • Review access tool and accommodations decisions and use • Evaluate and report on access tools and accommodations
Document Decisions about Accessibility Tools and Accommodations • Keeping track of training • Keeping track of decisions • General ed students • ELLs • Students with disabilities • Are access tools and accommodations used for instruction consistent with those used for assessments? • Keeping track of requests for unique accommodations
Document the use of accessibility tools and accommodations • Documenting on the PNP/ISAAP • Documenting inappropriate use of tools and accommodations on test day • Documenting that students received tools and accommodations on test day
Review accommodations decisions and use • Direct observation on test day • Online record reviews • On-site visits on a day other than test day • Interviews with students, teachers, and administrators about the use of accommodations
Evaluate and Report on Accommodations • Analyze accommodations • Report on accommodations • Revise accommodations policies
Questions to Consider • How can we support educators in keeping up with the different policies of each assessment? • What additional support is needed to effectively implement the PNP? • With all of the data these new systems can generate, what are the questions we should ask of the data?
Other Questions? Thank you! Laurene Christensen laurene@umn.edu
Beyond the PNP:Designing a Comprehensive State System for Systematically Monitoring Accommodations Melissa Gholson, Coordinator Office of Assessment and Accountability
Our System English Language Proficiency Standards Linked • Common Core Essential • Elements
2014 and Beyond • Stakeholders meeting • Reviewing current policies, processes and procedures • Develop short and long term goals • Supporting an ongoing research plan • Focus on improving our system
Improving Accommodations • Ensure that your state has a valid method for gathering data on which students are to receive specific accommodations in assessment, and a form to document what they receive on test day. (Electronic WVS.326 Process) • Document how your state analyzes its accommodations data, including timelines of analysis. (State & District Reports) • Establish a specific monitoring procedure that identifies issues in the selection of accommodationsfor individual students or the provision of accommodations for instruction or assessment. (Participation Guidelines) • Include information on any consequences that result from any irregularities in the administration of assessment accommodations. (Report analysis, investigation team and invalidation procedures) • Conduct studies that examine the link between instructionalaccommodations and assessment accommodations(Research agenda)
Accommodations Research • Using last 3 years of W2 data • Will be able to look for trends • The primary purpose of this report is to describe the accommodations assigned to SWD, LEP, and 504 students for the 2011 – 2013 school years. • Discussion/Interpretation: How appropriate are the accommodation assignments based upon student subgroup/performance? • How do these trends vary across time? • Report may be able to - Describe: Relationship of WT2 to Accommodation Assignments • Does WT2 performance predict accommodation assignments? 2010 assessment and 2011 accommodations • Do accommodation assignments influence WT2 performance? 2010 accommodations and 2011 assessment • Examine 2013-2014 online W2 data independently • After 2-3 years may be able to do DIF • Continue analyzing ELL data by ELP proficiency, achievement, by years in program, include ELL who have met exit criteria for two years.
Peer Review Requirements • Providing an appropriate variety of assessment accommodations • Ensuring that the use of assessment accommodations yields meaningful scores • Ensuring that the use of accommodations is consistent with instructional approaches, as determined by a student’s plan • Determining that assessment scores for students when administrated under accommodated conditions, allow for valid inferences about the students’ knowledge and skills • Establishing clear guidelines for including all students in assessment • Differentiate between accommodation policies for students with disabilities and English Language Learners.
Participation Guidelines • The document includes references to current state policies and procedures. • The document is updated and revised annually by a stakeholder group
Accommodations Electronic Application • Accommodations application supports the process for identifying students who need accommodations • Accommodations are updated on a daily delta and are available at the school, district and state level. • The accommodations application allows users to filter and sort to by school, student ID, group, name, grade or accommodation. • The application allows for printing and the creation of spreadsheets
Verification Process • Accommodations are verified locally prior to assessment. • WVEIS Accommodations App are compared to current student plans • The WVS. 326 is reviewed to prepare the testing environment for the student.
The WVS.326 Process:Accommodations Provisionand Monitoring • The WVS.326 form is provided for all state and district assessments. • One form for each student receiving identified supports and accommodations. • Electronic process which feeds accommodations research information to support better student outcomes. • Accommodations data collected informs accommodations policies, practice and research agendas. • The WVS.326 process provides feedback on accommodations provision for student plans. • Documentation of the WVS. 326 form is available http://wvde.state.wv.us/oaa/videos/WV_326/Documentation%20Procedures.zip
WVS.326 The WVS.326 form includes all the accommodations All pre-slugging and data entry one side Directions are on the back Coding process includesdata for accommodation refusals and is used for invalidations of over and under accommodations. Information about WVS. 326 process is included in the Participation Guidelines.
Let’s Talk About Codes If you mark Provided “Yes”- this means the student was given the identified accommodation. If you mark “No”- this means the student was not provided the identified accommodation. This is an accommodations breach UNLESS it is a: • Code 1-the student refused; or • Code 2-the accommodation is not allowed/applicable for this test If “no” is not marked with a code 1 or 2, then it constitutes an accommodations error that requires documentation. See the PG for specific details.
What to do about students who do not receive accommodations? All incidents for failing to provide an accommodation requires written documentation. • The CTC will review all accommodation breaches with principals/building level coordinators and report any findings to the Special Education, Title III and/or 504 director(s) and to the Office of Assessment and Accountability. • If a student has been denied accommodations listed within the IEP, 504 or LEP plan, there has been a breach in the integrity/accuracy of test results. Therefore, the CTC or county special education director should contact the principal who must inform the parent/guardian of the testing administration breach and provide the following options: • Test is invalidated and the student is retested using a breach form (if the testing window is still open). • Test is scored. • Test is invalidated and student receives a score of ―0. If the second or third option is selected—there must bea signed written agreement between the parent/guardian and principal/building level coordinator. Copies of the agreement must be kept on file with the CTC/county special education director, Section 504 director, or Title III director.
WVS 326 State and District Accommodations Report • Analyzed provision data for 2012 WESTEST 2 by content area • 7 reports were generated for each content area to summarize (1) provision rate, (2) refusal rate, (3) not allowable rate, and (3) number of students over-accommodated • State, district, school-level reports • Intended uses of these data • Set specific targets for provision rates • Identify common errors to inform your training • Eliminate under- and over-accommodation
