1 / 15

Modification Process for AA-MAST Students

Learn about a theoretical and data-based modification process that can help students eligible for the AA-MAST program. Discover the research, strategies, and analyses conducted by the CAAVES Project, funded by the US Department of Education. Find out how modifications can impact reliability, comparability, and proficiency test results.

ishmael-paul
Download Presentation

Modification Process for AA-MAST Students

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. How a Theoretical and Data-based Modification Process Can Help Students Eligible for an AA-MASThe Consortium for Alternate Assessment Validity and Experimental Studies (CAAVES Project) Presented by Ryan J. Kettler on April 15, 2009 at the annual meeting for the National Center for Measurement in Education San Diego, CA The CAAVES project is funded by the US Department of Education, Office of Elementary and Secondary Education

  2. CAAVES Project Research Questions We sought to answer the following questions about item modifications: Will modifications in testing conditions change the reliability of measurement? Will taking the test under modified conditions change the comparability of scores for eligible students? What specific modifications are the most helpful for making the scores of eligible students comparable to the scores of ineligible students? 2

  3. Multi-State Sample

  4. Typical Item in Original Condition

  5. Reading Item in Original Condition

  6. Item Modification Strategies • Universal Design Principles • Research on item answer choices • Cognitive Load Theory • Cognitive lab results

  7. CAAVES Item Modification Strategies For all items: • Removed least effective distractor • Increased white space For many items: • Bolded key vocabulary terms • Simplified language in the directions, stimulus, stem, and answer choices • Reorganization of layout • Added graphic support

  8. Reading Item in Modified Condition

  9. Mathematics Item in Modified Condition

  10. Reliability and Concurrent Validity with a Proficiency Test

  11. Mean Item Difficulty by Group and Condition in Reading * Student abilities were equated using a Rasch model.

  12. Mean Item Difficulty by Group and Condition in Math * Student abilities were equated using a Rasch model. CAAVES Modified Achievement CCSSO 2008

  13. Upcoming Analyses: Difficulty Change on Individual Items Students with Disabilities, Eligible Students without Disabilities, Ineligible

  14. Individual Modification Analysis • Reviewed items based on differential boost • Nine well-modified items were identified • Six poorly-modified items were identified • Patterns were revealed for two modifications • One modification was used in 7 of 9 well-modified items, and only 1 of 6 poorly-modified items • Another modification was used on all 3 poorly-modified reading items, and 0 of 4 well-modified items

  15. Thank you! • Any questions? Please email: r.j.kettler@vanderbilt.edu

More Related