1 / 34

Creating Innovative Tests: Applying Universal Design to Assessment Practices

Creating Innovative Tests: Applying Universal Design to Assessment Practices. Assessment Colloquium November 30, 2007. Manju Banerjee, Ph.D. Assistant Professor in Residence Special Education. Just imagine --- If there were no tests, no assessment, no accountability as we know it?

neola
Download Presentation

Creating Innovative Tests: Applying Universal Design to Assessment Practices

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Creating Innovative Tests: Applying Universal Design to Assessment Practices Assessment Colloquium November 30, 2007 Manju Banerjee, Ph.D. Assistant Professor in Residence Special Education

  2. Just imagine --- If there were no tests, no assessment, no accountability as we know it? Student perspective: Teacher perspective: Policy maker perspective:

  3. “Opportunities borne of new technologies, desires borne of new understandings of learning ---- a new generation of assessment beckons. To realize the vision, we must reconceive how we think about assessment, from purposes and designs to production and delivery.”(Mislevy, Steinberg, & Almond, 1999, p.6)

  4. Computer-based tests (CBT) are the “next frontier” in high stakes assessment(Thompson, Johnstone, & Thurlow, 2002)

  5. What is the appeal of computer-based tests? Opportunity to create tests that support accessibility needs of diverse test takers -- Universal Design (UD) UD is anchored in the belief that a design that works well for examinees with disabilities, improves usability for all individuals (Center for an Accessible Society, 2006) • What is universal design? (Center for Universal Design, 1997) • What makes a test universally designed? • * Seven Elements of a universally designed test (Thompson, Johnstone, & Thurlow, 2002)

  6. Application of Universal Design to High Stakes Tests Maximum usability Widest range of consumers Without design adaptations Minimize construct irrelevant features Disabilities, ELL, Non-traditional age Built-in from the start Include test taking features EXAMINEE CHOICE • Examinee choice is “flexibility to access and express in • the mode or methods that best suit the individual”(Hall, 2005, p. 2) • (Russell, Goldberg, & O’Conner, 2003)

  7. 1. Objective of Study • Inform product development of high stakes tests • Based on current research on features that support “examinee choice” in high stakes test design Test taking tools On-screen item display tools Access tools • Bridgeman, Lennon, & • Jackenthal, 2002 • Mazzeo & Harvey, 1988 • Pommerich, 2004 • Pommerich & Burden, 2004 • Goldberg & Pedula, 2002 • Peak, 2005 • Lunz & Bergstrom, 1994 • Vispoel et al., 2000 • Mandinach et al., 2005 • Sireci, Li, & Scarpati, 2003 • Tindal & Fuchs, 2000

  8. II. Background Information Features of Examinee Choice • Tindal, 1998 • CTB/McGraw-Hill, 2004 Construct neutral Construct related Test taking tools Access tools On-screen item display

  9. II. Background Information (Cont.) • U D increased accessibility for all examinees • Accessibility is maximized when examinees have choice over features of test design • Research on features of test design fall into three broad categories: • (1)Test taking tools (2)Item Display (3) Access tools • Some features are construct neutral/construct irrelevant; others are construct related (including test accommodations) • Allowing examinees to choose features of test design based on individual preferences needs to be explored for a wide range of features including features that affect test construct • U D suggest a framework but research is still emerging on the application of UD to high stakes CBTs.

  10. III. Methodology and Procedures Research questions

  11. III. Methodology and Procedures (cont.) Research Design, Instrumentation, Pilot Study

  12. III. Methodology and Procedures (contd.) Instrumentation – Test Features

  13. III. Methodology and Procedures (contd.) Instrumentation

  14. III. Methodology and Procedures (cont.) Instrumentation

  15. III. Methodology and Procedures (contd.) Instrumentation- Creating the 1st choice exercise • Given 4x3x7 (features) = 84 combinations • Select a unique group of 4 from 84 combinations

  16. III. Methodology and Procedures (contd.) Data Analysis Research Question 1 Research Question 2 Rank-ordered choice exercise data Voluntary top feature choice exercise data Rank-ordered Logit Regression Multinomial Logit Regression

  17. III. Methodology and Procedures (contd.) Data Analysis – Rank-ordered logit regression

  18. III. Methodology and Procedures (contd.) Data Analysis – Rank-ordered logit regression

  19. III. Methodology and Procedures (contd.) Data Analysis – Multinomial logit regression

  20. IV. Results – Participant demographics

  21. IV. Results - Model 1 *p<0.10, **p<0.05, ***p<0.01

  22. IV. Results - Model 1 (contd.) *p<0.10, **p<0.05, ***p<0.01

  23. IV. Results - Model 2 *p<0.10, **p<0.05, ***p<0.01

  24. IV. Results - Model 2 (contd.) *p<0.10, **p<0.05, ***p<0.01

  25. IV. Results - Model 3 (contd.)

  26. IV. Results - Model 3 (contd.)

  27. IV. Results (contd.) Test of equality of regression coefficients for LD/ADHD status

  28. IV. Results -Voluntary Top Feature Choice Exercise

  29. V. Summary of Results and Discussion

  30. V. Summary of Results and Discussion (contd.)

  31. V. Summary of Results and Discussion (contd.)

  32. V. Implications for Future Research

  33. V. Limitations of Study

  34. ***** END OF PRESENTATION

More Related