1 / 26

The Portal Assessment Design System

eman
Download Presentation

The Portal Assessment Design System

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

    1. ..

    2. Outline Goals of the project Assessment engineering Explicitness, representations, reusability, generativity Opportunities and constraints Evidence-centered design & PADI Assessment arguments Layers PADI structures

    3. Assessment Engineering Is assessment design art or science? Assessment engineering submits there are recurring structures and relationships we can exploit – in terms of language, data structures, representational forms, and processes – to enhance efficiency and validity in assessment/task design.

    4. Assessment Engineering Some historical roots Guttman, Suppes, Hively et al., Obsbourne, Bormuth ~ 60s. Roid & Haladyna (1982): Technology for Test-Item Writing Embretson (1985): Test design: Developments in psychology and psychometrics. National Research Council’s Knowing what students know Evidence-centered design (ECD) Mislevy, Steinberg, & Almond (2003) PADI project (NSF-supported)

    5. Opportunities and Constraints Can ECD ideas be gainfully applied to large-scale state-level assessment? Partnership with Pearson Educational Measurement and the State of Minnesota to try the ideas out in the Minnesota Comprehensive Assessments—Series II (MCA-IIs), in Science Lots of good work & thinking and established program to build on Constraints Must be consistent with Minnesota science content standards, test specifications, delivery environment Emphasis on explicitness, representations, reusability, generativity, … not just “creating great tasks” – even though we might… Jorge Luis Borges’ “Pierre Menard, Author of the Quixote”

    6. Assessment Arguments What complex of knowledge, skills, or other attributes should be assessed? What behaviors or performances should reveal those constructs? What tasks or situations should elicit those behaviors? (Messick, 1994) What tasks or situations should elicit those behaviors? That’s what the task models are about. Now let’s look at each of these models more closely.What tasks or situations should elicit those behaviors? That’s what the task models are about. Now let’s look at each of these models more closely.

    7. From Mislevy & Riconscente, in press

    8. From Mislevy & Riconscente, in press

    9. From Mislevy & Riconscente, in press

    10. From Mislevy & Riconscente, in press

    11. From Mislevy & Riconscente, in press

    12. From Mislevy & Riconscente, in press

    13. From Mislevy & Riconscente, in press

    14. From Mislevy & Riconscente, in press

    16. From Mislevy & Riconscente, in press

    17. From Mislevy & Riconscente, in press

    18. From Mislevy & Riconscente, in press

    21. From Mislevy & Riconscente, in press

    22. From Mislevy & Riconscente, in press

    23. From Mislevy & Riconscente, in press

    24. From Mislevy & Riconscente, in press

    25. From Mislevy & Riconscente, in press

    26. Integrate insights from PADI, PEM, & MN… to aid the design process by capturing / making explicit / making re-usable & regenerative the thinking that goes into tasks: PADI Data structures and software tools Design patterns,templates, design system, Wizards PEM Storyboard process, task ideas, re-usable elements and data structures for presentation, interaction, responding, response evaluation MN Science content standards, large-scale testing system

    27. Too many notes? As Emperor Joseph II said to Mozart. Power in general/abstract representations, Reveals fundamental similaries under assessments that look different on the surface. Right perspective for design of systems, but not necessarily for work within systems. Some people need to work from first principles, but… Scaling up for wide range of users requires exemplars, support tools, tuned interfaces. What’s learned in this project will benefit participants, but the field of assessment more broadly—especially large-scale assessment.

More Related