overview of the padi design system n.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
Overview of the PADI Design System PowerPoint Presentation
Download Presentation
Overview of the PADI Design System

Loading in 2 Seconds...

play fullscreen
1 / 18

Overview of the PADI Design System - PowerPoint PPT Presentation


  • 96 Views
  • Uploaded on

AERA April 2005. Overview of the PADI Design System. Robert Mislevy, University of Maryland Geneva Haertel, SRI International Robert Murphy, SRI International. 2. The Problem. Creating good science inquiry tasks is hard. How to do it both soundly and efficiently?

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Overview of the PADI Design System' - teo


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
overview of the padi design system

AERA April 2005

Overview of the PADI Design System

Robert Mislevy, University of Maryland

Geneva Haertel, SRI International

Robert Murphy, SRI International

the problem

2

The Problem
  • Creating good science inquiry tasks is hard.
    • How to do it both soundly and efficiently?
    • What’s the link with science standards?
    • What’s the psychological perspective represented?
    • How to design assessments for different content areas & ages of students?
    • How do you score it?
    • How to sort out confounded evidence?
    • How to leverage the use of technology?
the padi solution 1

3

The PADI Solution (1)
  • Conceptual frameworks and representational (“epistemic”) forms…
    • Design patterns for assessing inquiry
    • Task-design templates
    • Assessment delivery system
  • Object model for task design
  • Software tools to aid the design process
  • Optional scoring and calibration engine to sort out evidence
the padi solution 2

4

The PADI Solution (2)
  • Libraries of exemplars…
    • Design Patterns for assessing inquiry
    • Task templates and pointers to tasks
  • Worked-through applications
    • FOSS, BioKIDS; GLOBE and others
  • Evaluation studies
    • FOSS, BioKIDS
project objectives

5

Project Objectives

To produce a conceptual framework and collection of design tools and resources for supporting the design of assessments of science inquiry.

  • Formulate a design framework for science inquiry that is extensible, shareable, and practical
  • Lay out the framework in a open-system object model
  • Develop supporting software to create and work with design patterns and task templates
  • Develop an optional scoring and calibration engines for the sorting out of evidence
  • Provide an initial set of high quality exemplars to highlight the elements of a coherent assessment argument.
  • Test out the utility of the framework, object model, and tools with applications (GLOBE, FOSS, BioKids etc.)
scope of padi project

6

Scope of PADI project
  • Not an authoring system. Conceptual framework and object model provide infrastructure around which tailored authoring systems can be built.
  • Not a delivery system. Shared conceptions, representations, and object definitions enhance the efficiency of delivery system design by providing a common infrastructure (i.e. delivery system should be consistent with principles of ECD).
  • Does not set out to define inquiry based on a single perspective. Structures for expressing assessment arguments (Design Patterns) and instantiating them in tasks (Task Templates) independent of the psychological perspective.
main components of padi

Design

Evidence-

Models

centered design

(Conceptual)

"Scoring

engine"

Delivery

System

Architecture

Calibration

RunTime

Design

Tools

(Specific

incarnation)

Design

Three-tiered

Object

Use Cases

Software

architecture

Model

7

Main Components of PADI

PADI

Science Inquiry

Other

examples

Science

Cog Psych

PADI

Libraries

Standards

Literature

Applications

Exemplars

Design Patterns

Templates

Globe

BioKIDS

FOSS

Conceptual

Conceptual

models

models

Conceptual

Application

models

Application

Design

Design

Application

Design

Implementation

Implementation

Implementation

Evaluation study

evidence centered design

8

Evidence Centered Design
  • The assessment problem: To draw inferences about student proficiencies based on limited observations.
  • Evidence-Centered Design: Defines elements of the assessment argument and relationships between them.
    • What does it mean to know and do inquiry?
    • What constitutes evidence of knowing?
    • How can that evidence be elicited from students?
    • What are the appropriate statistical techniques for making valid student inferences about what students know from what students do?
  • PADI is implementing a special case of ECD to the study of science inquiry.
evidence centered design 1

Design

Evidence-

Models

centered design

(Conceptual)

Delivery

"Scoring

System

engine"

Architecture

Calibration

RunTime

9

Evidence-Centered Design (1)
assessment design models 2

Design

Evidence-

Models

centered design

(Conceptual)

"Scoring

Evidence Models

Task Models

Student Model(s)

engine"

Delivery

Evidence

Stat

System

Rules

model

Architecture

Feature

s

1.

xxxxx

Calibration

RunTime

2.

xxxxx

xxxxx

3.

10

Assessment Design Models (2)
assessment design models 3

The Four-Process Delivery System

Design

Evidence-

Models

centered design

(Conceptual)

Presentation

ActivitySelection

"Scoring

Process

Process

engine"

Administrator

Examinee

Delivery

Task/

Evidence

System

Composite

Architecture

Library

Calibration

RunTime

Task Level

Evidence

Feedback

Summary Feedback

Evidence

Accumulation

Identification

Process

Process

(Summary

(Response

Scoring)

11

Processing)

Assessment Design Models (3)
assessment design models 3a

The Four-Process Delivery System

Design

Evidence-

Models

centered design

(Conceptual)

Presentation

ActivitySelection

"Scoring

Process

Process

Administrator

engine"

Examinee

Delivery

Task/

Evidence

System

"Scoring

Composite

Architecture

Library

engine"

Calibration

RunTime

Task Level

Calibration

RunTime

Feedback

Evidence

Summary Feedback

Evidence

Accumulation

Identification

Process

Process

(Summary

(Response

12

Scoring)

Processing)

Assessment Design Models (3a)
paradigms for assessing science inquiry

Science

Inquiry

Other

examples

Science

Cog Psych

PADI

Standards

Literature

Applications

13

Paradigms for Assessing Science Inquiry
software support architecture for designing assessment objects

Design

Tools

(Specific

incarnation)

Design

Three-tiered

Object

Use Cases

Software

architecture

Model

14

Software Support & Architecture for Designing Assessment Objects
slide15

PADI Applications

PADI

Applications

Globe

BioKIDS

FOSS

Conceptual

Conceptual

models

models

Conceptual

Application

Application

models

Design

Design

Application

Design

Implementation

Implementation

Implementation

15

evaluation studies

BioKIDS

FOSS

Conceptual

Conceptual

models

models

Application

Application

Design

Design

Implementation

Implementation

Evaluation

study

16

Evaluation Studies
libraries of assessment objects

Evidence-

centered design

Science Inquiry

PADI

Libraries

Design

Tools

Applications

(Specific

incarnation)

Design Patterns

Exemplars

Templates

17

Libraries of Assessment Objects
slide18

The PADI Solution

  • PADI focus is on ...
    • Design framework (Design patterns, templates, delivery system)
    • Assessing inquiry in science
  • PADI focus is not on ...
    • Tasks, per se (although will be examples, through FOSS, BioKIDS, other exemplars)
    • Authoring systems
    • Delivery systems
      • (exceptions: scoring and calibration engine, GradeBook)