1 / 15

Using the PADI Framework to Design an Item

Using the PADI Framework to Design an Item. ?. GLOBE Task Template. Student Model. Measurement Model. Evaluation Procedures. Work Product Specs. Presentation Materials & Properties. Templates store item characteristics. Flow of Design. Flow of Inference.

umed
Download Presentation

Using the PADI Framework to Design an Item

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Using the PADI Framework to Design an Item ? PADI Item Design C. Kennedy 3/25/03

  2. GLOBE Task Template PADI Item Design C. Kennedy 3/25/03

  3. Student Model Measurement Model Evaluation Procedures Work Product Specs Presentation Materials & Properties Templates store item characteristics Flow of Design Flow of Inference 4 Building Blocks of Instrument Development Evidence Centered Design of Assessments PADI Item Design C. Kennedy 3/25/03

  4. Presentation Materials & Properties Item presented to student Response is captured PADI Item Design C. Kennedy 3/25/03

  5. Think about theSTUDENT MODEL What do you intend the assessment to measure? From the DP(s), look at Rationale, Focal KSAs, Additional KSAs to help determine the proficiency(ies) you want to measure. PADI Item Design C. Kennedy 3/25/03

  6. Think about theSTUDENT MODEL • Then, develop a Student Model (model of student progress in one or more aspects of proficiency—the Student Model Variables). • For a FOSS example, the SM is comprised of: • SMV1, Using Appropriate Tools • SMV2, Graphing • SMV3, Distance, Speed & Acceleration See under the hood: PADI Item Design C. Kennedy 3/25/03

  7. Think about theMEASUREMENT MODEL What do you intend the item to measure? Select one or more Student Model Variables for the item. We want the item to measure: SMV3, Distance, Speed &Acceleration   (In particular, we want to measure the student’s understanding of reference points and the distance equation.) Alternate set of SMVs: SMV1, Using Appropriate Tools SMV3, Distance, Speed & Acceleration PADI Item Design C. Kennedy 3/25/03

  8. Think about theMEASUREMENT MODEL What form should the item and response take? • Dichotomous (one correct answer). • Partial credit (different valuations (scores) for qualitatively different student responses). • Rating scale (all rubrics have similar scales and going from a level 2 response to a level 3 response means the same thing for each item). • Item bundle (linked prompts and responses, e.g. 2 connected items: “1) Here is some data, draw a graph showing the relationship of distance traveled to propeller winds, and 2) explain what the graph shows about the number of winds and the effect on distance”). PADI Item Design C. Kennedy 3/25/03

  9. Think about theMEASUREMENT MODEL What form should the item and response take? Partial credit – students will construct a response from a word description and graphic. They will show the steps they followed to answer the question. • “One person drove a truck and the other person drove a car. Which vehicle went farther? How much farther?” See under the hood: PADI Item Design C. Kennedy 3/25/03

  10. Think about theMEASUREMENT MODEL What form should the item and response take? Alternate structure: A series of dichotomous responses, comprising an item bundle: • Show the initial and final reference points for the car [truck]. • Show the equation for calculating the distance the car [truck] traveled. • How far did the car [truck] travel? • Which traveled farther? • How much farther? PADI Item Design C. Kennedy 3/25/03

  11. Think about theEVALUATION PROCEDURES What evaluation procedures will be followed to transform the student response to an observable variable that can be used to estimate proficiency? Define the scoring rubric(s) to be used in evaluating student work, for example, one for each SMV measured on the item (this step is often done in conjunction with step 1 when defining the Student Model of interest). PADI Item Design C. Kennedy 3/25/03

  12. 4 Understands speed when time or distance are not held constant 3 Understands speed for simple cases 2 Understands distance 1 Intuitive understanding 0 Unresponsive • Think about theEVALUATION PROCEDURES (NOTE: The example here was not developed as a scoring guide, but I’ve used it here for convenience). PADI Item Design C. Kennedy 3/25/03

  13. SMV1 SMV3 • Think about theEVALUATION PROCEUDRES An item may be associated with multiple student model variables. Item A The item involves both SMV1 and SMV3. Two scoring guides could be developed and the value of the observable variable could depend on combinations of scores on each of the SMVs. PADI Item Design C. Kennedy 3/25/03

  14. SMV1 SMV3 • Think about theEVALUATION PROCEUDRES Multiple responses may comprise an item bundle (even with multiple student model variables!). Response 1 Response 2 Response 3 The item has several responses, which are conditionally dependent. The value of the observable variable will depend on combinations of responses. PADI Item Design C. Kennedy 3/25/03

  15. Conclusion: Initial steps in item design • Think about the Student Model: What do you intend the assessment to measure? • Think about the Measurement Model: What do you intend each item to measure? • Think about the Evaluation Procedures: How will student responses be evaluated to provide evidence about what students know and can do? PADI Item Design C. Kennedy 3/25/03

More Related