1 / 12

Analyzing Data from Formative Evaluations

Analyzing Data from Formative Evaluations. April 14, 2008. Quantitative Relies on data collected from tallying users’ performance on pretests, practice tests, and post-tests. Numbers-oriented. Qualitative

lisle
Download Presentation

Analyzing Data from Formative Evaluations

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Analyzing Data from Formative Evaluations April 14, 2008

  2. Quantitative Relies on data collected from tallying users’ performance on pretests, practice tests, and post-tests. Numbers-oriented. Qualitative Relies on data collected from observing users’ experience interacting with the instruction. Narrative-oriented. Theoretical Approaches to Evaluating FE Data

  3. Analyzing Data from FE • Different formats of FE require different types of analysis and provide different kinds of data. • One-to-One Trials • Small Group Trials • Field Trials

  4. Qualitative Evaluation Learner characteristics and entry behaviors Direct responses to instruction Learning time Responses to a user questionnaire. Quantitative Evaluation Posttest performance One-to-One Data

  5. Revising from One-to-One Data • Revisions can be based on • Learner suggestions • Learner performance • Evaluator’s reactions/observations

  6. Challenges of Using One-to-One Data • Learners will be diverse. • Individuals will have unique experiences and responses. • General summaries, conclusions will be difficult to make. Use this data as a preview/frame for the data you will collect in the small group and field test collections.

  7. Qualitative Evaluation Responses to user questionnaire. Learning and testing time. Comments made directly on materials. Quantitative Evaluation Pretest performance Posttest performance Small Group/Field Test Data

  8. Compiling Small Group/Field Test Data • Create tables to compare: • Students’ scores for each performance objective (p. 318) • Students’ scores for pretest/postest (p. 320) • Students’ scores for achieving objectives mapped onto goal analysis (p. 321)

  9. Revising from Small Group/Field Test Data • Revisions can be based on • Learner suggestions • Learner performance • Evaluator’s reactions/observations

  10. Challenges in Using Small Group/Field Test Data • Observations of learner interaction will be more difficult to conduct. • Data will necessarily be more quantitative than qualitative. • Performance-oriented evaluation does not always suggest how the tests should be revised. Don’t rely on data collected in larger groups to give you substantive information on user experience.

  11. Interpreting FE Data • Once data is collected and summarized, look for trends in the following areas: • Entry Behaviors • Pretests and Posttests • Instructional Strategy • Learning Time • Instructional Procedures • These categories can also be used to frame debriefing/survey questions.

  12. Sample Instructional Revision Plan

More Related