1 / 27

Effective Implementation Formative and Summative Project Evaluation

Effective Implementation Formative and Summative Project Evaluation. When Bad things happen to Good projects!. Great ideas become ruined with bad implementations! Brad Wiley English teacher case-text Innovations get bad press with bad implementations! Failed “good ideas”.

odell
Download Presentation

Effective Implementation Formative and Summative Project Evaluation

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Effective ImplementationFormative and Summative Project Evaluation

  2. When Bad things happen to Good projects! • Great ideas become ruined with bad implementations! • Brad Wiley English teacher case-text • Innovations get bad press with bad implementations! • Failed “good ideas”

  3. Role of Implementation • Diffusion • Dissemination • Adoption • Stakeholders

  4. Adoption Process • Awareness • Interest • Evaluation • Trial • Full Adoption • Integration

  5. Adoption Models • Research based • CBAM--Concerns-Based Adoption Model • Focuses on the concerns of the users • Stages of Concern Questionnaire • Level of Use Questionnaire

  6. Implementation • Role as a designer • Recommendations • Experiences • Role as a teacher/user/learner • Limitations • Experiences

  7. Assignment • Using the case of Brad Wiley in Chapter 17, identify what principles of good implementation were ignored or violated and write a short implementation plan for Mr. Wiley’s project.

  8. Formative and Summative Evaluation of Design • Assessments were used to determine student achievement • Evaluation of the design and the instructional materials is the current topic.

  9. Occurs at 2 times • Evaluation of the materials and processes for the purpose of revisions--Formative • Once materials and processes in place-they are evaluated in terms of their effectiveness-Summative

  10. Formative • Occurs during development of materials • Revisions are expected! • In business-skipped due to cost!

  11. Formative evaluation is especially important if… • Designer is novice • Content is new • Technology is new • Audience is new • Designer new to ID • New or unfamiliar instructional strategy • Task performance is critical • Client expectations and later revision is not possible

  12. Formative Review Stages • Design reviews--before development • Expert reviews--after development before use • Learner validation--during use • Ongoing evaluation--during use

  13. Design Reviews • Goal Review • Environment and Learner Analysis Review • Task Analysis Review • Assessment Specifications Review

  14. Expert Reviews • Content experts • ID experts • Educational specialists • Teachers • Target audience specialist

  15. One-to-One Evaluation • Tries materials with 2-3 members of target audience • Sits down one-to-one--observation and questions • Typos, unclear directions, bad examples, poor illustrations, bad URLs

  16. Small Group Evaluation • Uses more learners…usually after the one-to-one revisions were made. • Usually only observation, data gathered through charting • Survey after can be used with learners. • pg336

  17. Field Trials • Determine effectiveness of revisions made after small group eval • Find problems with administration of materials in real life situations • Validate instruction with larger group to predict effectiveness

  18. Which stage of the formative evaluation procedure does each statement describe? Teachers/trainers report the appropriateness of the documentation that accompanies a package of computer based instructional software..

  19. Which stage of the formative evaluation procedure does each statement describe? Experts in the content area for which a slide-tape set was developed review the materials for accuracy and currency.

  20. Which stage of the formative evaluation procedure does each statement describe? Individual students review the storyboards of an instructional videotape with the designer.

  21. Which stage of the formative evaluation procedure does each statement describe? At least 30 students go through the instruction.

  22. Which stage of the formative evaluation procedure does each statement describe? Designer only assists students when they encounter severe difficulties.

  23. Which stage of the formative evaluation procedure does each statement describe? Students read and think aloud as they interact with written instructional materials.

  24. Which stage of the formative evaluation procedure does each statement describe? Test materials are tried out with groups of individuals some of whom are “masters” of the content, some of whom are “nonmasters.”

  25. Which stage of the formative evaluation procedure does each statement describe? Data are collected regarding problems with the instructionafter the instruction is implemented in the target contexts.

  26. Summative Evaluation • Final evaluation of effectiveness of the design, materials and learning. • Usually will impact continued use • Determine goals of the evaluation--usually determined by client

  27. Group Assignment • Determine what formative evaluation process you could use with your instructional design. • Discuss and decide as a group • Complete Formative Evaluation Documents and submit to Cheryl by e-mail…one per group

More Related