Evaluating Computer and Web-Based Instruction

1. Evaluating Computer and Web-Based Instruction There are two different approaches to the evaluation of computer and web-based instruction: 1. Assessing the learners 2. Assessing the program

2. Assessing the Learner Kirkpatrick’s Four Levels of Evaluation: 1. Reaction • Participants initial reactions -- • satisfaction and perception of value • Measured with surveys and focus groups 2. Learning • Measured by criterion-referenced test -- • the criteria are course objectives

3. 3. Transfer • The amount of material participants use • in everyday work 6 weeks to 6 months • from now • Measured with tests, observations, • surveys and interviews 4. Business Results • The financial impact of the course on • the bottom line of the organization • 6 months to well into the future • Can be assessed by tracking business • measures, observations, surveys and • qualitative measures -- difficult!

4. Assessing the Program 1. Instructional Design • Are objectives clear? What will • learners be able to do after they run • the program? • Does the content reflect the objectives? • Is there a target audience? • Instructional approach: Does it achieve the objectives? Does it match the target audience ?

5. 2. Interactivity • Are there a variety of interactions • available? • Does the learner make important, • meaningful responses? • Does the learner receive feedback? How often? If questions are involved, are they relevant to what was presented? Is it clear how to respond to questions ? Is corrective feedback supplied with the questions?

6. 3. User Control • Are program’s controls easy to find • and use? • Is it easy to navigate from one screen • to another? • Can users choose a variety of paths • through the program? • Is there an exit option available? • Is there other aids to navigation such • as course maps and instructions? • Overall, do learners feel in charge of • their learning?

7. 4. Motivation • Does the program get the learner’s • attention and maintain it? • Does the program inspire confidence • or does it make the learner feel lost • or frustrated? • Does the learner feel challenged? • Does the program use a variety of • techniques (novelty, questioning, • humor, game elements, etc.) to keep • the learner engaged?

8. 5. Aesthetics/Tone • Does the program’s physical design • “fit” with its instructional purpose? • Is the program appealing to both • eyes and ears? • Does it otherwise meet principles of • good design? • Does the program avoid being trite, • condescending, pedantic and so on?

9. 6. Choice of Media • Does the program effectively employ • all the different media it uses? • Is the combination of media used • appropriately and effectively? • Is the gratuitous use of these media • avoided? 7. Absence of Flaws • Is spelling and grammar correct? • Do all controls work?