Inline Tutorial Self Assessments: Improving Satisfaction (and possibly learning)

1. Inline Tutorial Self Assessments: Improving Satisfaction(and possibly learning) Robert Schudy Dan Hillman

2. Problem Statement Conventional web pages and “talking at them” lectures have serious pedagogic weaknesses: • When the presentation is done students don’t know what they learned and didn’t, so they are unsure about their preparation for subsequent topics. • Most assessments (feedback) are done on a weekly, non-immediate basis, delaying guidance and slowing learning. • If the particular presentation didn’t work for a student an alternative presentation is not automatically presented.

3. The Test Yourself Solution • We defined and implemented, using JavaScript, the ability to introduce interactive tutorial self assessments directly in lecture web pages. • Dan named these “Test Yourself.” • All Test Yourself formats include the ability to incorporate tutorial material, often per answer. • We have used these in several courses in the MSCIS and Admin Sciences online and on-campus programs, and they have been well received by students.

4. Test Yourself Styles • Single answer “show answer” • Single answer “radio buttons” • Multiple answer “check boxes”

5. Single answer “show answer”

6. Note answer and detailed mathematical problem solution

7. 2. Single answer “radio buttons”

8. Note indications of student answer, correct and incorrect answers, and tutorial material for each answer.

9. 3. Multiple answer “check boxes”

10. Note the tutorial material for each answer, which helps students focus their study on the answers they missed, without needing to search for anything.

11. Evidence that students love Test Yourself • In-course surveys were very positive. • Students have praised Test Yourself in the student evaluations. • When courses didn’t have Test Yourself, students who experienced them in earlier courses requested them during the course and in the evaluations. • When Test Yourself was in some but not all pages of a course, students requested them in all pages.

12. Evidence that suggests that Test Yourself Improves Learning • Students reported that Test Yourself improves the time efficiency of their learning. • There is some evidence that where we put Test Yourself items we have improved quiz scores; we need to assess this more formally. • It would be interesting to conduct a formal controlled learning experiment, but we may need to involve the human subjects institutional review board because we have enough evidence that it helps learning that we would risk disadvantaging the control group.

13. Disclaimers and Realities • We have not conducted a formal research evaluation of how effective this is in promoting learning. • Developing the content requires about one hour of faculty time per Test Yourself. • We don’t have the infrastructure to collect discriminant analysis, usage, or other data. • Implementation requires a little knowledge of HTML. This is easy online, where we have professional instructional designers, but may be problematic without.

14. Lessons Learned

15. “What did I put for my answer?”

16. “Do the checks mean statementsare true or that my answers are correct?”

17. Make it obvious. Correct/ incorrect feedback

18. Make it obvious. Correct/ incorrect feedback Student answer

19. Make it obvious. Correct/ incorrect feedback Student answer True or false prefaces

20. Future Work • This is a small step in making our web sites more interactive and supportive, and adapting the learning to the student, but much more is possible.