Web Opportunity: Interactive Quizzes

1. Web Opportunity: Interactive Quizzes Rance Harmon CS 500, Montana State University April 17, 2008

2. Presentation Outline • Commercial opportunities • Features of web quizzes • Samples • Web quizzes for education • Research • Benefits • Recommendations • Security issues and solutions

3. Definitions • Quiz--the act or action of quizzing; specifically: a short oral or written test • Survey--the act or an instance of surveying: querying (someone) in order to collect data for the analysis of some aspect of a group or area Source: Merriam-Webster online

4. Quizzes for Commercial Sites • Can increase traffic to your website • Tips • Make quiz easy for users to share with friends (pdf, email) etc1 • Make sure that the difficulty level not too hard or easy2 • Tell users that the answers can be found somewhere on the site (encourages visitors to look at the entire site, increasing their chances of a transaction)2 Sources: 1Hersey[2006] 2Haring [2000]

5. Commercial Quiz Development Tips • What is the main product that I want to sell to my customers? • Who are the people who make up my key target audience? • What will interest that target audience the most? • What quiz topic relates best to what their interest is and what my product can offer them? • What is the key question they want answered that incorporates all these elements? http://www.iconiso.net/Love%20Quiz.htm -- Shane Hershey C.E.O. of Iconiso.com Source: Hersey [2006]

6. Important Features • Clear directions • Timed or un-timed • Scoring criteria • How many points each • Number of questions • Purpose of quiz • Navigation • Feedback on answers • Number of questions per page • Same or different questions each time?

7. Sample Quiz (1) • http://www.w3schools.com/html/html_quiz.asp

8. Important Features • Clear directions • Timed or un-timed • Scoring criteria • How many points each • Number of questions • Purpose of quiz • Navigation • Feedback on answers • Number of questions per page • Same or different questions each time?

9. Sample Quiz (2) • http://www.plentysmart.com/comp/gen-acronyms/gen-acronyms.php

10. Important Features • Clear directions • Timed or un-timed • Scoring criteria • How many points each • Number of questions • Purpose of quiz • Navigation • Feedback on answers • Number of questions per page • Same or different questions each time?

11. Sample Quiz (3) • http://www.gotoquiz.com/what_american_accent_do_you_have

12. Important Features • Clear directions • Timed or un-timed • Scoring criteria • How many points each • Number of questions • Purpose of quiz • Navigation • Feedback on answers • Number of questions per page • Same or different questions each time?

13. Sample Quiz (4) http://java.sun.com/developer/Quizzes/jsptut/

14. Important Features • Clear directions • Timed or un-timed • Scoring criteria • How many points each • Number of questions • Purpose of quiz • Navigation • Feedback on answers • Number of questions per page • Same or different questions each time?

15. Research: Quiz Benefits • Enhance student learning and retention • More accurate measure of student efficacy than traditional high-stakes testing • Important to provide multiple online evaluations • mitigates the effects of student stress • allows the students to “practice” before the final test • motivates them to acquire the expected skills • “Students best attain the expected skills when many online evaluations are incorporated into the course, and in this case, traditional, non-online evaluations are unnecessary.” • Discarding 30-50% of the lowest marks reduces student stress while maintaining motivation. Source: Woit and Mason [2003]

16. Research: Effective Quiz Practices (1) • A known way to encourage the use of educationally beneficial tools is to convert them from student-driven to assessment-driven tools • Student-driven--created to assist student learning • But, use is not required • Do not count towards the student’s course grade. • Many visualization and simulation tools are student-driven. • Assessment driven--tools that the students must use in order to complete their assignments Source: Brusilovsky[2005]

17. Research: Effective Quiz Practices (2) • Individualized exercises can • significantly reduce cheating • improve student understanding and exam performance • Self-assessment quizzes strongly benefit student performance • When in-class quizzes directly related to self-assessment quizzes, the use of the self-assessment quizzes increased significantly Source: Brusilovsky[2005]

18. Quiz Recommendations • Set a limited time for quizzes. • Frequent low-stakes assessment better than infrequent, high-stakes tests and exams. • Display feedback and correct answers. • If you’re concerned about students sharing answers after they’ve taken the quiz: • randomize the question and answer order • make some of the questions random (test bank) • As an additional assignment, have students bring a question to class about a quiz item they missed. Source: Cole [2007]

19. Quizzes As a Tool to Get Students to Read Course Materials • Encourage students to do the reading so they can do well on the quiz. • Students get feedback on how well they understood the reading assignment. • Give instructor data about: where to focus course time by showing which parts of the reading that students have: • Mastered • Found confusing Source: Cole [2007]

20. Quizzes as Practice for Test Taking • Practice tests give students an idea about what to expect on real tests. • Practice tests can be based on old questions similar to the current test questions. (e.g., use last year’s final) • Use a zero point test with questions in random order with random choices. • Allow students to take the test as many times as they wish. • Display feedback, but not correct answers so it presents more of a challenge. Source: Cole [2007]

21. Quizzes as Source of Feedback About Course Materials • Use quizzes to see how well students understood class materials. • If using with a traditional class • Have quiz available for a limited time • Allow students to take it once and display feedback and correct answers Source: Cole [2007]

22. Potential Problem Printing questions or feedback and correct answers Solutions Randomize order of questions and choices Give quizzes with random subsets of questions in test bank Quiz Security Issues (1) Source: Cole [2007]

23. Potential Problem Using the textbook or outside resources Solutions If you are giving a chapter check quiz, then not a problem because the goal is for students to use the textbook Use timed quizzes—e.g., 30 seconds per multiple choice question Ask students to apply their knowledge to novel situation: synthesis and application questions can’t be looked up Quiz Security Issues (2) Source: Cole [2007]

24. Potential Problem Working with friends Solutions random question order random choice order and questions randomly pull questions from a test bank. have a time limit to complete quiz Quiz Security Issues (3) Source: Cole [2007]

25. Potential Problem Have someone else take the test Solutions proctored exam where students need to show ID low stakes quizzes Quiz Security Issues (4) Source: Cole [2007]

26. Potential Problem Students can read answers on the client (e.g., the questions and answers are contained in JavaScript, php) Solutions Keep answers on the server-side Quiz Security Issues (5) http://www.palomar.edu/at/hotpot/firstlines02.htm

27. Presentation Summary • Commercial opportunities • Features of web quizzes • Samples • Web quizzes for education • Research • Benefits • Recommendations • Security issues and solutions

28. Sources (1) • Haring, Rachael. Entertain Me! A Guide to Website Interactivity. 5 Oct 2000. WebProNews.com. Accessed 12 April 2008. http://archive.webpronews.com/archives/100500.html • Hersey, Shane. Creating an online quiz and you may increase hits by more than 50%. 9 Feb 2006. AliveOnline.com. http://www.alive.com.au/site/articles/18/1/Creating-an-online-quiz-and-you-may-increase-hits-by-more-than-50/Page1.html Accessed 12 April 2008. • Brusilovsky, P. and Sosnovsky, S. 2005. Engaging students to work with self-assessment questions: a study of two approaches. In Proceedings of the 10th Annual SIGCSE Conference on innovation and Technology in Computer Science Education (Caparica, Portugal, June 27 - 29, 2005). ITiCSE '05. ACM, New York, NY, 251-255. DOI= http://doi.acm.org/10.1145/1067445.1067514

29. Sources (2) • Merriam-Webster OnLine. accessed 12 April 2008. http://www.merriam-webster.com/dictionary. • Woit, D. and Mason, D. 2000. Enhancing student learning through on-line quizzes. In Proceedings of the Thirty-First SIGCSE Technical Symposium on Computer Science Education (Austin, Texas, United States, March 07 - 12, 2000). S. Haller, Ed. SIGCSE '00. ACM, New York, NY, 367-371. DOI= http://doi.acm.org/10.1145/330908.331887 • Woit, D. and Mason, D. 2003. Effectiveness of online assessment. SIGCSE Bull. 35, 1 (Jan. 2003), 137-141. DOI= http://doi.acm.org/10.1145/792548.611952