Using a personal response system for formative and summative assessment with instant feedback.

1. Using a personal response system for formative and summative assessment with instant feedback. Martin Sellens and Caroline Angus Centre for Sport and Exercise Science University of Essex HLST Conference 2009. Assessment and Feedback – What’s the Answer?

2. Overview • Simple rationale for using PRS. • Getting you started as a class. • Presentation of some research findings. • Demonstration of (how to author) different question styles. • Demonstration of outputs. HLST Conference 2009. Assessment and Feedback – What’s the Answer?

3. Rationale for using PRS • Large class sizes present a number of challenges to learning and assessment. • It is more difficult to engage the students. • Students often feel intimidated or nervous asking and answering formative questions. • It is difficult to tell if any learning is taking place. • Summative assessment can consume an inordinate amount of staff time and effort. • It is difficult to monitor attendance. HLST Conference 2009. Assessment and Feedback – What’s the Answer?

4. What is PRS? HLST Conference 2009. Assessment and Feedback – What’s the Answer?

5. Clicker overview • Power Switch - for switching the handset on and off. • LCD Display - displays the answers that students send, as well as notification that an answer has been received. • Enter/Send - key transmits the responses to the receiver. • Input Keys - variety of buttons for answering multiple-choice, true/false and numeric questions. • Scroll Up and Down - keys enables you to scroll through options on the LCD display. • Backspace/Cycle - you can use this to clear a character in an input field or cycle through options on the LCD display. • Menu Key - displays the set up menu. HLST Conference 2009. Assessment and Feedback – What’s the Answer?

6. Joining a class HLST Conference 2009. Assessment and Feedback – What’s the Answer?

7. Joining a class (step 1) Turn your handset on by moving the Power Switch to the I position. If the handset falls asleep (the LCD Display goes blank), press any key to wake it up. If you get stuck at any point, you can turn off the handset and start again. The LCD Display should momentarily read; (also showing battery status) HLST Conference 2009. Assessment and Feedback – What’s the Answer?

8. Joining a class (step 2) After a few seconds, the LCD Display should read: Followed by: HLST Conference 2009. Assessment and Feedback – What’s the Answer?

9. Joining a class (step 2) After a few seconds, the LCD Display should read: Press the green enter key You should now see the following: HLST Conference 2009. Assessment and Feedback – What’s the Answer?

10. Joining a class • Alternatively, you can directly select a class. • You should see the following box somewhere on the screen: • Note: The important part is the circled letter (or number) between the angled brackets. Eg. <B> • On the handset, press the Input Key that corresponds to this letter (B). • Press Enter/Send. HLST Conference 2009. Assessment and Feedback – What’s the Answer?

11. Entering this class. Is everything clear so far?(Press 1 to 5 then enter when the timer starts) • As clear as Dom Perignon • As clear as Stella Artois • As clear as Old Peculiar • As clear as Guinness • As clear as mud. HLST Conference 2009. Assessment and Feedback – What’s the Answer?

12. Engaging students. Typical student attention span during a lecture.Not necessarily a big one! (Adapted from Gibbs and Habeshaw, 1998) HLST Conference 2009. Assessment and Feedback – What’s the Answer?

13. Using PRS to engage students • Formative questions (MCQs) to review material or assess prior knowledge. • Voting on controversial issues. HLST Conference 2009. Assessment and Feedback – What’s the Answer?

14. Which of the following statements is not true? • The scapula is dorsal to the clavicle. • When the hand is supinated, the pollex is lateral to the digits. • In abduction the axillary angle decreases. • A mid-sagittal section bisects the umbilicus. • The femur is distal to the acetabulum. HLST Conference 2009. Assessment and Feedback – What’s the Answer?

15. Should the taking of performance enhancing drugs be ‘legalised’? • YES • NO. HLST Conference 2009. Assessment and Feedback – What’s the Answer?

16. Would you take a performance enhancing drug even if you knew it was bad for your health? • YES • NO • Don’t know HLST Conference 2009. Assessment and Feedback – What’s the Answer?

17. Research • PRS records enable analysis of • Attendance • Test scores • Time of response HLST Conference 2009. Assessment and Feedback – What’s the Answer?

18. Using PRS to identify and support struggling students (and run summative assessments). • A summative assessment (∑10% of module marks) took place every week for five weeks . • Students who scored >40 were contacted by the lecturer within 24 hours of the assessment. • Tutorial support provided if requested. HLST Conference 2009. Assessment and Feedback – What’s the Answer?

19. Performance in a series of five weekly summative assessments (Cell Biology for SES). HLST Conference 2009. Assessment and Feedback – What’s the Answer?

20. Analysis. • There was a moderate significant correlation between the students’ average performance in the five summative assessments and both MCQ performance (r=0.43, P<0.001) and exam performance (r=0.47, P<0.001) HLST Conference 2009. Assessment and Feedback – What’s the Answer?

21. Performance of 17 ‘follow-up’ students. HLST Conference 2009. Assessment and Feedback – What’s the Answer?

22. Using PRS for attendance monitoring. HLST Conference 2009. Assessment and Feedback – What’s the Answer?

23. Attendance at Anatomy and Physiology lectures. Unfortunately there was no correlation between lecture attendance and subsequent examination performance. HLST Conference 2009. Assessment and Feedback – What’s the Answer?

24. What do the students think about PRS? Focus groups and questionnaires. Questionnaire responses (200/200). HLST Conference 2009. Assessment and Feedback – What’s the Answer?

26. Summary of free text responses. (One good thing [124] one bad thing [84] about PRS). HLST Conference 2009. Assessment and Feedback – What’s the Answer?

27. Classification and number of positive comments received from students following one term of PRS use in lectures. HLST Conference 2009. Assessment and Feedback – What’s the Answer?

28. Classification and number of negative comments received from students following one term of PRS use in lectures. HLST Conference 2009. Assessment and Feedback – What’s the Answer?

29. Some different question styles. HLST Conference 2009. Assessment and Feedback – What’s the Answer?

30. From the music below identify the correct nursery rhyme • Ba ba black sheep • Twinkle twinkle little star • Mary had a little lamb • Old McDonald had a farm HLST Conference 2009. Assessment and Feedback – What’s the Answer?

31. The following statement describes a person with which learning style?‘When I spell I write down the word to find out if it feels right’ • Visual • Auditory • Kinaesthetic and Tactile • None of the above HLST Conference 2009. Assessment and Feedback – What’s the Answer?

32. Smokingshould be banned in public places • True (I agree) • False (I disagree) HLST Conference 2009. Assessment and Feedback – What’s the Answer?

33. What is the calorific value of 1 g fat? • Please type in the correct value. HLST Conference 2009. Assessment and Feedback – What’s the Answer?

34. What does the ‘D’ stand for in Franklin. D. Roosevelt? • Please enter your response. HLST Conference 2009. Assessment and Feedback – What’s the Answer?

35. Advantages to students Immediate feedback on knowledge level Easy way to identify strengths and weaknesses Anonymity: peer group do not know if a question is answered incorrectly Interesting Makes lectures less boring Helpful for preparing for exams Motivation to attend class Motivation to learn in preparation for and subsequent to class Improve performance Disadvantages to students Must actively participate in lectures, cannot hide and disengage Expense of paying a deposit for the clicker. HLST Conference 2009. Assessment and Feedback – What’s the Answer?

36. Advantages to lecturers Get students to confront current misconceptions Increase students’ retention of what you teach Test student’s understanding Measure what students know before you start to teach them Measure student attitudes Find out if students have done the reading Make some kinds of grading and assessment easier and quicker Facilitate testing of conceptual understanding Facilitate discussion and peer instruction Increase class attendance?? Improve performance Increase active learning rather than passive listening Disadvantages to lecturers Technical difficulties Time to set up the system at the start and end of lecture Disruption in class and time taken to settle students after a question has been posed. HLST Conference 2009. Assessment and Feedback – What’s the Answer?

37. Benefits of PRS • Record test results within PRS • Monitor performance quickly and easily HLST Conference 2009. Assessment and Feedback – What’s the Answer?

38. Benefits of PRS • Monitor and increase class attendance HLST Conference 2009. Assessment and Feedback – What’s the Answer?

39. Practicalities • Every 1st year student in the Department of Biological Sciences has a handset. • Hand sets and portable systems are available for ‘casual’ use. • Software has been installed across campus on all AV facilities(and receivers). • Question banks in correct format already available from some publishers. HLST Conference 2009. Assessment and Feedback – What’s the Answer?

40. I will consider embedding this learning technology in my institution. • Strongly agree • Agree • Neither agree nor disagree • Disagree • Strongly disagree HLST Conference 2009. Assessment and Feedback – What’s the Answer?