Health Sciences and Practice & Medicine Dentistry and Veterinary Medicine Higher Education Academy Subject Centres

1. Health Sciences and Practice &Medicine Dentistry and Veterinary MedicineHigher Education Academy Subject Centres Julie Attenborough Head of Department, Education Development Unit, City University eLearning in Health 2011 conference collaboration, sharing and sustainability in the current environment Using formative audio feedback via Moodle to feed-forward and enhance learner engagement in assessments

2. School of Community and Health Sciences Our aim To enhance learner ability to feed-forward learning from formative assessment through effective use of audio & written feedback Education Development Unit

3. School of Community and Health Sciences We do not know how effective audio feedback is in enabling feed-forward from formative work… We know from the Sounds Good1 project that: • Students and staff like audio feedback: students frequently remarked approvingly about its personal nature and the detail provided. • Some staff did manage to save time by speaking rather than writing their feedback. • Great majority of lecturers were pleased with the experience and noted that students received more, and higher-quality, feedback than they otherwise would have done. • Sounds Good concluded that most UK teachers in higher education would find it worth giving audio feedback an extended trial with at least with some of their assessment work.” 1www.soundsgood.org.uk Education Development Unit

4. School of Community and Health Sciences According to JISC (2010)2 … “While not all learners can be considered competent users of technology for educational purposes, this research3 (involving nearly 3,500 survey respondents, focus groups and individual interviews) suggests that institutional assessment and feedback practices need to take account of the digitally enhanced landscape in which learning now takes place” 2www.jisc.ac.uk/assessresource 3www.jisc.ac.uk/learnerexperience Education Development Unit

5. School of Community and Health Sciences National Student Survey… • ...consistently shows lower levels of satisfaction with assessment & feedback that with other aspects of HE experience • National Student Forum (2009)4 “We would like to see all universities and colleges implement a systematic policy to enhance traditional methods with new technologies [and] leverage technology to provide innovative methods of assessment & feedback” 4 www.nationalstudentforum.com Education Development Unit

6. School of Community and Health Sciences Background • Mentorship 15 credits degree level CPD module • Requires students to write a 500 word formative assessment at mid-point of the taught element of the course • Aim is to provide feedback on formative work to enable feed-forward to summative assessment Education Development Unit

7. School of Community and Health Sciences Our Pilot • Group 1 – recorded MP3 files and inserted these on word documents and emailed them to students (students sent an email with instructions on how to submit online; but chose to submit via email) • Group 2 – used audacity to record and saved files as MP3 files & uploaded files on Moodle (students shown in class and given handout for online submission; most submitted via Moodle) Education Development Unit

8. School of Community and Health Sciences Conclusions from the pilot • Audacity offered the easiest way to record; save files as MP3 and upload to Moodle • Uploading files to Moodle would ensure secure access to audio files without getting lost in students emails • Students must be shown in class and given a handout on how to submit assignments online • Need to ensure tutors are trained and have ongoing support to provide audio feedback Education Development Unit

9. School of Community and Health Sciences Methodology Phase 1a – Getting tutors on board • Gained ethical approval • Hands-on training session for tutors • Gained consent from tutors – commitment emphasised • Installation of audacity on tutors PCs and provision of headsets with microphones Education Development Unit

10. School of Community and Health Sciences Methodology Phase 1b – Getting students on board • Researchers introduced students to the project aims, methods and their involvement in class • Gained consent from students for data collection • Student handouts with instructions on online submissions • - Students could submit online for audio feedback even if they chose not to participate in the data collection part of the project Education Development Unit

11. School of Community and Health Sciences Methodology • Phase 2 – Giving and receiving audio feedback • Individual tutors recorded and uploaded audio feedback on the formative assessments that were submitted on Moodle • Students received automatic emails from Moodle once the audio feedback files was uploaded on their grade sheet • Students could then access, open and listen to the audio file Education Development Unit

12. School of Community and Health Sciences Methodology Phase 3 – Data Collection • Students completed questionnaires at the end of their course • Individual student interviews have been arranged to understand how students used the audio feedback to inform their summative work and what forms of feedback are most beneficial • A tutor focus group is to be conducted in July to explore tutor experiences of giving audio feedback and what strategies help in giving meaningful feedback Education Development Unit

13. School of Community and Health Sciences Methodology • Phase 4 – Analysis and Synthesis • Questionnaire data analysis informed the questions asked in the student and tutor interviews • All interview data will be transcribed and analysed qualitatively to identify themes related to giving and receiving feedback • The research team is currently developing a toolkit to support tutors to give feedback. The analysed results will contribute to the guidelines that will be one of the key features of this tool kit Education Development Unit

14. School of Community and Health Sciences Conclusions from the pilot • 24 completed questionnaires / 10 did not receive audio feedback • Those with clear instructions and Moodle access easily submitted online • 7 students from CGs group had no audio saved in the word document • Majority (12) students listened to feedback at home; 2 students listened at work Education Development Unit

15. School of Community and Health Sciences Results – Questionnaire data The response rate was 49% Number of respondents: 49Expected number of respondents: 100Response rate: 49.0%Launch date: 16 Mar 2011Close date: 11 May 2011 Education Development Unit

16. School of Community and Health Sciences Technology and access to feedback • While majority of students found it easy to access and upload the essay to Moodle (n=38); not all of these students could download and listen to audio and written feedback with the similar ease Education Development Unit

17. School of Community and Health Sciences Listening to the audio feedback • 18 respondents who could download and listen to the audio feedback • - Listened to it more than once • Strongly agreed or agreed that feedback quality, length and clarity were appropriate • Some comments: • “Clearly described what I can improve” “All points were mentioned” “Too quiet” “voice was a bit shaky” “clear but not specific” Education Development Unit

18. School of Community and Health Sciences Using audio feedback “I will write my audio feedback in bullet points” “did not submit online” “did not receive audio feedback” “unable to access feedback” Education Development Unit

19. School of Community and Health Sciences Audio vs. Written feedback • Student who received audio feedback only said they preferred audio feedback over written • Audio felt more personal as compared to written “written feedback just had a few marked out areas of correction therefore audio feedback gave more” “whilst audio feedback gives emphasis on what I need to improve on I still believe that the same emphasis is still given in written feedback” Education Development Unit

20. School of Community and Health Sciences Audio vs. Written feedback • Students who could not access audio feedback or did not upload their essays on Moodle said they preferred written feedback • These students strongly agreed or agreed that they preferred written feedback because it was easier and quicker to read compared to listening to audio feedback Education Development Unit

21. School of Community and Health Sciences Audio vs. Written feedback Despite highlighting preference for audio or written feedback in earlier questions majority of students concluded that they preferred if written and audio feedback are given together “Both have different uses. Audio is good for content but written gives examples of incorrect grammar” Education Development Unit

22. School of Community and Health Sciences Teacher presence and sense of caring • Students who accessed audio feedback identified that receiving audio feedback helped them experience tutor’s presence “I appreciated the signposting” • However when comparing audio with written majority (56%) of the respondents felt both written and audio feedback demonstrated teacher’s sense of care “I believe that audio and written demonstrate tutor’s sense of caring on my work” Education Development Unit

23. School of Community and Health Sciences Students’ engagement Students felt encouraged to revise their work after listening to audio and written feedback... Slightly higher preference towards audio feedback Education Development Unit

24. School of Community and Health Sciences What forms of feedback will help you in the future? “May be subtitled audio feedback on request for hearing impaired” “Need to feedback easier to find in Moodle” “Video feedback would probably enhance the experience” “We could also have group feedback” Education Development Unit

25. School of Community and Health Sciences Next stages of the project… • Focus group with tutors • Individual interviews with students • Project extended to include another module • Audio feedback for students compulsory discussion postings at mid-point • of the course • Aim to complete project by September 2011 – release of project report Education Development Unit