Derek France ( d.france@chester.ac.uk ) Kenny Lynch ( klynch@glos.ac.uk )

1. Enhancing student feedback through digital audio technology; an evaluation of staff & student experience Derek France (d.france@chester.ac.uk) Kenny Lynch (klynch@glos.ac.uk) GEES NSS Conference Manchester November 2010

2. Brief context: assessment and feedback • Assessment – central to the student experience: • “frames learning, creates learning activity and orients all aspects of learning behaviour” (Gibbs, 2006, 23). • Feedback – central to learning from assessment: • “feedback quantity and quality are the probably the most important factors in enhancing students’ learning (Race, 1999, 27). • However: • “the literature on student experiences of feedback tells a sorry tale” (Handley et al, 2007, 1). • “many students commented on ‘cryptic’ feedback which often posed questions, but gave no indication of where they went wrong”(GfK, 2008, 8)

3. Provisional sector results 2009+2010 (Full time totals)

5. Brief Context: Literature • ‘Greater focus on technology will produce real benefits for all’ (Department of Education and Skills, UK, 2005, p.2) • HEFCE, UK (2009, p.6) more cautiously states that, ‘focus should be on student learning rather than on developments in technology per se, enabling students to learn through, and be supported by technology’ • Prensky (2009) now advocates ‘Digital Wisdom’ and ‘Digital Enhancement’

6. Models of Podcasting (Nie, 2007) Model 1: Support Lectures Screencasting, podcating lectures Lecture summaries Pre-lecture listening materials (complex concepts) Lecture recordings Model 2: Support Fieldwork “iWalk”: Location-based information Instruction on technique & equipment use Video footage prepare for field trip Digital Story-telling Model 3: Support 3-Dimensional Learning Anatomical Specimens (Structures, tissues, dissections) Model 4: Support Practical-based Learning Software teaching & learning (replace text-based instructions) Model 5: Assessment Tool Student-created podcast based on field trips Student-created podcast to address climate change Model 6: Provide Feedback Model 7: Supplement Lectures Bring topical issues Guidance & tips Assessment tasks Skills Development Supplement Online teaching

7. Chester examples

8. The case study • One year, 2008 – 2009: • Two modules – Level 4 (69 students); Level 6 (34 students). • One formative and summative assessment exercises (L6) & four generic large group feedback opportunities (L4). • For each assignment: • Summative (Sm) -generic overview commentary combined with bespoke feedback on the group presentation • Formative (Fm) - informal podcast based on the e-postcard • Sm and Fm sent to the feedback section of each student’s VLE-based e-portfolio • Larger group generic feedback of four coursework assessments and placed in the online module space.

9. Feedback Uploading & Tracking • Feedback portal within the institutional VLE • Upload via modular e-learning areas

10. Formative Feedback by Wordle.net N = 87

11. Summative Feedback N = 87

12. Generic Large Group Feedback N = 90

13. Gloucestershire examples

14. Project aims GEES-funded small project November 2008 – March 2009, with the aims to: • develop a straightforward procedure for creating and delivering audio feedback; • follow a group of academics through the process of introducing audio feedback in a range of modules; and • evaluate the experience

15. Project members • Bill Burford (Landscape) • James Kirwan (CCRI) • Dave Milan (Geography) • Chris Short (Geography) • Claire Simmons (Broadcast Journalism) • Elisabeth Skinner (Community Development) • Alan Howe (Social Work)

16. Staff responses and Issues • Initially added to workload, but as become used to it, generally perceived as neutral [maximum?] • Initial concern about content preparation, led to scripting, but gradually moved towards notes/marking sheets and spontaneous recording [skill development and confidence] • Concerns about accuracy of delivery – mistakes were made in sending to students • Need for careful management of the medium – tone of voice, intimacy, trust • Quality – FASQ, mark moderating • Security, privacy & identity – misdirected files, archive, anonymous marking, team-based feedback • Handling grades – on recording or on work? • Embedding in practice - 12 months later staff still using the technologies

17. Nature and content of the feedback • More personalised: • “This feedback felt that the work had really been looked at and evaluated personally.” • ‘I listened to this at home and it felt like you were in the room with me and I wasn’t totally comfortable with that’. • More understandable? • You get “the tone of voice with the words so you could understand the importance of the different bits of feedback.” • Responsiveness to receiving information verbally: • “Don’t just briefly read it, you actually listen to it and take it in.” ” • “Better, goes in more. Can remember feedback from podcast but not from written.” • Greater sensitivity to the spoken word: • ‘I liked the feedback for what it was, but I also found it a bit depressing. It was very personal… I felt I let you down’. • “May be harder to hear a poor mark, rather than receiving it in writing.”

18. Conclusions • Students valued the medium as well as the message [Large, Generic Formative] • Staff found it easier than expected after initial concerns – liked it and if carefully set up may save time • NSS results highlight feedback as an issue; these pilots suggest this technology may increase student engagement with feedback and raise satisfaction in the lowest scoring NSS areas.

19. References • Department for Education and Skills (2005) Harnessing technology transforming learning and children’s services Available online at http://www.dfes.gov.uk/publications/e-strategy/ (accessed 6nd April 2008). • France, D., and Ribchester, C. (2008) Podcasts and Feedback. In Salmon, G., Edirisingha, P. (Ed.) Podcasting for Learning in Universities, pp. 70-79. Milton Keynes: Open University Press. • France, D., and Wheeler, A. (2007) Reflections on Using Podcasting for Student Feedback. Planet 18, 9-11.http://www.gees.ac.uk/pubs/planet/p18/df2.pdf • Gibbs, G. (2006). How assessment frames student learning. In C. Bryan and K. Clegg (Eds.), Innovative Assessment in Higher Education (pp 23-36). London: Routledge. • GfK (2008) NUS/ HSBC Students Research. GfK Financial London, Study Number 154021 • Handley, K., Szwelnik, A., Ujma, D., Lawrence, L., Millar, J. & Price. M. (2007). When less is more: Students’ experiences of assessment feedback. Paper presented at the Higher Education Academy Annual Conference, July 2007. Retrieved June 5, 2008 from • http://www.heacademy.ac.uk/assets/York/documents/events/conference/E5.doc • HEFCE. (2009). E-learning strategy. Retrieved August 20, 2009 from www.hefce.ac.uk/pubs/hefce/2005/05_12/ • Nie, M. (2007). Podcasting for GEES Subjects. Paper presented at the IMPALA 2 workshop, Dec 2007. Retrieved June 5, 2008 from http://www2.le.ac.uk/projects/impala2/presentation/2nd%20Workshop/Presentations/Ming%20Nie • Prensky, M. (2009) H. Sapiens Digital: From Digital Immigrants and Digital Natives to Digital Wisdom. Innovate, 5, No3, pp1-9. (accessed 20th August 2009). http://innovateonline.info/index.php?view=article&id=705&action=article • Race, P. (1999). Enhancing student learning. Birmingham: SEDA. • Salmon, G. & Edrisingha, P. (2008). Eds. Podcasting for Learning in Universities. Maidenhead: Open University Press. Including companion website: http://www.atimod.com/podcasting/index.shtml