Effective and Efficient Feedback to Undergraduate Students: It is possible to do both!

1. Effective and Efficient Feedback to Undergraduate Students: It is possible to do both! Dr Julie Hulme Staffordshire University j.a.hulme@staffs.ac.uk

2. Acknowledgements • Dr Mark Forshaw • Miss Beverley Ayres • Miss Liz Dalgarno • HEA Psychology Network • Staffordshire University Learning and Teaching Fellowship Scheme

3. What are the issues? • Increasing student numbers, and staffing pressures – need for efficiency • Students as “customers” – convenience, technology focussed – need for effectiveness (promotes learning) • National Student Survey – feedback is area with which students are least satisfied

4. What are the issues? • Picture in psychology mirrors national trends across subject disciplines • Hulme and Forshaw (2009).

5. Two projects • Hulme and Forshaw: Effectiveness of feedback provision for undergraduate psychology students. Funded by HEA Psychology network. • Hulme: Promoting reflective learning and use of feedback amongst undergraduate psychology students. Funded by Staffordshire University Learning and Teaching Fellowship.

6. Methodology 1 • Focus groups to produce and revise recommendations in consultation with both staff and students • Establish stimulus for constructivist dialogue

7. Focus Groups • Three universities – Staffordshire (staff and students), Keele (staff only) and Leeds Trinity (staff and students); 4-6 participants per group • Staff groups led by JH, student groups led by PG student not teaching those students • Role of feedback, preferred methods, suggestions for improvement

8. Quotes • “…I think it’s very easy to say what you think is good and bad about the particular piece of work but not necessarily identify what somebody has to do, to make it better. I think that’s the bit that’s often missing from feedback, and that’s actually the most important bit in some ways” (tutor 4, Keele) • “I suppose good feedback is erm, quite specific and not only says that these parts are good these parts need improving but this is how you could go and improve them and kind of giving you a bit of a jolt into knowing what areas you should go into to try and improve that, rather then just saying this is a bit weak because a few people have, you get differences in markers” (student 1, Leeds Trinity)

9. Summary of focus group findings • Lots of agreement between staff and students, and also with the findings from the questionnaires (Hulme and Forshaw, 2009) • Important themes = constructive, ambiguity, consistency between tutors, student perceptions and disengagement • Suggestions for improving practice were shared between students and tutors

10. E-Delphi Groups • Webpage containing recommendations • Staff and students invited to view these and provide comments anonymously, only identified as staff or student • 7 tutors and 6 students participated actively (national) • Recommendations revised accordingly • More comments invited on revised recommendations and on other comments • Final version produced

11. Final recommendations • See handout

12. Staff training sessions • Several workshops have been delivered, in Psychology and other subjects • Initial resistance, but attendance resulted in enthusiastic response and positive feedback, with ideas being adopted at least in part by several members of staff

13. Feedback from staff training sessions “Thank you for delivering a stimulating and thought provoking session on effective and efficient feedback. Colleagues appreciated the interactive nature of the workshop and the way in which they were encouraged to share their own experience of providing students with feedback. Your practical tips - supported by very credible research - were particularly helpful; I know that some of the points you made have been picked up and put into action by my colleagues (for example your idea about asking students the question about what aspects of feedback they would like their lecturer to focus on)”

14. Student training • Level 2 module – core for GBC – Biological Bases of Behaviour (up to 160 students; 5 tutors) • Article review week 4, seen exam end of module • Seminar for coursework return. Feedback structured, WP, focussed on transferrable skills taken from marking criteria • Work returned with feedback, mark with-held until end of session • Reflective activities – what was good? What can you improve? Action planning. • Staff and student evaluation collected

15. Student evaluation comments • Very detailed feedback • Constructive feedback – said how to improve, clear and well broken down into stages • Timely – not the end of the module • Waiting for mark made me anxious – but it did make me reflect on the feedback

16. Student evaluation • Quantitative analysis supported open comments • When asked to compare the feedback given on BBB with other modules (worse > better, 1-5), mean = 3.96, sd = 0.81 • Students appreciate the opportunity to discuss feedback with tutor and believe that their reflections will help them to improve in the future

17. Tutor evaluation • Myself + 3-5 other tutors, 3-4 of whom are PGwT • First and second year – increased workload from typing comments, but could give worthwhile and useful feedback • Third year – training of staff in efficient e-marking practice – tutors said took about the same amount of time and output was much more detailed and higher quality

18. Changes by year • 2007 – no intervention • 2008 – detailed skill-based feedback and reflection in seminar • 2009 – formative work and feedback, and detailed skill-based feedback with reflection in seminar

19. Mean marks (± sd)

20. Preliminary analysis: significant differences • C/wk 2007 vs 2009 (improved) and 2008 vs 2009 (improved) • Exam 2008 vs 2009 (improved) • Change 2008 vs 2009 (less change!) • Module 2007 vs 2009 (improved) and 2008 vs 2009 (improved)

21. Preliminary analysis • Cohort differences are the obvious suggestion; could also argue for assessment differences. • Possibility – feedback enhances exam performance if no prior training given • BUT if formative feedback is given, then the improvement occurs prior to first assessment – further feedback then maintains standards • ??????

22. Next! • Further analysis of marks including controlling for semester 1 academic achievements • Evaluation of portfolio system and personalised questions • Trial of e-marking systems (eg Blackbox) and evaluation • Further staff and student training • IMPORTANT: to include opportunities for formative feedback and pre-assessment training

23. Conclusions • It is difficult to balance efficiency with effectiveness…but… • …It is possible to provide efficient and effective feedback • Initial formative opportunity, skills based and constructive when summative (recommendations), group reflection • Rewards of this for both staff (job satisfaction) and students (improved learning and transferable skills) • Group reflection provides opportunity for dialogue without massive time commitment or need for students to be proactive • Largely consistent with O’Siochru and Hopkins (PLAT, 2010) – development using feedback over long term, discursive and targeted to future work.

24. Questions? • Reference: • Hulme, J.A. and Forshaw, M.J. (2009). Effectiveness of feedback provision for psychology undergraduate students. Psychology Learning and Teaching, 8, 1, 34-38. • A further PLAT article is in preparation