Engaging in the student voice via the Peer Assisted Teaching Scheme www.monash/pats

1. Engaging in the student voice via the Peer Assisted Teaching Scheme www.monash/pats Associate Professor Angela Carbone National Senior Teaching Fellow 2012 Office of the Pro Vice-Chancellor (Learning and Teaching) Monash University Sponsored by CADAD

2. Presentation overview • Why engage in the students voice? • Engaging in the student voice through Peer Assisted Teaching Scheme (PATS) • What is PATS? • PATS process • Results from trials of PATS • PATS | Using the student voice as a driver for quality enhancement • PATS | Exhibits an organisational culture that values the student voice Keynote Annual Learning and Teaching Forum | Faculty Of Design & Creative Technologies

3. WHY ENGAGE IN THE STUDENTSVOICE Peer Assisted Teaching Scheme Keynote Annual Learning and Teaching Forum | Faculty Of Design & Creative Technologies

4. Lenses used to enhance teaching practice Insights can be achieved by: • Teachers’ autobiographies as teachers and learners • Colleagues as critical friends • Teachers’ reference to the theoretical literature • Understanding students’ experiences of their learning Diagram sourced from Dr Maureen Bell’s Peer Review presentation as part of the PATS workshops Keynote Annual Learning and Teaching Forum | Faculty Of Design & Creative Technologies

5. Merits in engaging in student voice • “The most fundamental meta-criterion for judging whether or not good teaching is happening is the extent to which teachers deliberately and systematically try to get inside students’ heads and see classrooms and learning from their point of view” [Brookfield, 1995, p35] • Brookfield argues that • There is merit in seeking to understanding students’ experiences of teaching, in support of effective pedagogy and unit management. • simply seeking to understand students experiences of their learning is a key indicator of good teaching • However engagement can only occur when academics seek feedback, acknowledge and act on it Keynote Annual Learning and Teaching Forum | Faculty Of Design & Creative Technologies

6. What is the Peer Assisted TEACHING SCHEME (PATS)? Peer Assisted Teaching Scheme Keynote Annual Learning and Teaching Forum | Faculty Of Design & Creative Technologies

7. What is Peer Assisted Teaching Scheme (PATS)? PATS is a scheme in which academics within a faculty are partnered and follow an informal process to discuss strategies to improve unit quality and develop educational innovations. MENTOR + MENTEE RECIPROCAL PARTNERSHIP • Aims • improve student satisfaction with units • improve the quality of teaching/units • build leadership capacity amongst teachers Keynote Annual Learning and Teaching Forum | Faculty Of Design & Creative Technologies

8. Theoretical underpinning • Benefits of peer assisted learning (PAL)/ peer mentoring programs • Improved academic performance, widening retention • increased cohesion of the student group, reassurance about study concerns and increased confidence and persistence (Green, 2011; Topping, 2001; Boud, 2001, Arendale 2004; Hall et al. 2005) • Applies the principles of social-cultural theory (Vygotsky 1978) • Social interaction plays a fundamental role in the development of cognition • the range of skills one can develop with a knowledgeable other exceeds that can be obtained alone “Zone of proximal development” • Draws on Lave’ssituated learning theory (1988) Learning is constructed in social situations and takes place in an authentic context Keynote Annual Learning and Teaching Forum | Faculty Of Design & Creative Technologies

9. PATS Process Workshops: 1. Interactive Lecturing 2. Planning Your Teaching 3. Peer Observation of Teaching • Deliverables: • Strategy Plan • Summary of feedback • Peer Observation of Teaching • Critical Reflection Keynote Annual Learning and Teaching Forum | Faculty Of Design & Creative Technologies

10. PATS Guides and Workbook • Complete Tasks • - Pre-semester (Tasks1-3) • During semester (Tasks 4-5) • Post semester (Tasks 6-7) monash.edu/pats Keynote Annual Learning and Teaching Forum | Faculty Of Design & Creative Technologies

11. My journey with PATS 2011-2012 2012-current 2008-2010 2010-2011 CADAD Keynote Annual Learning and Teaching Forum | Faculty Of Design & Creative Technologies

12. PATS | USING THESTUDENT VOICE AS A DRIVER TO ENHANCE UNITS Peer Assisted Teaching Scheme Keynote Annual Learning and Teaching Forum | Faculty Of Design & Creative Technologies

13. End of semester unit evaluation survey Unit Evaluation Questions Item 1: The unit enabled me to achieve its learning objectives Item 2: I found the unit to be intellectually stimulating Item 3: The learning resources in this unit supported my studies Item 4: The feedback I received in this unit was helpful Item 5: Overall I was satisfied with the quality of this unit Two open-ended questions: What were the best aspects of the unit? What aspects of this unit are most in need of improvement? Keynote Annual Learning and Teaching Forum | Faculty Of Design & Creative Technologies

14. Better understanding of student concerns What aspects of this unit are most in need of improvement? • Units needing critical attention • 13 ICT units (281 comments) 2010 • 31 ENG, ICT, SCI units (756 comments) Physical science units 2010 • Programming Units • 4 units (477 comments across 4 universities) 2011 (Carbone & Ceddia, 2012; Carbone & Ceddia, 2013; Carbone, Ceddia, Simon, Mason, D’Souza, 2013) Keynote Annual Learning and Teaching Forum | Faculty Of Design & Creative Technologies

15. Course Demographics Note: Overall satisfaction ‘range’ due to multiple campuses Keynote Annual Learning and Teaching Forum | Faculty Of Design & Creative Technologies

16. Keynote Annual Learning and Teaching Forum | Faculty Of Design & Creative Technologies

17. T3 – Setting goals for unit enhancement • Identify 2-4 goals for unit enhancement • What was the driver for your goals? • Produce a strategy plan to address the identified areas • What data will you gather to evidence whether your goal(s) have been met Day & Tosey, 2011 Keynote Annual Learning and Teaching Forum | Faculty Of Design & Creative Technologies

18. Using student voice to set goals for improvement Keynote Annual Learning and Teaching Forum | Faculty Of Design & Creative Technologies

19. EXHIBITING A CULTURE THAT VALUES THE STUDENT VOICE Peer Assisted Teaching Scheme Keynote Annual Learning and Teaching Forum | Faculty Of Design & Creative Technologies

20. T4 – Engage students’ voice early in semester 1. Discuss and plan, how, when, where to engage in student voice 4. Communicate , enact and respond within the same teaching period 2. Engage, invite and collect student feedback Both inaction and action are valid decisions 3. Review and reflect on students comments, and with peer discusses and formulates appropriate responses Keynote Annual Learning and Teaching Forum | Faculty Of Design & Creative Technologies

21. Multi-institutional PATS Trial • Five institutions / 40 units • Monash University (9) • The University of Newcastle (9) • Griffith University (12) • University of Sunshine Coast (3) • Edith Cowan University (7) • Completion of written comments in workbook optional • Task 4 completion in workbook • 24 participants submitted the PATS workbook activities • 22 participants (92%) completed Task 4 Keynote Annual Learning and Teaching Forum | Faculty Of Design & Creative Technologies

22. PATS supports academics to engage, invite and collect the student feedback • Written questionnaire • Informal questionnaire (6) • PATS workbook questionnaire (4) • Existing institutional questionnaire (4) • Modified critical incident questionnaire (1) • Other online questionnaire (1) NOTE “Critical incidents are vivid happenings that for some reason people remember as significant” (Brookfield, 1995,p114) • Written other • Stick note exercise (3) • Stop/ start/ continue (1) • Informal discussion • Inclass (3) • Online (1) Keynote Annual Learning and Teaching Forum | Faculty Of Design & Creative Technologies

23. PATS requires academics to review and reflect on students comments • Collegial support • Clarifies and analyses feedback • Helps focus attention & draw themes Brookfield 1995; Clegg et al . 2002 “Student comments are the start of the journey, not the end” and through review, reflection and response are student comments transformed into engagement with the student voice Cashin, 1994 Keynote Annual Learning and Teaching Forum | Faculty Of Design & Creative Technologies

24. PATS requires academics to communicate act and respond • “Overwhelming… you would prefer that the unit was lecture/tutorial format rather than lecturer seminar. I take this feedback seriously. There are several reasons why we moved to the seminar format … However considering this feedback will we move to …” • “ I was able to gather some aspects which can be improved and which cannot, or are beyond my control as a facilitator… I gave an honest assessment of what I could directly do..” • “One consistent feedback from students about the module is “too heavy and dry” and that …” Keynote Annual Learning and Teaching Forum | Faculty Of Design & Creative Technologies

25. Academics perspectives –engaging the students’ voice • Gathering comments early in the semester allowed them to make improvements to the unit in its current iteration • “ This exercise was worthwhile to conduct at the beginning of the semester as I found myself projecting my voice further and slowing down..” • I was pleased that none of the deltas [negatives] in week one were re-appeared in week twelve • Collecting informal student comments is a quick and effective way to check if the course is on track .. • “The positive feedback … was great… The negative feedback was well received as it was constructive” Keynote Annual Learning and Teaching Forum | Faculty Of Design & Creative Technologies

26. Students appreciation • “It was noted that many students sent positive emails, and gave positive feedback about how valuable such an informal discussion about the course was. Many stated that they were happy that this course improvement forum was undertaken, and wished other courses did a similar feedback forum” Keynote Annual Learning and Teaching Forum | Faculty Of Design & Creative Technologies

27. PATSOUTCOMES Peer Assisted Teaching Scheme Keynote Annual Learning and Teaching Forum | Faculty Of Design & Creative Technologies

28. Positive changes to unit evaluation results Table 1: PATS Pilot in FIT unit evaluation results 2008-2012 (Carbone, Wong and Ceddia2011; Carbone, Ross & Ceddia 2013) Keynote Annual Learning and Teaching Forum | Faculty Of Design & Creative Technologies

29. Focus group sessions – PATS Experience reflection Relationship-building constructive accountability fostering discussions Learning and Teaching Effectiveness encouraging collaborative supportive Sharing ideas review & plan Interpretative guidance communication (Carbone 2011) Keynote Annual Learning and Teaching Forum | Faculty Of Design & Creative Technologies

30. Overall faculty performance Unsolicited Email Hi Ange, ... I just wanted to let you know that all your hard work has paid off. Cheers, Academic Staff member Faculty of Information Technology 12/5/2012 The Faculty was ranked fifth in the university – the highest ranking achieved in the last 5 years; Our unit evaluations have been improving constantly, and we have the lowest percentage of units now in the “needing improvement” category since 2009. Additionally, we have the highest proportion of units considered “outstanding” since 2009. We have the highest response rate (45%) across the University. This should mean our performance data is quite reliable. For many semesters a significant focus of our development efforts have been units that are, in university terminology, "in the red zone". Such units have been given additional support in the form of programs such as PATS. These efforts have paid off and together we have successfully managed to move many units out of this range.... • Faculty of Information Technology • Semester 1 Rankings • 2008 ranked 9/10 (mean 3.61, Uni 3.79)* • 2009 ranked 6/10 (median 3.91, Uni 3.94) • 2010 ranked 7/10 (median 3.93, Uni 3.96) • Faculty of Information Technology • Semester 2 Rankings • 2008 ranked 8/10 (mean 3.68, Uni 3.78)* • 2009 ranked 8/10 (median 3.90 Uni 3.94) • 2010 ranked 6/10 (median 3.95, Uni 3.99) Keynote Annual Learning and Teaching Forum | Faculty Of Design & Creative Technologies

31. Unexpected Outcomes • Mentees and mentors publishing innovations: eg: “conversation sim” • Mentees and mentors submitting grant proposals • Mentees went on to become mentors • Mentors granted promotion • Mentors appointed in positions like Associate Deans (Learning and Teaching), HoS, Directors • Several mentors/mentees nominated for a national award Keynote Annual Learning and Teaching Forum | Faculty Of Design & Creative Technologies

32. PATS WEBSITEhttp://monash.edu/pats Peer Assisted Teaching Scheme Keynote Annual Learning and Teaching Forum | Faculty Of Design & Creative Technologies

33. PATS Website • About PATS (background, partnership types, process and resources) • Impact (changes in unit evaluations, video experiences of participants, outreach, publications) • News and Events (newsletters, events and symposia) • Online workbook (registration, PATS co-ordinator, PATS participants) Keynote Annual Learning and Teaching Forum | Faculty Of Design & Creative Technologies

34. References • Brookfield, S. (1995). "Becoming a critically reflective teacher". San Francisco: Jossey-Bass. • Carbone, A. (2011). Building peer assistance capacity in faculties to improve student satisfaction of units. Proceedings of the 34th Annual International Conference, Higher Education Research Development Society Australasia (HERDSA), Gold Coast, Queensland 4-7 July 2011.  • Carbone, A., Wong, J., Ceddia, J., (2011). A scheme for improving ICT units with critically low student satisfaction. Proceedings of 16th Annual Innovation and Technology in Computer Science Education (ITiCSE'11) Darmstadt, Germany, 27-29 June, 2011. • Carbone, A., and Ceddia, J. (2012) Common Areas for Improvement in ICT Units that have Critically Low Student Satisfaction, Proceedings of the Fourteenth Australasian Computing Education Conference (ACE2012), Melbourne, Australia, 31 January - 3 February, 2012. • Carbone, A. (2012) A Peer Assist Teaching Scheme – A Way of creating, sustaining and developing new connections. Proceedings of the Annual International Conference of the Higher Education Research and Development Society of Australasia (HERDSA 2012), University of Tasmania, Hobart, Tasmania, (in press). • Carbone, A. & Ceddia, J. (2013) Common Areas for Improvement in Physical Science Units that have Critically Low Student Satisfaction, Learning and Teaching in Computing and Engineering, (LaTiCE 2013 ), Macau, 21-24 March 2013. • Carbone, A., Ceddia, J., Simon, D'Souza, D., Mason, R. (2013) Common Concerns in Introductory Programming Courses, Proceeding of the Fifteenth Australasian Computing Education Conference (ACE2013), Adelaide, South Australia, 29 January - 1 February, 2013 • Carbone, A. (2012). Opportunities and challenges faced in attempting to introduce a Peer Assisted Teaching Scheme to improve units with critically low student satisfaction, Higher Education Research Development (in press). • Carbone, A. Phelan, L., Ross, B., Cottman, C., Drew, S., Stoney, S., Lindsay, K., Readman, K. (2013) From taps on the shoulder to PATS on the back: Experiences engaging academics with the Peer Assisted Teaching Scheme, Studies in Higher Education (submitted for review) Keynote Annual Learning and Teaching Forum | Faculty Of Design & Creative Technologies

35. References • Galbraith, C., Merrill, G., Kline, D. (2012) Are Student Evaluations of Teaching Effectiveness Valid for Measuring Student Learning Outcomes in Business Related Classes? A Neural Network and Bayesian Analyses. Research in Higher Education 53:353–374 • Kember, D. and Leung, D. Y. P. (2011). Disciplinary Differences in Student Ratings of Teaching Quality, Research in Higher Education, 52, 3, 278-299. • Lave, J. & Wegner, E. (1991) Situated Learning: Legitimate peripheral participation. Cambridge: Cambridge University Press • Ramsden, P. (1991). A performance indicator of teaching quality: The Course Experience Questionnaire, Studies in Higher Education, 16, 2, 129-150. • Vygotsky, L.S. (1978). Mind in Society. Cambridge, MA: Harvard University Press. Keynote Annual Learning and Teaching Forum | Faculty Of Design & Creative Technologies

36. Questions Associate Professor Angela CarboneDirector Education ExcellenceOffice of the Pro Vice-Chancellor (Learning & Teaching)angela.carbone@monash.edu (03) 9903 4481 PATS Program Officer Ms Joanne Rae joanne.rae@monash.edu(03) 9903 4494 PATS Research Assistant Dr Bella Ross Bella.ross@monash PATS Website: monash.edu.au/pats Keynote Annual Learning and Teaching Forum | Faculty Of Design & Creative Technologies