eVALUAte: development and validation of a teaching survey

1. eVALUAte: development and validation of a teaching survey Beatrice Tucker, Beverley Oliver and Julie-Ann Pegden

2. eVALUate surveys for student feedback eVALUate unit survey (for all current students enrolled in course work units) eVALUate teaching survey (enables students to give feedback to individual teachers on their request)

3. Literature Review • Systems for Measuring Teaching Quality – controversial, validity is questionable (Pounder 2007, Kulik 2001, Kember & Leung 2008, Remedios & Lieverman 2008, Burdsal & Harrison 2008) • Used for different purposes • teacher development, promotion, awards • for improving quality of courses, quality audits, benchmarking, funding allocation • student choice of study (Tucker, Jones, Straker, & Cole, 2003, Spencer & Schmelkin, 2002, Ballantyne, 2003; Ory, 2000, Department of Education Science and Training, 2004, Carey & Gregory, 2003) • Students and teachers have differing opinions about what they perceive to be good teaching (McLean 2001)

4. eVALUate surveys at Curtin eVALUate unit survey – for unit review, annual and comprehensive course review, student choice of units, benchmarking, promotions eVALUate teaching survey - for teacher reflection and development, promotions, awards

5. Development of survey • 2003 – committee • students • teaching academics • Deans of teaching and learning • academics with expertise in survey design and evaluation systems (statisticians) • Literature was reviewed • excellent teacher characteristics, current instruments used • Unit and teaching items included in one survey Oliver, Tucker, Gupta, & Yeo (2008)

6. Method - focus group de Bono's Six Thinking Hats Students Managers Teaching academics

7. Method - item development Characteristics of a good teacher

8. eVALUate teaching survey • Appears knowledgeable in this subject areaThe teacher seems to have a good understanding of the subject. • Is enthusiastic in teaching this unitThe teacher makes the subject interesting and conveys his or her enthusiasm for the subject. • Is well organisedThe teacher has material prepared on time, is punctual and structures activities in ways that help learning. • Communicates clearlyThe teacher is easy to understand in face-to-face, online, written and other formats and explains concepts clearly. • Is approachableThe teacher encourages students to ask questions and seek help. • Provides useful feedbackThe teacher provides timely and helpful feedback so you can learn. • Is an effective teacherOverall, this teacher helps you to learn.

9. eVALUate teaching survey 8. Please comment on [the teacher]'s teaching strengths. 9. Please comment on how you think [the teacher] might improve the teaching and learning in this unit.

10. Testing and statistical analysis • Piloted (x4) to student groups in 2003-4 tested for face validity • Pilot 4 - Feedback from teaching staff and students Surveys separated • Trialled to all Faculties in 2006 • Factor analysis and Rasch analysis – tested the categorical scale and psychometric properties of the items (instrument measures multiple teacher attributes and not just one dominant trait)

11. How students give feedback

12. Uptake by teachers

13. Teaching surveys drive unit survey response rates *significant difference p<0.001

14. Conclusion • All teaching staff are required to conduct a teaching survey annually • Teaching surveys help drive unit response rates • Teaching eVALUate reports are used to reward staff: for promotion and awards