potential biases in student ratings as a measure of teaching effectiveness l.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
Potential Biases in Student Ratings as a Measure of Teaching Effectiveness PowerPoint Presentation
Download Presentation
Potential Biases in Student Ratings as a Measure of Teaching Effectiveness

Loading in 2 Seconds...

play fullscreen
1 / 22

Potential Biases in Student Ratings as a Measure of Teaching Effectiveness - PowerPoint PPT Presentation


  • 308 Views
  • Uploaded on

Potential Biases in Student Ratings as a Measure of Teaching Effectiveness Kam-Por Kwan EDU Tel: 2766 6287 E-mail: etkpkwan Your beliefs about student ratings

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Potential Biases in Student Ratings as a Measure of Teaching Effectiveness' - elina


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
potential biases in student ratings as a measure of teaching effectiveness
Potential Biases in Student Ratings as a Measure of Teaching Effectiveness

Kam-Por Kwan

EDU

Tel: 2766 6287

E-mail: etkpkwan

your beliefs about student ratings
Your beliefs about student ratings
  • Answer the 10 questions on the worksheet by stating whether you think each of the statements is ‘True’ or ‘False’. (Click here for the questions in PDF format)
  • There is no right or wrong answer. The main purpose is to find out more about your own beliefs and views about student ratings of teaching.
uses of student ratings
Uses of student ratings
  • As a measure of student perceptions or satisfaction about the teaching
  • As a measure of teaching effectiveness of the instruction
  • As a measure of teaching effectiveness of the teacher
validity concerns about ratings
Validity concerns about ratings
  • Do ratings accurately reflect the perceptions and satisfaction of the students?
  • Do ratings accurately reflect the teaching effectiveness of the instructional context?
  • Do ratings accurately reflect the teaching effectiveness of the teacher?
ratings as student perceptions
Ratings as student perceptions
  • Biases exist if the ratings fail to measure accurately what students really feel about the teaching
    • Little dispute in the validity of ratings for this use
ratings as te of the instruction
Ratings as TE of the instruction
  • Biases exist if ratings are influenced by factors unrelated to teaching effectiveness
    • Lower ratings for larger classes  no bias
    • Lower ratings for ‘better-looking’ teachers  existence of bias
ratings as te of the teacher
Ratings as TE of the teacher
  • Biases exist if ratings influenced by factors beyond the control of the instructor
    • Higher ratings for more emphatic teachers  possibly no bias
    • Higher ratings for teachers of smaller class  potential bias or unfairness
criticisms of student ratings
Criticisms of student ratings
  • Students cannot make consistent judgments about teaching
  • Student ratings are popularity contests
  • Students cannot make accurate judgments until they have graduated
  • Student ratings are unrelated to amount of learning
  • Staff and students disagree on what constitutes good teaching
some more criticisms
Some more criticisms
  • Student ratings are influenced by
    • time and day of the teaching
    • class size
    • level of the course
    • rank of instructor
    • nature of the course: required or elective
    • difficulty level of course / assignments
    • expected grades
    • disciplinary differences
what research evidence is there
What research evidence is there?
  • Over 70 years of research
  • More than 2000 studies
  • A huge body of research evidence
  • Some of the criticisms are valid, some are not
ratings are reasonably reliable
Ratings are reasonably reliable
  • Well-constructed student rating forms are highly reliable (alphas in the 0.8 / 0.9 range)
  • Ratings are stable over time (r > 0.8)
  • High correlation exists between ratings of same instructor and course (r = 0.7 to 0.89)
ratings are reasonably valid
Ratings are reasonably valid
  • Staff and students generally agree on the important dimensions of good teaching
  • Student ratings are moderately correlated with achievement (r = about 0.5)
  • Student ratings correlate moderately with alumni ratings, classroom observations, and self-evaluation by staff
  • Distinguishable ‘profiles’ of teaching can be revealed from student ratings
course and teacher effects
Course and teacher effects
  • Small associations are found between student ratings and the following factors:
    • class size (ratings >for smaller classes)
    • level of course (ratings >for higher-level courses)
    • nature (ratings >for elective courses)
    • discipline (languages & art > social sciences > engineering & science)
    • rank (ratings >for higher-rank teachers)
course and teacher effects 2
Course and teacher effects (2)
  • Inconsistent / no associations are found between student ratings and:
    • gender of instructor
    • day and time of the course
student factors
Student factors
  • Small associations found between student ratings and:
    • prior subject interest (ratings >for higher prior interest)
    • students’ major (ratings >for major)
    • perceived workload / difficulty (ratings >for higher workload / more difficult courses)
    • expected grades (ratings >for higher grade)
overall effect
Overall effect
  • About 15 to 20 percent of the variation in ratings can be explained by the combined effects of the background (course, teacher and student) variables
ratings are basically unbiased
Ratings are basically unbiased
  • The existence of course, teacher and student effects on ratings generally support rather than refute the validity of student ratings as a measure of teaching effectiveness
  • In most cases, the effects are quite small
ratings can be unfair
Ratings can be unfair
  • Teacher evaluation based on raw student ratings can be ‘unfair’ to individual teachers because:
    • a lot of factors affecting teaching effectiveness are outside the control of the teacher (although the effects are quite small)
    • there are big differences in the context in which different teachers operate
conclusion
Conclusion
  • Student ratings are generally valid as a measure of student perceptions/ satisfaction
  • Student ratings are reasonably valid as a measure of teaching effectiveness of instruction
  • Student ratings can be unfair when they are used for making personnel decisions concerning individual teachers
implications
Implications
  • Student ratings are useful for
    • understanding student satisfaction and perceptions
    • improving teaching
  • Student ratings must be interpreted and used cautiously and in context when used for making judgements and personnel decisions about individual teachers
references
References
  • Aleamoni, L. M. (1987). Student rating myths versus research facts. Journal of Personnel Evaluation in Education, 1, 111-119.
  • Feldman, K.A. (1996). Identifying exemplary teaching: using data from course and teacher evaluations. New Directions for Teaching & Learning, 65. Jossey-Bass Publishers.
  • Marsh, H.W. (1987). Students’ evaluation of university teaching: research findings, methodological issues, and directions for future research. Int. J. Edu Res., 11, 305-329.
answers to t f questions
1. F

2. F

3. F

4. T

5. F

6. F

7. T

8. F

9. T

10. T

Answers to T/F Questions