Teaching @IITB – Some Data

1. Teaching @IITB – Some Data Institute Faculty Meeting Indian Institute of Technology Bombay, Mumbai February 10, 2010

2. Teaching @IITB Teaching • Important activity • Less ‘discussed/tracked’ compared to research • Data, data every where! • Data? Number game here too? • Looking at data can help • If we start looking at the data, what we collect and the process of collecting it will improve.

3. Teaching @Aero –Survey by Students

4. Teaching – Measures? • Quality. How well?  Student feedback? • Influenced by liberal grading • Senior students evaluate stringently • Teach less, teach well • Etc.

5. Teaching @Aero • 5 Years, 10 Semesters, 22 faculty • Number of courses • Total offered = 293 • Not evaluated = 77 (26%) • Each course has • Credits 4, 6 or 8 • Taught by 1, 2 or 3 faculty • Average grade awarded 0 to 10 • Average student evaluation 0 to 100

6. Teaching @Aero – Evaluation Vs Grading? Correlation Coefficient = 0.28 Average grade = 6.77 Average evaluation = 75.9

7. Teaching @Aero – Sr Students Evaluate Stringently!

8. Teaching @Aero – Load Each ‘’ represents one faculty

9. Teaching @Aero– Quantity Vs QualityData for 2002 to 2006 Each ‘’ represents one faculty

10. Teaching @AeroEvaluation Vs Class Strength! Larger the class, tougher to get good evaluation

11. Teaching Data of IITB 1999-2007

12. UG – Grades & Evaluations

13. PG – Grades & Evaluations

14. UG+PG Teaching Load

15. Trends in Grading

16. I used to think! “Academic standards have fallen. Students are not as good as they used to be, etc” Underlying assumption  students are the problem See the data 

17. Over the years! Dept A : Course-1 CS-152

18. Over the years! Dept A : Course-2 CS-207

19. Over the years! Dept B : Course-1 This course presents a trend of reducing grades. But if this one data point is discarded then we have a flat variation CS-212 (EE)

20. Over the years! Dept B : Course-2 EE-002

21. Over the years!Dept C : Course-1 AE-152

22. Over the years! Dept C : Course-2 AE-330

23. Several comments come to mind • Quality of data • What additional data must be collected • What can be done with the data • But, a more thorough study required • We must pay more attention to these things

24. Publications in the area of Education1999-2009

25. Publications in the area of Education1999-2009

26. Publications in the area of Education1999-2009

27. Publications in the area of Education1999-2009

28. Publications in the area of Education1999-2009 • > 160 Journals covering education • 35 Journals covering engineering education

29. Several Initiatives Worldwide • National Academy of Engineering, USA is concerned about “Educating the Engineer of 2020” • Interventions recommended & tried out • FYEP (First Year Engineering Projects) • Purdue EPICS Project in experiential learning • Etc. • CDIO - Educational framework for producing the next generation of engineers. (Conceiving, Designing, Implementing, Operating real-world systems and products).

30. We need to take teaching lot more seriously Thank You

31. Extra slides

32. Some Suggestions • Validate student evaluations with registration details before accepting • Capture data on faculty status ‘Lien’, ‘Sabbatical’, ‘Not teaching this sem’, etc. Above data on ‘Faculty man semesters’ cannot account for those who are in the department but do not teach. • Enable logging of unequal sharing of courses by faculty (ie. If 2 faculty are sharing a course presently they get credit of 0.5 each)

33. UG Teaching Load * Courses are normalized to 6 credit courses # Only a sub-set of the dept faculty may be involved in UG Teaching. This is the average over the faculty who are involved in UG teaching

34. PG Teaching Load * Courses are normalized to 6 credit courses # Only a sub-set of the dept faculty may be involved in PG Teaching. This is the average over the faculty who are involved in PG teaching

35. Teaching Data 1999-2007 This summary based on teaching data for 9 years is presented with following comments • Study is more to see what data can ‘tell’ • Since the data may not have been captured with a view to use it thus, we may have to tighten the processes to correctly capture the data • Some observations & suggestions have also been made

36. Some Suggestions/Recommendations • Max & Average feedback for a course to be intimated to faculty designated for a course • Course evaluation to be done for all courses. • Capture un-equal sharing of course load • Need to log summer courses • 21 Qs in Course Evaluations?

37. Grades, Evaluations : Some Observations • In all departments many courses have gone without getting evaluated. • Most departments have shown more than one course that has got evaluated by more students than are registered for it. To be looked into! • Civil has highest average grade for both UG & PG with least standard deviation. Metallurgy comes next. Summary of data 

38. Teaching – Histograms No of courses that had class strength between 0 to 5

39. Teaching – Histograms Average grade = 6.77 How does this compare across departments? Aero students find our grading very stringent.

40. Teaching – Histograms Average evaluation = 75.92

41. Teaching – Course / Student Load Each ‘’ represents one faculty Average Courses/faculty/sem = 1.25 Average students/course = 30 (normalized to 6 credit course)

42. Teaching Quality - Comparison • Same course • Wide variation across faculty • Less variation for same faculty

43. Some Suggestions/Recommendations • Max & Average feedback for a course to be intimated to faculty designated for a course • Course evaluation to be done for all courses. • Targeted evaluation > 70 • Capture un-equal sharing of course load • Need to log summer courses • 21 Qs in Course Evaluations too much?

44. Detailed study of teaching related data planned Data available 1999 onwards Student evaluations 2002 onwards

45. Some Data! • Numbers, numbers  Quality? • Numbers alone not sufficient . . . • Numbers may be necessary indicator . . . • Assorted data collected over 2-3 years • Aim • Not to judge anyone /anything • Not to make a point • Data can help if captured thoughtfully and processed with care • We must also start talking about teaching! It is important.

46. i = 1, Nc Teaching – Across Faculty Each faculty  • Nsem = no of semesters taught ≤ 10 • Nc = total courses taught • Ci = credits, • Fi = 1.0 not shared, = 0.5 shared with another • Ni = no of students registered • Ei = Evaluation by students

47. Teaching – Some Indices Teaching load related • Equi 6 Cr courses, Nc6 = (Nc Fi Ci )/ 6 • Avg courses/sem, Nc-sem = Nc6 / Nsem • Avg students/course, Ns-c = (Nc Fi Ci Ni )/ (Nc Fi Ci ) = (Nc Fi Ci Ni )/ (6 Nc6) • Avg students/sem, Ns-sem = Ns-c * Nc-sem

48. Teaching – Some Indices Evaluation related Avg evaluation, E = (Nc Fi Ci Ni Ei)/(Nc Fi Ci Ni ) = (Nc Fi Ci Ni Ei)/(6 Nc6 Ns-c)

49. i = 1, Nc Teaching – Across Faculty Each faculty  • Nsem = no of semesters taught ≤ 10 • Nc = total courses taught • Ci = credits, • Fi = 1.0 not shared, = 0.5 shared with another • Ni = no of students registered • Ei = Evaluation by students

50. Teaching – Some Indices Teaching load related • Equi 6 Cr courses, Nc6 = (Nc Fi Ci )/ 6 • Avg courses/sem, Nc-sem = Nc6 / Nsem • Avg students/course, Ns-c = (Nc Fi Ci Ni )/ (Nc Fi Ci ) = (Nc Fi Ci Ni )/ (6 Nc6) • Avg students/sem, Ns-sem = Ns-c * Nc-sem