Supplementing lectures with additional online materials

1. Supplementing lectures with additional online materials Matthew Juniper, CUED June 2007

2. Introduction Lecture

3. Introduction Lecture Work on examples papers

4. Introduction Lecture Supervision Work on examples papers

5. Introduction Additional materials available on-line Lecture Supervision Work on examples papers

6. Effectiveness study 2005-06

7. Effectiveness Study Audio-visual concept descriptions Textual concept descriptions Hint 1 (usually text) Additional materials available on-line Hint 2 (usually text) Work on examples papers

8. Source : Survey data Student feedback STUDENTS' PERCEPTION OF THE USEFULNESS OF THE ADDITIONAL MATERIALS Overall, how useful were the additional materials for each question? Incompressible Flow Boundary Layers Very Fairly mean mean A bit Not much Not at all Question Number Question Number

9. Source : Survey data Student feedback STUDENTS' PERCEPTION OF THE USEFULNESS OF THE ADDITIONAL MATERIALS Overall, how useful were the additional materials for each question? Incompressible Flow Boundary Layers Very A-V Textual Audio-visual Textual Fairly mean mean A bit mean Not much Not at all Question Number Question Number

10. Source : Survey data Student feedback STUDENTS' PERCEPTION OF THE USEFULNESS OF THE ADDITIONAL MATERIALS Overall, how useful were the additional materials for each question? new concepts introduced Incompressible Flow Boundary Layers Very A-V Textual Audio-visual Textual Fairly mean mean A bit mean Not much Not at all Question Number Question Number

11. Source : Survey data (37 student responses, 18 supervisor responses) Student and supervisor feedback STUDENTS' AND SUPERVISORS' OPINIONS ON THE EFFECT OF THE AMs Do the AMs hinder or aid students' ability to answer the question? Do the AMs hinder or aid discussion in the supervision? Do the AMs hinder or aid students' understanding of the concepts? Strongly aid Strongly aid Strongly aid supervisors students students supervisors students supervisors No effect No effect No effect Strongly hinder Strongly hinder Strongly hinder

12. Source : Survey data Student and supervisor feedback COMMENTS ON THE EFFECT OF AMs ON STUDENTS' ABILITY TO ANSWER THE QUESTIONS Students Supervisors Getting started: Getting started: "Helped to get started on the subject." (IF) "The AMs usually helped because students often seemed to get stuck at the first hurdle." (IF) "I would not even know where to begin were it not for the hints" (BL) "The students found the AMs useful, especially when they didn't know how to start the question." (BL) Directing thought processes: "Both students specifically said that they would not have been able to answer the questions without the AMs" (BL) "Visualization of the problem in Q16 was slightly easier." (IF) "The hints were generally at a good level – they helped but didn't do it all for you." (IF) Directing thought processes: "The AMs for Q11 have been very useful as they clarify how students should work out this question." (BL) "The AMs helped direct my thinking when the questions were written vaguely." (BL) Getting further: Getting further: "The student that used the AMs was able to get through the material faster than the other student" (IF) "Mostly very well thought out, with the level of help given just right. I still got stuck a couple of times but that is what supervisions are for." (IF) "One student submitted attempts with and without the AMs, getting much further through the questions by using them." (BL) "The students who used the AMs could get a lot further in the questions. Those who did not use them despaired and gave up, expecting to be told all in the supervision." (BL) "I would have got nowhere in Q4 without the AMs but in the end I still got a lot of it wrong. Half way answers would really help." (BL) "One of the students had used the AMs and was able to attempt all the questions, unlike the other student." (BL) How to use the AMs "I looked at the AMs after completing the questions in order to consolidate." (IF) Specific comments "It was especially useful for Q4" (BL) "I found it useful to look at the hints after I had got stuck on a question." (BL) "The AMs are most useful for the latter half of the BL questions [these are the hardest questions]" (BL) "Particularly useful for Q12" (IF)

13. Source : Survey data Student and supervisor feedback COMMENTS ON THE EFFECT OF AMs ON DISCUSSION IN THE SUPERVISION Students Supervisors Focus discussion: Focus discussion: "Having all looked at the materials we could generally take a more unified approach." (BL) "The AMs gave me a better understanding so I was better prepared for the supervision and could go into more depth." (IF) "The students who were fuzzy on the Coriolis force had a better understanding of Q11 having used the AMs. This made discussion easier." (BL) "Very helpful for discussion in supervision and understanding principles but I still got stuck on the maths." (IF) Save time: "For Q4 the video was useful for targeting discussion towards at an area that I had not understood." (BL) "For students who put in less time the AMs can complete my explanations – very useful." (IF) "Less time was required for going over answers so more time was available for discussion." (IF) "In the supervision the AMs provided a discussion point and we referred to them a lot when going through Q4" (BL) "We spent less time grinding through the questions" (BL) "We were starting from a point further along than we otherwise would have done." (BL) Problems: "In supervisions I conduct some spot checks to check that students understand. These are similar to the AMs so now I don't know whether they really understand or have simply seen it already on the AMs." (IF) "On one question they didn't need me to point out the challenging issue. However, since they had figured out the question themselves, I couldn't use it to reinforce the concept." (IF) "It is slightly more difficult to start discussion because students get there without the supervisor's help." (IF) "Only one student had used the AMs, which resulted in an unbalanced supervision." (BL) "They didn't ask me anything related to what they saw on the additional materials." (BL)

14. Source : Survey data Student and supervisor feedback COMMENTS ON THE EFFECT OF AMs ON STUDENTS' CONCEPTUAL UNDERSTANDING Students Supervisors General: Process: "The AMs helped because I could replay the audio-visual presentations if I didn't get the concepts right in lectures." (IF) "The BL topic is one of the most difficult and the AMs act like a tutor for students while they are doing the problems." (BL) "The AMs were generally good at explaining the concepts behind the questions but on some questions I still could not see how to apply the concepts, even after reference to the AMs." (IF) "The AMs aid with conceptual understanding because the students do not get as bogged down in trying to solve questions and so have time to ponder the concepts." (BL) "Audio-visual aids very helpful for understanding concepts" (BL) "The concepts come from discussion in class. The AMs tend to focus on mathematical developments." (BL) "The AMs provided extra insight into particular issues that may not have been covered in lectures." (BL) "The excellent AV concept descriptions are a good remedy for the difficulty of the BL course." (BL) Specific: "The explanation of complex transformations was useful so that this concept could be fully clarified by the supervisor." (IF) "They seemed to understand the concepts even without using the materials." (IF) "The audio-visual concept description for Q9 was very helpful." (IF) "For Q12 there was a good visualization of the transformation from one plane to another." (IF) "Conceptual descriptions for Q4 and Q7 were particularly helpful [these are audio-visual]." (BL)

15. Source : Survey data Student and supervisor feedback GENERAL COMMENTS ON THE AMs Students Supervisors Audio-visual AMs: Supervisors' use of AMs: "I found the audio-visual concept description the most helpful part of the AMs. It was good that I could replay the presentation over and over again." (IF) "I use them to look for useful ways to teach the concepts" "I take a look to make sure that my explanations and answers are fully correct." "Audio-visual descriptions very helpful" (BL) "I used the AMs to obtain better understanding of question 4" (BL) "Audio-visual hints very useful indeed" (BL) "I always use the AMs for (a) a good route through the questions (b) good diagrams and ways to teach the theory and (c) extra understanding of any questions I found hard." (BL) "I found the audio-visual presentation particularly helpful because I could replay it on demand." (IF) Comments on the adverse effects of AMs "The hints were only useful for one question. They could do the rest without any difficulty." (IF) "Hints for Q7 confused me. My supervisor's explanation was much simpler" "I try to avoid using the AMs and treat them as a last resort. It was difficult to avoid being tempted to use them more than necessary. From what I have seen so far [first 11 questions of IF course] I would not invest in expanding the system."

16. Conclusions and implications Conclusions Additional materials are most useful when: - they are audio-visual rather than textual Implications - Reduce AV production time so that all additional materials can be presented audio-visually - Reduce reliance on other people to produce the additional materials

17. Conclusions and implications Conclusions Additional materials are most useful when: - they are audio-visual rather than textual Implications - Reduce AV production time so that all additional materials can be presented audio-visually - Reduce reliance on other people to produce the additional materials - a new concept is introduced - Use additional materials to explain the most difficult concepts, perhaps in a different way to that in lectures

18. Conclusions and implications Conclusions Additional materials are most useful when: - they are audio-visual rather than textual Implications - Reduce AV production time so that all additional materials can be presented audio-visually - Reduce reliance on other people to produce the additional materials - a new concept is introduced - Use additional materials to explain the most difficult concepts, perhaps in a different way to that in lectures - a question is particularly difficult - Create challenging questions and back them up with additional materials

19. Conclusions and implications Conclusions Additional materials are most useful when: - they are audio-visual rather than textual Implications - Reduce AV production time so that all additional materials can be presented audio-visually - Reduce reliance on other people to produce the additional materials - a new concept is introduced - Use additional materials to explain the most difficult concepts, perhaps in a different way to that in lectures - a question is particularly difficult - Create challenging questions and back them up with additional materials Additional materials are not useful when: - questions are on familiar concepts - Don't use additional materials for simple questions

20. Conclusions and implications Conclusions Additional materials are most useful when: - they are audio-visual rather than textual Implications - Reduce AV production time so that all additional materials can be presented audio-visually - Reduce reliance on other people to produce the additional materials - a new concept is introduced - Use additional materials to explain the most difficult concepts, perhaps in a different way to that in lectures - a question is particularly difficult - Create challenging questions and back them up with additional materials Additional materials are not useful when: - questions are on familiar concepts - Don't use additional materials for simple questions - Do worked examples that are similar, but not identical to, the questions