The Use of Technology in Mathematics Support - approaches used and lessons learnt at ITT Dublin.

1. The Use of Technology in Mathematics Support - approaches used and lessons learnt at ITT Dublin. • Noel Gorman • Donal Healy • Martin Marjoram • Ciaran O’Sullivan • Paul Robinson Irish Maths Support Network 4th Irish Workshop on Maths Learning and Support Centres 4th December, 2009 DCU. 

2. Use of Technology in Mathematics Supportat ITTDublin Overview Approaches using technology Use of CALMAT Moodle-based Key Skills Testing in Mathematics Use of MapleTA Use of an E-book Lessons learnt about Promotion of active learning Using technology differently depending on the profile ( age, motivation) of students being supported. Importance of providing cognitive scaffolding to aid students reflect appropriately

3. Mathematics Support at ITTDublin - Background • Mathematics Support at ITT Dublin is very closely aligned with teaching/learning efforts on courses particularly in year 1 because: • Experience of a large % of students in year 1 groups facing significant challenges in Mathematics • Very active ACCS ( part-time) programmes at ITT Dublin with a large % of students who percieve Mathematics as a challenge and hence inhibitior to engaging in third level courses. • Mathematics Lecturing staff as a group: • Encourage active learning • Pro-actively promote structures to allow for the revision/vison of key areas for mature students returning to college • Increasingly promote development of student skills in area of reflection

4. Use of CALMAT • CALMAT(http://www.calmat.gcal.ac.uk), a computer based Mathematics Learning package which ITT Dublin has used for for use for: • Tutorial support for and also to teach entirely all of semester 1 • 02 – 03 and 03 -04 Mechanical Engineering Groups taught through CALMAT • “In 2003 we used CALMAT for two groups, giving 57 students in the 3 groups. For these groups, and for those with 40% or less in Test 1 (31 students), the mean improvement was 22% with a standard deviation of 17%. …………The most comprehensive range of data we have is for the Mechanical Engineering Full Time Certificate at IT Tallaght. If we filter out the students taught through CALMAT, the remainder (420 students) have a mean improvement of only 14%.”(source: Seven Years of First Year Algebra at IT Tallaght M. Marjoram, C. O’Sullivan, P. Robinson, C. Taylor, Proceedings of Engineering Education Conference 2004 Volume 2 – Recruitment and Retention Practice pp 189-194. ) • Worked best for part-time students so used for FLASHE. • To facilitate PRIMER/Semester 1 innovation in Mathematics for the FLASHE programme. ( see later slides) • For visiting second level students.

5. Calculator skills session Diagnostic test PRIMER Mathematics Semester 1 Mathematics Mathematics for FLASHE Bring prospective students back in contact with mathematics in a way which will boost their confidence in their mathematical ability. Use of CALMAT to enable move from ‘sage on the stage’ to ‘guide on the side’ ITT Dublin 2007

6. The Primer Mathematics Course Create a mathematics learning environment which is different to the ‘chalk/talk/ get left behind’ 12 weeks – allow time for students to reflect and practice. Delivered using CALMAT. 2 hours in college and 2 hours at home. Assesssment driven:Mark based on CALMAT assessments and 4 CALMAT based tests( one every 3 weeks). Content:number manipulation, introduction to algebra, graph manipulation. Students complete a reflective diary entry No Credits for this course. ITT Dublin 2009

7. Semester 1 Mathematics Course • Diagnostic test on entry. -High threshold test • Delivered using CALMAT. • 2 hours in college and 2 hours at home. • Assesssment driven: Mark based on CALMAT assessments and 4 CALMAT based tersts( one every 3 weeks). • Sufficient work done and good marks allows module pass without sitting the end of term examination: • 70% of all CALMAT tutorials set • 100% on 24 CALMAT Assessments • Get more than 70% on each of four tests ITT Dublin 2009

8. Cognitive Scaffolding:Reflective Diary Outline Week 1 Date: . Topic/issue being explored:________________________ My reflections……….(high point of the class, low point of the class, my mood, my learning, key points learned….) What have I learned from these reflections? (One thought to carry forward, aspects of learning of topic which are still of concern …….) ( see appendix slides for details on student responses) ITT Dublin 2009

9. Student engagement • Calmat Usage PRIMER Mathematics: • Adult learners seem willing to accept a CALMAT based, assessment driven course. • has the flexibilities they require • seems to address some of the anxiety issues they face coming back to learning mathematics. • Calmat Usage Semester 1 Mathematics: • Full time groups in 2002, 2003, 2004 averaged less than 20 hours use in a 36 hour semester. • The 4 FLASHE groups in years ……. averaged over 39 hours of usage. ITT Dublin 2009

10. Student performance ITT Dublin 2009

11. Mathematics Key Skills – Why? • Students do not bring key knowledge with them from one semester to the next • Students do not master the basics • Students need to refresh their key mathematics knowledge continuously BUT • Students will not concentrate on anything with no marks attached • In later semesters, the basics are not tested directly

12. Mathematics Key Skills – What? • Key Skills consists of • many categories of multi-choice questions • Designed to test material our students MUST be able to do. • Each question comes with immediate feedback and reference to a book chapter and an electronic resource. • the tests draw randomly from particular categories of questions • The tests are Moodle multichoice quizzes • We allow the tests to be repeated several times over a semester • Only a high mark is rewarded with credit • different tests • For different groups and in different semesters. • We test material from earlier semesters that we consider to be “Key Skills” for the current semester.

13. Mathematics Key Skills – What? • Key Skills Tests • Test key skills required for this semester • Randomized tests which can be retaken • High threshold competency based tests • Re-usable questions • Automatically marked • Immediate feedback • Secure • Best mark counts as 15% of module mark See appendix slides for more details on • Tests • Marks • Multichoice Moodle quizzes • Quiz creation Tools and “calculator” Tools • Implementation issues • Key Skills Moodle Site at Tallaght

14. Student Performance 14 ITT Dublin 2009

15. Key Long term question Does frequent Key Skills testing promote deeper learning and lasting knowledge retention in students? All student records are kept in the same location, which will allow us to do a longitudinal analysis of student performance over several semesters. Student performance 15 ITT Dublin 2009

16. Cognitive Scaffolding Time delay on re-takes of tests When students complete a real test they can also review it later and see question feedback BUTstudents may not repeat a test for at least 24 hours, preferably longer. Want students to be active learners - the delay between tests is there to allow students to consider question feedback and review their test attempts. Reflection Sheets Since September 2009 have been piloting a structured reflection sheet to prompt students to identify and record areas in nwhich they need to do revision work ahead of their next test attempt (See next slide) .

17. Keyskills - cognitive scaffolding Reflection Sheets Only being piloted since September 2009 so can’t give definitive evaluation ………but….. seems to be having beneficial effects. To give a flavour included are the 4 reflection sheets for 2 students. Student 1 has marks sequence across 5 tests (and so 4 reflections) of: 8  8  8  13  13 Student 2 has marks sequence across 5 tests (and so 4 reflections) of : 6  6  8  10  13

18. Student 1 Sheet 1(see handout )

19. Student 1 Sheet 2(see handout )

20. Student 1 Sheet 3(see handout )

21. Student 1 Sheet 4(see handout )

22. Student 2 Sheet 1(see handout )

23. Student 2 Sheet 2 (see handout )

24. Student 2 Sheet 3 (see handout )

25. Student 2 Sheet 4 (see handout )

26. Maple TA ( see appendix slides for more detailed commentary) • Web – based, computer package for learning, testing and assessment. • Content designed by instructor – random generation features (particularly suited to mathematics). • Instructor creates “Question Banks” – database of questions – and “Assignments” • “Gradebook” keeps record of students’ results – easily imported to spreadsheet.

27. Some Advantages of Maple TA • Encourages active and independent learning. • Offers more practice/credit for work done. • Immediate feedback for student and lecturer. • Removes/reduces grading. • Lecturer and student “collaborate”.

28. Some Disadvantages of Maple TA • Labour intensive/time-consuming for instructor. • Minimal impact on attendance? • Learning may be superficial. • Computers distract when lecturing. • Unsuitable for large groups?

29. Observations • Student performance • Student feedback

30. Needed to provide a mechanism for Mathematics revision for POTENTIAL mature students wanting to start on the FESCAP delivery mode for the Level 7 Degree in Mechanical Engineering ( ……and unfortunately the 12 week Primer model not an option as they had not started the course yet!!!) Blended solution: 4 week (2 hours a week contact) course in May 2009 based around an e-book based on Foundation Maths Plus MyMathLab, 4/E, by Croft / Davison / Addison-Wesley. The e-book aspect is course o'sullivan76748 can be accessed at the Pearson MyMathLab site http://www.coursecompass.com ( but you need a student access key) E-book 30

31. Chose this e-book approach because it: Had been used by Dr.P Kirwan in WIT successfully Was at the right level for the course Was easily and quickly customisible by lecturer to the material and exercises wanted Provided the practice self testing environment which had been a success for our mature students in CALMAT Provided a ‘contact the tutor facility by e-mail’ to the lecturer. E-book Value Pack consists of Foundation Maths, (4/e by Croft and Davison) and value-added component MyMathLab/MyStatLab Student Access Kit ( Be careful of the 2 platforms!) 31

32. Provides the independent learning aspect in between classes and Also provides the potential for students to continue their work during the summer months before September entry. Advantages for the college: Web based so no need for potential students to have access to college computers, so no registration/ logging-on issues Lifelong Learning Office has a Mathematics Preparation package ready to go to hand/recommend to potential students Cost of the technology transferred to the student Usefulness – Too early to give detailed data but 59 students enrolled , got the value pack and 40% made active use of the e-book aspect with another 17 % making some use of e-book aspect . Students did use the materials during the summer. A small sample of students who had attended the course and now are on year 1 programmes at ITT Dublin were surveyed in December 2009 ex[pressed the view that the e-book combination of traditional text book and web-based exercises had been an invaluable aid to getting them ready. E-book 32

33. Use of Technology in Mathematics Supportat ITTDublin Concluding Comments Lessons have been learnt about: Promotion of active learning Using technology differently depending on the profile ( age, motivation) of students being supported. Importance of providing cognitive scaffolding to aid students reflect appropriately Thanks for listening and ….any questions?

34. Appendix 1: Mathematics for FLASHE course

35. The Primer Mathematics Course Create a mathematics learning environment which is different to the ‘chalk/talk/ get left behind’ environment which the prospective students may have experienced last in their mathematical education. 12 weeks – allow time for students to reflect and practice. Delivered using CALMAT. Students given CALMAT Student Edition so that they can merge home and college records. 2 hours in college and 2 hours at home. 35 ITT Dublin 2007

36. The Primer Mathematics Course (ctd.) • Assesssment driven: Mark based on CALMAT assessments and 4 CALMAT based tersts( one every 3 weeks). • Content: number manipulation, an introduction to algebra, and finally see a topic that will have easy application to the engineering - graph manipulation. • Students complete a reflective diary entry after each CALMAT session. • No Credits for this course: • Idea is to build confidence, allow students to progress at their own pace and instil an element of self directed learning. 36 ITT Dublin 2009

37. Semester 1 Mathematics Course • Diagnostic test on entry. • High threshold test - mark of 80% (was 90%) allows student to skip PRIMER and do the 12 week Semester 1 course. • Delivered using CALMAT. Students given CALMAT Student Edition so that they can merge home and college records. • 2 hours in college and 2 hours at home. • Assesssment driven: Mark based on CALMAT assessments and 4 CALMAT based tersts( one every 3 weeks). • Idea is to build confidence, allow students to progress at their own pace and instil an element of self directed learning. 37 ITT Dublin 2009

38. Semester 1 Mathematics Course (ctd.) • Sufficient work done and good marks allows module pass without sitting the end of term examination: • 70% of all CALMAT tutorials set (148 tutorials comprising 3-4 questions each). • 100% on 24 CALMAT Assessments (which may be retaken and comprise 6-12 questions on a lesson). • Get more than 70% on each of four tests (15-20 questions on groups of lessons). 38 ITT Dublin 2009

39. The Primer Refelctive Diaries Many students noted how useful the first two hour class on using the calculator given to them was. All students recorded their four test scores over the course, and indicated how they felt about it, even though the diary or lecturer made no mention of recording this. All students expressed anxiety at the start of the course, how it was a long time since they did mathematics, that they always found mathematics hard, noting early in their diaries that they must do more at home, expressing concern about how hard the course might be, worrying that other students were further ahead than them. 39 ITT Dublin 2009

40. The Primer Reflective Diaries Many comments on CALMAT or network technical problems. Many comments on CALMAT being too fussy. No students expressed a dislike of the CALMAT class. Most were complementary about not only the software but the structure of the class, for example: “CALMAT tutorials are a better way to learn for me as everything is explained with plenty of worked examples. The tutor is on hand to help when I need – if only school had been like this 20 years ago!” 40 ITT Dublin 2009

41. Appendix 2 Keyskills Extra slides:

42. Mathematics Key Skills – What? Tests Some overlap of material between semesters - want the student to appreciate that, semester after semester, they will be required to know certain things Each test has a partner practicetest All real tests are supervised When students complete a real test they can also review it later and see question feedback BUT: students may not repeat a test for at least 24 hours, preferably longer. we want students to be active learners - the delay between tests is there to allow students to consider question feedback and review their test attempts.

43. Mathematics Key Skills – What? Marks Each student’s best mark in the semester’s Key Skills test counts as half of the continuous assessment for that semester (typically 15% of the module mark) for a 15 question test, a student gets (as a percentage of the module mark) 0% for less than 10 correct. 5% for 10 or 11 correct, 10% for 12 or 13 correct, 15% for 14 or 15 correct

44. Mathematics Key Skills – How? Multichoice Moodle quizzes 45 question categories Covers Semesters 2 to 6 in Engineering about 700 questions and 120 practice questions Quiz creation Tools Detailed quiz creation documentation Quiz creation Tools and “calculator” Tools Key Skills Moodle Site at Tallaght

45. Implementation Key Challenge provide sufficient test opportunities, both in terms of time and lab space. testing opportunities One test opportunity for each group within their own class slot and with their own lecturer taking part After this first test students emailed and texted about further opportunities to do a test outside their normal lecture times.

46. Key Skills Tests – Issues? Tips identify early all free computer lab slots, lab slots that coincide with mathematics classes and all slots that can be used to run a test for each group. Obtain student mobile phone numbers and invest in an sms texting facility from your PC. Email to students is also useful, but is checked less frequently. Involve each group’s mathematics lecturer in the test process. Show them how to log into Moodle and change test times and passwords. Difficulties providing timely warning that a test is available coordinating time, group and test supervisor

47. Dissemination and NDLR Available: all of the material produced to date. complete course with its categories and tests. detailed documentation on Moodle quiz creation and quiz management. Each category as a learning object, together with any files used to help create such questions and upload instructions to Moodle.

48. Appendix 3 notes on MAPLE TA

49. Appendix 3 MapleTA Notes on MapleTA: Maple TA is a web-based package for testing and assessment, which can be accessed remotely off-site or used in regular class time. Instructors create their own content, consisting of a database of questions, together with assignments, exercises and tests, which students access for practice and credit. The software allows the inclusion of parameters whose values are randomly-generated, thereby giving different versions of the same basic question. The package is particularly suited to mathematics since it is built around the Maple symbolic manipulation program, but the variety of question types gives it a general applicability. A feature of Maple TA is the presence of the “Gradebook” which keeps records of all results obtained by each student for each attempt, each question and each exercise and this information is readily transferrable to a spreadsheet for processing.

50. Appendix 3 ctd Some Advantages of MapleTA: Exercises on Maple TA, interspersed with course content, provide a way of encouraging students to engage with the course material and learn actively, rather than to remain passive in a traditional lecture. Independent learning is also facilitated by the opportunity to repeat/revise previously attempted exercises and.by the possibility of remote access to assignments, outside normal class times. Students can be given credit for all of the work they undertake on the system and can, in principle, practice as much as they need.  When students complete an exercise, or a test, the can immediately grade their work and obtain instant feedback on their performance. The instructor allows has access to this information and can assess individual, or class, progress. Because the computer does the tedious work of marking, the instructor can reduce, or eliminate, grading work. The system allows the instructor to grade students’ work by hand in cases where it would be advantageous/appropriate to do so. A welcome bonus to using Maple TA is that students sometimes come to see the instructor as a collaborator helping them to negotiate the hurdles presented by the computer, improving the instructor-student dynamic and fostering a more cooperative environment.