1 / 42

Webinar Series

Webinar Series. This session is part of e-Assessment Scotland 2013 28 Aug 2013: Assessing algebra concepts - the potentials and limitations of online delivery Robyn Pierce, Caroline Bardini and Kaye Stacey (University of Melbourne, Australia) Your hosts

kitty
Download Presentation

Webinar Series

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Webinar Series This session is part of e-Assessment Scotland 2013 28 Aug 2013: Assessing algebra concepts - the potentials and limitations of online delivery Robyn Pierce, Caroline Bardini and Kaye Stacey (University of Melbourne, Australia) Your hosts Professor Geoff Crisp, Dean Learning and Teaching, RMIT University geoffrey.crisp[@]rmit.edu.au Dr Mathew Hillier, Teaching and Educational Development Institute, University of Queensland mathew.hillier[@]uq.edu.au

  2. Assessing algebra concepts. The potentials and limitations of online delivery Robyn Pierce (Presenter) Caroline Bardini Kaye Stacey University of Melbourne

  3. Australia, Victoria, Melbourne

  4. Presentation authors Robyn Pierce Presenter Kaye Stacey Caroline Bardini

  5. Presentation Outline Pilot project funded by the Australian government’s Office of Learning and Teaching. • Preliminary online mathematics survey –issues of framing items for use online. • Preliminary findings about first year university students' understanding of functions • Examples of  Specific Mathematical Assessmentsthat Reveal Thinking (SMART tests) proposed to identify student thinking and provide feedback and advice to teaching staff.

  6. Project rationale Recent report from Australia’s Chief Scientist (Mathematics, Engineering & Science in the National Interest(Chubb et al, 2012)) The proportion of students at school studying mathematics and science is still declining, especially at the highest level, and in universities is virtually flat. Figure 1.4 Student participation in mathematics, 1995-2010

  7. Project rationale • The need to increase the success rate in the mathematical sciences at first year and improve their sense of mastery and confidence to continue is vital. • “This is not about ‘dumbing down’. It is not in Australia’s interest to produce a raft of people with second rate skills and knowledge.” (Chubb et al, p20).

  8. Previous research has shown that many people who are apparently ‘successful’ secondary school students harbour incorrect or incomplete conceptions of fundamental mathematical ideas. • Through extensive targeted practice in Year 12, students may ‘paper-over’ their fundamental difficulties and become adept at answering examination questions and so proceed to university without robust understanding.

  9. Project rationale • Strategies already developed to support students include Mathematics and Statistics Learning Centres where individual assistance is available to students, especially those in first year. • Diagnostic tests are not conducted at Melbourne because they are currently • too expensive • too time consuming • there is limited opportunity for remediation.

  10. Project rationale Assessment for Learning

  11. Assessment for learning Diagnosing Interpreting & planning Teaching Diagnostic assessment Tutors Content and Curriculum Knowledge Tutor’s Mathematics Pedagogical Content Knowledge Expect university mathematics tutors have strong CK but limited MPCK:

  12. Project details -Stage 1 Aim Develop a currentevidence base for incorrect or incomplete fundamental algebraic conceptions prevalent amongst commencing first year undergraduate mathematics and statistics students.

  13. Project details – Stage 1 Evidence (i) data from all first year undergraduate mathematics and statistics students at the University of Melbourne via a brief survey and timed screening task through the university’s Learning Management System (ii) ‘think aloud’ interviews with volunteer students (iii) interviewing experienced teaching staff

  14. Project details -Stage 2 Make innovative use of technology to produce sample Specific Mathematical Assessments that Reveal Thinking (SMART tests) to add a different dimension to the assessment for learning that has been offered in the past to first year students.

  15. Project details -Stage 3 Produce online feedback, illustrating appropriate teaching strategies and examples for use with students who are identified as having specific misconceptions.

  16. Project details: focus Pilot project limited to algebra. Undergraduates’ misconceptions in two key areas of algebra to be identified: “variables, unknowns and parameters” and “functions and variation”. Resources will target the most common first year “hot spots”.

  17. Focus on functions • Verbalise: define or describe • Recognisefunctions in multiple representations: • Algebraic • Graphic • Numeric • Manipulatefunctions

  18. On-line quiz

  19. Maths quiz via LMS –Advantages • Platform widely used by uni teachers across faculties and subjects (repository of lecture notes, announcements –not exclusive to assessment) • Students are familiar with it (no extra burden on technology skills) • LMS provides direct link to students’ information (subject, etc.) • Variety of types of assessments (anonymous surveys, tests, etc – each with different features) • Export of students’ responses directly to Excel (however in a rather undigestible format…)

  20. Features of our quiz • One attempt only • Limited on time (35 minutes) – most questions to be answered «quickly» (encouraging students’ first ‘reactions’), no calculations should be needed • Track record on number of accesses (with or without attempts), date, time, duration of attempt (‘unaswered’ responses at the end of quiz by a student who spent 35min might indicate a lack of time rather than ‘I don’t know how to do it’)

  21. Disadvantages – item design Maths editor is limited e.g. no simultaneous equations. Need to use Word editor, print screen, crop, save as image and then « attach » (see next slide). Lengthy process!

  22. Maths editor limitations

  23. Disadvantages –item design Quite rudimentary tools. Less friendly environment for students and little flexibility in using multiple features at once (e.g. cannot have multiple choice for different sub-questions – see next slide)

  24. Rudimentary look/tools Originally: (takenfrom SMART test) Multiple choice for eachresponse

  25. Rudimentary look/tools On-line: (LMS)

  26. Pen-and-paper vs online Originally: Wehoped to have multiple choices for eacha,b,c,d (as in e.g. a menu). But limitations forced ‘blanks’.

  27. Pen-and-paper vs online Online: To avoid constructed response (i.e. type into blanks as in pen-and-paper), we chose multiple choice. But this forced to separate each sub-question

  28. Pen-and-paper vs online • Whatwasone question in pen-and-paper has become4! • Mightdiscouragestudents as itaddslengthto the test • Burden on unnecessaryreading(by repeatingsame stimulus 4 times)

  29. Disadvantages – item responses • Althoughexported to Excel, a couple of hourshad to bespent on re-arranging data into a more readable format (yetstill hard to interpret) • Maths editor not available for students’ responses. Variousconsequences: • On design of the item (e.g. Originally : “f(x) = x2-x+2. Find f(f(x))”. Online: multiple choice for the coefficient of x2 in f(f(x))). Relieve students from typing a complex expression and avoid typos. • On the format of the answers collected (see next slide)

  30. Disadvantages – format of answers Need to have a human to look ateachanswer to decide if right or wrong (e.g. assumingtypos, acceptingwholeequation–and not onlyanswer, acceptingnon expandedform, etc.)

  31. Looking back… Even with its limitations, good support for our purposes

  32. Results to date • Misconceptions reported in the literature are still prevalent

  33. Reading, classifying, key word search

  34. Dichotomy – auto-marked

  35. Diagnostic test results and feedback For automatically marked items: • Number correct • Items correct – incorrect • No diagnosis of misconceptions • Advice based on score would be vague

  36. We can do better – but this requires a more sophisticated system • The LMS quiz provided a quick screening tool providing us with evidence of the ‘hot spots’ to target • Develop tests targeting prevalent, incomplete or incorrect conceptions and feedback for teachers and students.

  37. SMART tests: what are they?

  38. SMART tests: example

  39. SMART test: feedback for teachers

  40. SMART test: feedback for teachers Teaching suggestions References

  41. Project – to be done • Development and trialling of SMART tests targeting important ‘hotspots’ for starting University mathematics • still to be completed…

  42. Thank you Further questions? Robyn Pierce r.pierce@unimelb.edu.au Caroline Bardini c.bardini@unimelb.edu.au Kaye Stacey k.stacey@unimelb.edu.au Try yourself at http://www.smartvic.com

More Related