Can a computer marked exam improve retention
This presentation is the property of its rightful owner.
Sponsored Links
1 / 33

Can a computer-marked exam improve retention? PowerPoint PPT Presentation


  • 43 Views
  • Uploaded on
  • Presentation posted in: General

Can a computer-marked exam improve retention?. Jon Rosewell Communication & Systems, MCT EATING, 21 st June 2012. A tale of several short modules. Based on Relevant Knowledge, 10 point, 10 week, level one modules with mid-course CMA and final EMA A module with poor completion

Download Presentation

Can a computer-marked exam improve retention?

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Can a computer marked exam improve retention

Can a computer-marked exam improve retention?

Jon Rosewell

Communication & Systems, MCT

EATING, 21st June 2012


A tale of several short modules

A tale of several short modules

Based on Relevant Knowledge, 10 point, 10 week, level one modules with mid-course CMA and final EMA

  • A module with poor completion

  • A module that satisfies student needs

  • A module that is a paragon of good practice

    Retention = module completion (not pass)

    Progression = continue to study another OU module


A module with poor completion

A module with poor completion

  • The mid-module CMA is low-stakes and easy to submit

    • ~80% students submit mid-course CMA

  • The final EMA is a project, submitted via eTMA system

    • Only 50% students submit final EMA

  • Many new students do not continue to study

  • Very few students subsequently complete a qualification


A one off module

A ‘one-off’ module

  • Most new students want this module, not a qualification

  • Most new students won’t register for a further module

  • Many students therefore see no need to stress over EMA

    • Only 50% of students submit EMA

  • Most students complete the course material

    • ~80% complete CMA

    • Engaged on forums


Can you guess what it is yet

Can you guess what it is yet?

  • A module with poor completion

    • T180 Living with the net

  • A module that satisfies student needs

    • T180 Living with the net


Does one off poor retention

Does ‘one-off’  poor retention?

An ‘M-world’ module

  • T180 Living with the Net (2004-2006) as an example

  • Some studying as ‘one-off’

  • Students enjoyed and completed course but daunted by EMA

  • Students who submitted nearly all passed – too late!

  • Not submitting EMA not completing module poor retention statistic (51%) HEFCE penalty

  • Setting up students to fail?


Can a final cma improve retention

Can a final CMA improve retention?

  • Submission of CMA generally high (T184: 87%, n=2158)

  • iCMA encourages engagement and motivation

  • Not as daunting as typical EMA

  • So add final CMA(CME) and reduce EMA

  • CME is a component of final assessment submission = course completion

    Apologies for the acronym nightmare!


Can an icme improve retention

Can an iCME improve retention?

  • Moodle quiz

    • Same format as mid-course iCMA

    • Open at start of course; single submission by traditional cut-off date

    • Questions can be revisited

    • Question level feedback immediately after cut-off date (cf OU anodyne performance profile 3 months later…)

  • Even without adaptive feedback, learner can judge their own performance

    • Builds confidence

    • Lowers barrier to pass: relatively low score on EMA programming and essay questions sufficient to pass


T184 robotics and the meaning of life a practical guide to things that think

T184 Robotics and the meaning of life:a practical guide to things that think

  • Relevant Knowledge

  • 10-point, 10-week, 100 hours

  • 2003 – 2011, 2387 students

  • Components:

    • Website

    • OU-RobotLab practical

    • software with LEGO kit or simulator

    • Isaac Asimov: I, Robot

    • James May on DVD

    • Moderated, peer-support forums


T184 assessment strategy

Original

10% – mid-course iCMA

90% – final written EMA

Part I – short answer qns

Part II – prog & essay qns

Threshold:

40 marks on EMA

New

10% – mid-course iCMA

30% – final iCME

60% – final written EMA

prog & essay qns

Thresholds:

0 marks on iCME

0 marks on EMA

T184 assessment strategy


Hypothesis

Hypothesis:

  • Students will attempt iCME first, before EMA

     They will thus complete the module

  • This will improve their confidence

  • They will then be motivated to attempt EMA

    They will thus pass the module


Did more submit icme

Did more submit iCME?

T184 2011E & 2011J presentations


Did students submit icme first

Did students submit iCME first?

T184 2011E & 2011J presentation


Can a computer marked exam improve retention

T184 2011J


Can a computer marked exam improve retention

T184 2011J


Interviews with students

Interviews with students

I thought the balance of assessment was about the best I’d come across. Interesting computer-based tests … plus the programming elements … an interesting [EMA] with a choice of two essays. … The assessment contributed to the enjoyment of the whole course.

By the time I’d finished the assessment I’d felt I’d learnt more. … I also felt fairly confident that I’d got enough marks on the computer based test.


Interviews with students1

Interviews with students

[Early on] get an achievement and get a bit of success, … it was really easy that first CMA. … So an early success and then you know at the end you’ve got some short answers in the CMA that you think pick up your score a bit, and then you’ve got the EMA where you can do even better. I think it’s spot on.


Interviews with students2

Interviews with students

… as long as they feel it’s rigorous … it didn’t have that kind of ITV quiz thing, you know ‘what’s the capital of England’ or something. … There was a combination with ‘you’ve done this bit now, and now you need to tell us about it, now you need to describe it, now you need to do some programming and show us your work’. It felt like there was real rigour to it so it didn’t feel like ‘this is a quick and dirty kind of interactive quiz’.


Interviews with students3

Interviews with students

Student:I think it was excellent. … The only thing I don’t like is having to wait till September for my result!

Me: So that’s another reason for introducing that computer marked exam … it gives students feedback immediately rather than a long delay.

Student:Yes, so you know where you’ve gone wrong. That’s really good, and it cheers you up as well, not an idiot after all!


Interviews with students4

Interviews with students

“I enjoyed all of it really. I thought the course materials were brilliantly laid out, I thought it was very clear, concise. It gave you enough about the subject to make you interested and wanted you to do more.”

“As I say it was a pleasure doing it. I hope I’ve passed, but I just think it was a joy doing it, it was really fun, you learnt something, it really complemented what I learnt last year and it’s just made me keen to do other stuff, or try and find out more about it.”


The end of t184

The end … of T184

The birth … of TM129 Block 3


  • Login