opening up multiple choice assessing with confidence n.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
Opening up multiple choice: Assessing with confidence PowerPoint Presentation
Download Presentation
Opening up multiple choice: Assessing with confidence

Loading in 2 Seconds...

play fullscreen
1 / 19

Opening up multiple choice: Assessing with confidence - PowerPoint PPT Presentation


  • 91 Views
  • Uploaded on

Opening up multiple choice: Assessing with confidence. Jon Rosewell J.P.Rosewell@open.ac.uk. Self-assessment: strategies and software to stimulate learning HEA workshop, The Open University, 11 th June 2012. Pros objective marking reliable marking easy to implement Cons

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Opening up multiple choice: Assessing with confidence' - raffaello


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
opening up multiple choice assessing with confidence

Opening up multiple choice: Assessing with confidence

Jon RosewellJ.P.Rosewell@open.ac.uk

Self-assessment: strategies and software to stimulate learning

HEA workshop, The Open University, 11th June 2012

mcq open question
Pros

objective marking

reliable marking

easy to implement

Cons

distractors trivial

distractors engender misconceptions

working backwards

Pros

numeric easy to mark

tests deeper learning

can find misconceptions

Cons

Free text very difficult to mark reliably

MCQ Open question
how not to do mcq
How (not) to do MCQ…
  • Give the name of a supervolcano:
      • Yellowstone
      • Redstone
      • Bluestone
  • Which is a plate boundary:
      • Conservative
      • Destructive
      • Liberal
      • Labour
  • What policy did the Chinese introduce to limit population:
      • One child policy
      • Two child policy
      • The child catcher

Reported from BBC Bite Size

cbm scoring
CBM scoring

Tentative & correct

Confidently correct

Cocksure – and wrong!

Gardner-Medwin & Curtin (2007) Certainty-Based Marking (CBM) for Reflective Learning and Proper Knowledge Assessment[http://www.ucl.ac.uk/lapt/REAP_cbm.pdf]

cbm motivations
CBM motivations
  • Rewards care and effort
  • Greater engagement
  • Encourages reflective learning
  • Encourages self-assessment
open cbm advantages
Open CBM advantages
  • Mechanically, question remains simple MCQ
    • answer matching is trivial
    • easy to implement
    • possible to reuse existing question banks
  • But transformed closed to open question
    • student must formulate answer to decide confidence
    • must decide answer without +ve or –ve clues.
    • cannot work backward
    • will not be led into misconceptions.

 must answer the question as set.

mcq open cbm
Pros

objective marking

reliable marking

easy to implement

Cons

distractors trivial

distractors engender misconceptions

working backwards

Pros

open question

reflection

...as MCQ

Cons

not always applicable

intimidating?

personality dependent?

MCQ Open CBM
possible benefits
Possible benefits
  • Students
    • better engagement
    • improved learning
    • more reflective learners
  • Staff
    • easier question setting
esteem project plan
eSTEeM project plan
  • Implement open CBM question type
    • in OpenMark
  • Trial in course assessment
    • controlled experimental design
    • pilot (formative) on T216 CISCO networking
  • Log measures
    • Assignment scores
    • Time spent on task
  • Structured interviews to probe attitudinal aspects
  • User-lab trials
t216 cisco networking
T216 Cisco networking
  • Questions from existing bank
  • Offered as revision quizzes
  • One set for each of four blocks
  • 25 questions drawn randomly from bank of ~100
  • Randomly offered as either open CBM or MCQ format
  • Issues:
    • Some questions bad!
    • OpenMark report doesn’t show confidence
    • Too much randomness
demo questions scores
Demo questions – scores

Easy

Tricky!

Difficult

mcq open cbm1
Pros

objective marking

reliable marking

easy to implement

Cons

distractors trivial

distractors engender misconceptions

working backwards

Pros

open question

reflection

...as MCQ

Cons

not always applicable

intimidating?

personality dependent?

MCQ Open CBM