heriot watt university breakout session mcq with confidence
Download
Skip this Video
Download Presentation
Heriot Watt University Breakout Session MCQ with Confidence

Loading in 2 Seconds...

play fullscreen
1 / 25

Heriot Watt University Breakout Session MCQ with Confidence - PowerPoint PPT Presentation


  • 92 Views
  • Uploaded on

Heriot Watt University Breakout Session MCQ with Confidence. Dr Phil Davies Division of Computing & Mathematical Sciences Department of Computing FAT University of Glamorgan. E-Assessment can be viewed from different perspectives. Commercial / Large Scale Implementations

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about ' Heriot Watt University Breakout Session MCQ with Confidence' - tymon


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
heriot watt university breakout session mcq with confidence

Heriot Watt UniversityBreakout SessionMCQ with Confidence

Dr Phil Davies

Division of Computing & Mathematical Sciences

Department of Computing

FAT

University of Glamorgan

e assessment can be viewed from different perspectives
E-Assessment can be viewed from different perspectives
  • Commercial / Large Scale Implementations
    • Question Mark’s Perception
    • MOODLE
    • Triads (25 different types of Multiple Choice Type Questions!)
    • TOIA (FREE!) Tool for On-line Interoperable Assessment
    • Blackboard Assessment (version!)
    • Standards (IMS QTI Specification)
  • Try it and see
how easy to do mcq cost
How easy to do MCQ?Cost?
  • Question
  • <normally>
  • Key =
  • Distracter One, Distracter Two, ……………..
  • Note
    • Key(s) must always be correct
    • How many students would you expect to get it right for it to be a GOOD question?
    • Distracter One / Two / Three equal # of selections
    • Throw away BAD questions?
too hard
TOO HARD .. ?
  • Too time consuming
  • Often needs to be an iterative process
  • Does it matter if a question is NOT perfect?
  • Seeking perfection not feasible
  • I suggest that assessment MUST be included within a module, not only at end > make it embedded within TEACHING & LEARNING experience
  • Formative / Summative / Both
my entry into caa
My Entry into CAA
  • 1st Attempt TOTAL FAILURE – 1998 OLAL
  • 5 minutes all finished apart from one student
  • Guessing
    • Why should the student care if incorrect?
    • How to make it worthwhile – slow students down – make them think
  • Negative marking used +2 and -1 (actually becomes -3 –> double whammy!)
  • Allow TWO PASSES of the test
  • By concentrating on first pass
    • will get reward on second pass
    • Formative
  • IMPORTANT – WORTH MARKS
own background objective testing via mcq
Own Background = Objective Testing via MCQ
  • Four Tests MCQ .. Two passes 50/50 (became graduated)
  • Cumulative Tests 5%, 10%, 15%, 20%
  • Amateur .. No Research Pedigree (CAA Conference)
  • Improved Class results > not easier – more directed
  • Weaker students improved dramatically
  • Assessed the learning outcomes of subject
  • “END JUSTIFIES THE MEANS”
what did the students say about it
What did the students say about it?
  • Excellent way of gaining knowledge
  • 2nd Test Allowed us to learn
  • Good assistant to learning
  • Encourages Revision
  • Very good, promotes learning of a subject early on
  • Better than sex!!!!
what decides the quality of a question
What decides the QUALITY of a question?
  • In the 2004 World cup qualifier between England and Greece, who scored the all important equalizing goal?
  • David Beckham
  • Victoria Beckham
  • Brooklyn Beckham
  • Romeo Beckham
rewards
Rewards?
  • Know it
  • On the tip of my tongue
  • If I see the answers I’ll know it
  • Deduction
  • Guess
  • Know it … oh no I’ve got it wrong .. misinformation
slide11

TRIED ONE CONFIDENCE TEST

ONLY SHOWS ANSWERS ONCE CONFIDENCE SELECTED

RELATES TO MARKS +4, +2, +1, -2, -1, 0

proportion of questions correct
Proportion of Questions Correct
  • High Confidence => 84% correct
  • Fairly Confident => 58% correct
  • Not confident => 47% correct
  • 72% of all questions answered with high confidence
student thoughts on confidence
Student thoughts on Confidence
  • “Eliminates guesswork”
  • “I was waiting for a leggy blond to bring on my prize at the end”
  • “separated the lucky students from me”
  • “at least I didn’t lose marks for selecting no confidence”
  • “are you testing my confidence or knowledge”
bite the bullet
“Bite The Bullet”
  • Use Confidence Testing for all four tests (10%, 15%, 20% & 25%)
  • Real SHOCK to students
  • Weighting of Passes (50/50, 60/40, 70/30 & 80/20)
  • How to judge if the students had ‘really’ learned?
  • Select five questions that were answered ‘poorest’ in test one & pass through to test two
  • Select five questions that were answered ‘poorest’ in test two & pass through to test three ( + original five) …. Etc.
identifying quality of a question
Identifying Quality of a Question
  • Negative on first pass e.g. > -180
  • Difference between first and second pass
  • Too big an improvement may just signify a very poor question e.g. >500?
  • Create heuristic that identifies degree of difficulty of a question
passed through to test four
Passed through to Test Four
  • 13 of the ‘hardest’ question
  • 6 new questions based upon additional work between tests 3-4
  • 21 questions (cross section of questions and topics from previous tests)
improvement for question 5
Improvement for Question 5
  • Stronger Students improved quickly through the tests
  • Became more confident in answering the questions
  • Weaker Students improved less quickly than stronger through the test progress, but DID improve
quality of a multiple choice question
Quality of a multiple choice question?
  • In the 2004 World cup qualifier between England and Greece, who scored the all important equalizing goal?
  • Teddy Sheringham
  • Michael Owen
  • Robbie Fowler
  • QUALITY OF QUESTION NOT DISTRACTERS
developmental v assessment
Developmental v Assessment
  • Need not be at alternative ends of the scale
  • Student motivation / attendance increased considerably
  • Testing knowledge (not done at moment for higher order skills)
  • Good students have best results i.e. number of questions correct
comments
COMMENTS?
  • HMCQ
  • Confidence
  • Two Pass
  • Formative & Summative
  • Graduated Assessment
  • Continual Assessment
ad