Heriot watt university breakout session mcq with confidence
Download
1 / 25

Heriot Watt University Breakout Session MCQ with Confidence - PowerPoint PPT Presentation


  • 92 Views
  • Uploaded on

Heriot Watt University Breakout Session MCQ with Confidence. Dr Phil Davies Division of Computing & Mathematical Sciences Department of Computing FAT University of Glamorgan. E-Assessment can be viewed from different perspectives. Commercial / Large Scale Implementations

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Heriot Watt University Breakout Session MCQ with Confidence' - tymon


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Heriot watt university breakout session mcq with confidence

Heriot Watt UniversityBreakout SessionMCQ with Confidence

Dr Phil Davies

Division of Computing & Mathematical Sciences

Department of Computing

FAT

University of Glamorgan


E assessment can be viewed from different perspectives
E-Assessment can be viewed from different perspectives

  • Commercial / Large Scale Implementations

    • Question Mark’s Perception

    • MOODLE

    • Triads (25 different types of Multiple Choice Type Questions!)

    • TOIA (FREE!) Tool for On-line Interoperable Assessment

    • Blackboard Assessment (version!)

    • Standards (IMS QTI Specification)

  • Try it and see


How easy to do mcq cost
How easy to do MCQ?Cost?

  • Question

  • <normally>

  • Key =

  • Distracter One, Distracter Two, ……………..

  • Note

    • Key(s) must always be correct

    • How many students would you expect to get it right for it to be a GOOD question?

    • Distracter One / Two / Three equal # of selections

    • Throw away BAD questions?


Too hard
TOO HARD .. ?

  • Too time consuming

  • Often needs to be an iterative process

  • Does it matter if a question is NOT perfect?

  • Seeking perfection not feasible

  • I suggest that assessment MUST be included within a module, not only at end > make it embedded within TEACHING & LEARNING experience

  • Formative / Summative / Both


My entry into caa
My Entry into CAA

  • 1st Attempt TOTAL FAILURE – 1998 OLAL

  • 5 minutes all finished apart from one student

  • Guessing

    • Why should the student care if incorrect?

    • How to make it worthwhile – slow students down – make them think

  • Negative marking used +2 and -1 (actually becomes -3 –> double whammy!)

  • Allow TWO PASSES of the test

  • By concentrating on first pass

    • will get reward on second pass

    • Formative

  • IMPORTANT – WORTH MARKS


Own background objective testing via mcq
Own Background = Objective Testing via MCQ

  • Four Tests MCQ .. Two passes 50/50 (became graduated)

  • Cumulative Tests 5%, 10%, 15%, 20%

  • Amateur .. No Research Pedigree (CAA Conference)

  • Improved Class results > not easier – more directed

  • Weaker students improved dramatically

  • Assessed the learning outcomes of subject

  • “END JUSTIFIES THE MEANS”


What did the students say about it
What did the students say about it?

  • Excellent way of gaining knowledge

  • 2nd Test Allowed us to learn

  • Good assistant to learning

  • Encourages Revision

  • Very good, promotes learning of a subject early on

  • Better than sex!!!!


What decides the quality of a question
What decides the QUALITY of a question?

  • In the 2004 World cup qualifier between England and Greece, who scored the all important equalizing goal?

  • David Beckham

  • Victoria Beckham

  • Brooklyn Beckham

  • Romeo Beckham


Rewards
Rewards?

  • Know it

  • On the tip of my tongue

  • If I see the answers I’ll know it

  • Deduction

  • Guess

  • Know it … oh no I’ve got it wrong .. misinformation


Heriot watt university breakout session mcq with confidence

TRIED ONE CONFIDENCE TEST

ONLY SHOWS ANSWERS ONCE CONFIDENCE SELECTED

RELATES TO MARKS +4, +2, +1, -2, -1, 0


Proportion of questions correct
Proportion of Questions Correct

  • High Confidence => 84% correct

  • Fairly Confident => 58% correct

  • Not confident => 47% correct

  • 72% of all questions answered with high confidence


Student thoughts on confidence
Student thoughts on Confidence

  • “Eliminates guesswork”

  • “I was waiting for a leggy blond to bring on my prize at the end”

  • “separated the lucky students from me”

  • “at least I didn’t lose marks for selecting no confidence”

  • “are you testing my confidence or knowledge”


Bite the bullet
“Bite The Bullet”

  • Use Confidence Testing for all four tests (10%, 15%, 20% & 25%)

  • Real SHOCK to students

  • Weighting of Passes (50/50, 60/40, 70/30 & 80/20)

  • How to judge if the students had ‘really’ learned?

  • Select five questions that were answered ‘poorest’ in test one & pass through to test two

  • Select five questions that were answered ‘poorest’ in test two & pass through to test three ( + original five) …. Etc.


Identifying quality of a question
Identifying Quality of a Question

  • Negative on first pass e.g. > -180

  • Difference between first and second pass

  • Too big an improvement may just signify a very poor question e.g. >500?

  • Create heuristic that identifies degree of difficulty of a question


Passed through to test four
Passed through to Test Four

  • 13 of the ‘hardest’ question

  • 6 new questions based upon additional work between tests 3-4

  • 21 questions (cross section of questions and topics from previous tests)




Improvement for question 5
Improvement for Question 5

  • Stronger Students improved quickly through the tests

  • Became more confident in answering the questions

  • Weaker Students improved less quickly than stronger through the test progress, but DID improve



Quality of a multiple choice question
Quality of a multiple choice question? wrong?

  • In the 2004 World cup qualifier between England and Greece, who scored the all important equalizing goal?

  • Teddy Sheringham

  • Michael Owen

  • Robbie Fowler

  • QUALITY OF QUESTION NOT DISTRACTERS


Developmental v assessment
Developmental v Assessment wrong?

  • Need not be at alternative ends of the scale

  • Student motivation / attendance increased considerably

  • Testing knowledge (not done at moment for higher order skills)

  • Good students have best results i.e. number of questions correct


Comments
COMMENTS? wrong?

  • HMCQ

  • Confidence

  • Two Pass

  • Formative & Summative

  • Graduated Assessment

  • Continual Assessment