Heriot watt university breakout session mcq with confidence
Download
1 / 25

Heriot Watt University Breakout Session MCQ with Confidence - PowerPoint PPT Presentation


  • 92 Views
  • Uploaded on

Heriot Watt University Breakout Session MCQ with Confidence. Dr Phil Davies Division of Computing & Mathematical Sciences Department of Computing FAT University of Glamorgan. E-Assessment can be viewed from different perspectives. Commercial / Large Scale Implementations

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about ' Heriot Watt University Breakout Session MCQ with Confidence' - tymon


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Heriot watt university breakout session mcq with confidence

Heriot Watt UniversityBreakout SessionMCQ with Confidence

Dr Phil Davies

Division of Computing & Mathematical Sciences

Department of Computing

FAT

University of Glamorgan


E assessment can be viewed from different perspectives
E-Assessment can be viewed from different perspectives

  • Commercial / Large Scale Implementations

    • Question Mark’s Perception

    • MOODLE

    • Triads (25 different types of Multiple Choice Type Questions!)

    • TOIA (FREE!) Tool for On-line Interoperable Assessment

    • Blackboard Assessment (version!)

    • Standards (IMS QTI Specification)

  • Try it and see


How easy to do mcq cost
How easy to do MCQ?Cost?

  • Question

  • <normally>

  • Key =

  • Distracter One, Distracter Two, ……………..

  • Note

    • Key(s) must always be correct

    • How many students would you expect to get it right for it to be a GOOD question?

    • Distracter One / Two / Three equal # of selections

    • Throw away BAD questions?


Too hard
TOO HARD .. ?

  • Too time consuming

  • Often needs to be an iterative process

  • Does it matter if a question is NOT perfect?

  • Seeking perfection not feasible

  • I suggest that assessment MUST be included within a module, not only at end > make it embedded within TEACHING & LEARNING experience

  • Formative / Summative / Both


My entry into caa
My Entry into CAA

  • 1st Attempt TOTAL FAILURE – 1998 OLAL

  • 5 minutes all finished apart from one student

  • Guessing

    • Why should the student care if incorrect?

    • How to make it worthwhile – slow students down – make them think

  • Negative marking used +2 and -1 (actually becomes -3 –> double whammy!)

  • Allow TWO PASSES of the test

  • By concentrating on first pass

    • will get reward on second pass

    • Formative

  • IMPORTANT – WORTH MARKS


Own background objective testing via mcq
Own Background = Objective Testing via MCQ

  • Four Tests MCQ .. Two passes 50/50 (became graduated)

  • Cumulative Tests 5%, 10%, 15%, 20%

  • Amateur .. No Research Pedigree (CAA Conference)

  • Improved Class results > not easier – more directed

  • Weaker students improved dramatically

  • Assessed the learning outcomes of subject

  • “END JUSTIFIES THE MEANS”


What did the students say about it
What did the students say about it?

  • Excellent way of gaining knowledge

  • 2nd Test Allowed us to learn

  • Good assistant to learning

  • Encourages Revision

  • Very good, promotes learning of a subject early on

  • Better than sex!!!!


What decides the quality of a question
What decides the QUALITY of a question?

  • In the 2004 World cup qualifier between England and Greece, who scored the all important equalizing goal?

  • David Beckham

  • Victoria Beckham

  • Brooklyn Beckham

  • Romeo Beckham


Rewards
Rewards?

  • Know it

  • On the tip of my tongue

  • If I see the answers I’ll know it

  • Deduction

  • Guess

  • Know it … oh no I’ve got it wrong .. misinformation


TRIED ONE CONFIDENCE TEST

ONLY SHOWS ANSWERS ONCE CONFIDENCE SELECTED

RELATES TO MARKS +4, +2, +1, -2, -1, 0


Proportion of questions correct
Proportion of Questions Correct

  • High Confidence => 84% correct

  • Fairly Confident => 58% correct

  • Not confident => 47% correct

  • 72% of all questions answered with high confidence


Student thoughts on confidence
Student thoughts on Confidence

  • “Eliminates guesswork”

  • “I was waiting for a leggy blond to bring on my prize at the end”

  • “separated the lucky students from me”

  • “at least I didn’t lose marks for selecting no confidence”

  • “are you testing my confidence or knowledge”


Bite the bullet
“Bite The Bullet”

  • Use Confidence Testing for all four tests (10%, 15%, 20% & 25%)

  • Real SHOCK to students

  • Weighting of Passes (50/50, 60/40, 70/30 & 80/20)

  • How to judge if the students had ‘really’ learned?

  • Select five questions that were answered ‘poorest’ in test one & pass through to test two

  • Select five questions that were answered ‘poorest’ in test two & pass through to test three ( + original five) …. Etc.


Identifying quality of a question
Identifying Quality of a Question

  • Negative on first pass e.g. > -180

  • Difference between first and second pass

  • Too big an improvement may just signify a very poor question e.g. >500?

  • Create heuristic that identifies degree of difficulty of a question


Passed through to test four
Passed through to Test Four

  • 13 of the ‘hardest’ question

  • 6 new questions based upon additional work between tests 3-4

  • 21 questions (cross section of questions and topics from previous tests)




Improvement for question 5
Improvement for Question 5

  • Stronger Students improved quickly through the tests

  • Became more confident in answering the questions

  • Weaker Students improved less quickly than stronger through the test progress, but DID improve



Quality of a multiple choice question
Quality of a multiple choice question? wrong?

  • In the 2004 World cup qualifier between England and Greece, who scored the all important equalizing goal?

  • Teddy Sheringham

  • Michael Owen

  • Robbie Fowler

  • QUALITY OF QUESTION NOT DISTRACTERS


Developmental v assessment
Developmental v Assessment wrong?

  • Need not be at alternative ends of the scale

  • Student motivation / attendance increased considerably

  • Testing knowledge (not done at moment for higher order skills)

  • Good students have best results i.e. number of questions correct


Comments
COMMENTS? wrong?

  • HMCQ

  • Confidence

  • Two Pass

  • Formative & Summative

  • Graduated Assessment

  • Continual Assessment


ad