1 / 30

The Continual Assessment of Confidence or Knowledge with Hidden MCQ? Short Paper W.I.P.

The Continual Assessment of Confidence or Knowledge with Hidden MCQ? Short Paper W.I.P. Phil Davies School of Computing University of Glamorgan. Conference 1999 Learning Through Assessment. Phil Davies School of Computing University of Glamorgan. Implementation of OLAL.

kirk
Download Presentation

The Continual Assessment of Confidence or Knowledge with Hidden MCQ? Short Paper W.I.P.

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. The Continual Assessment of Confidence or Knowledge with Hidden MCQ?Short Paper W.I.P. Phil Davies School of Computing University of Glamorgan

  2. Conference 1999Learning Through Assessment Phil Davies School of Computing University of Glamorgan

  3. Implementation of OLAL • Tests Weeks 3,6,8,12 (60% Subject Mark) • Percentage Marks 5%,15%,15%,25% • “Beep Test” • +2 Correct -1 Incorrect(Really -3)

  4. Student Concentration • Guessing • Why should the student care if incorrect? • Allow TWO PASSES of the test • By concentrating on first pass • will get reward on second pass • formative

  5. Results of The Tests Improvement of Test Results Pass One to Two

  6. Who benefited from the test?

  7. Student Feedback • Excellent way of gaining knowledge • 2nd Test Allowed us to learn • Good assistant to learning • Encourages Revision • Very good, promotes learning of a subject early on • Better than sex!!!!

  8. “There’s no confidence in multiple-choice testing …” CAA Conference 2002 Phil Davies

  9. What decides the QUALITY of a question? • In the 2001 World cup qualifier between England and Greece, who scored the all important equalizing goal? • David Beckham • Victoria Beckham • Brooklyn Beckham • Rebecca Loos

  10. Rewards? • Know it • On the tip of my tongue • If I see the answers I’ll know it • Deduction • Guess • Know it … oh no I’ve got it wrong .. misinformation

  11. Proportion of Questions Correct • High Confidence => 84% correct • Fairly Confident => 58% correct • Not confident => 47% correct • 72% of all questions answered with high confidence

  12. Student thoughts on Confidence • “Eliminates guesswork” • “I was waiting for a leggy blond to bring on my prize at the end” • “separated the lucky students from me” • “at least I didn’t lose marks for selecting no confidence” • “are you testing my confidence or knowledge”

  13. Quality of a multiple choice question? • In the 2001 World cup qualifier between England and Greece, who scored the all important equalizing goal? • Teddy Sheringham • Michael Owen • Robbie Fowler • QUALITY OF QUESTION NOT DISTRACTERS

  14. “Bite The Bullet” • Use Confidence Testing for all four tests (10%, 15%, 20% & 25%) • Weighting of Passes (50/50, 60/40, 70/30 & 80/20) • How to judge if the students had ‘really’ learned? • Identify improvement > • Select five questions that were answered ‘poorest’ in test one & pass through to test two • Select five questions that were answered ‘poorest’ in test two & pass through to test three ( + original five) …. Etc.

  15. How to identify ‘Poorest’ • Using confidence, degree of correctness of answering • Using confidence scores to provide a degree of ‘poorness’ i.e. +4, +2, +1, -6, -5, -4 • High degree of ‘poorness’ could just be a ‘BAD’ question • Swing between 1st and 2nd pass of degree of correctness should be ‘Reasonable’

  16. Identifying Poor Questions Test One

  17. Identifying the Poorest Questions Test two to three

  18. Identifying Poorest Questions from Three to Four

  19. Passed through to Test Four • 13 of the poorest question • 6 new questions based upon additional work between tests 3-4 • 21 questions (cross section of questions and topics from previous tests)

  20. Improvement of Results Through Tests

  21. Improvement Pass One of Questions

  22. Improvement for Question 5 • Stronger Students improved quickly through the tests • Became more confident in answering the questions • Weaker Students improved less quickly than stronger through the test progress

  23. Is it just Confidence or Knowledge i.e. questions right or wrong?

  24. Some Concerns

  25. Overall Comments • Work in Progress • Results look very positive • Correctness Factor is dependant upon weighting of scores • Guessing not so much of a problem • Changed students mode of revising / study • Fine granularity in the QUALITY OF A QUESTION Possibility of Computerized Adaptive Testing However note the quality of the question change throughout the course of the cumulative tests • Does gender come into it??

  26. Contact Information • Email: pdavies@glam.ac.uk • Phone: 01443 482247 School of Computing University of Glamorgan Pontypridd Mid – Glamorgan South Wales

More Related