assessment challenge for learners markers
Skip this Video
Download Presentation
Assessment: Challenge for learners & markers

Loading in 2 Seconds...

play fullscreen
1 / 39

Assessment: Challenge for learners markers - PowerPoint PPT Presentation

  • Uploaded on

Assessment: Challenge for learners & markers. Professor Colin Mason. "Assessment is the most powerful tool teachers possess in moulding student effort and learning." Graham Gibbs and Trevor Habeshaw, 1988

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation

PowerPoint Slideshow about 'Assessment: Challenge for learners markers' - talen

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
why assess
"Assessment is the most powerful tool teachers possess in moulding student effort and learning."

Graham Gibbs and Trevor Habeshaw, 1988

"If we wish to discover the truth about an educational system, we must look into its assessment procedures .. the spirit and style of student assessment defines the de facto curriculum."

Derek Rowntree, 1987

Why assess?
pick the best answer why are lectures still used so frequently in higher education
Pick the best answerWhy are lectures still used so frequently in higher education?

So that lecturers can demonstrate how expert they are in their subject.

Because they are a very efficient means of transmitting information to large numbers of students.

To provide a common space where students can be enthused by a lecturer exploring key concepts of their subject.

Students expect them as part of university (rather than school) education and prefer listening to having to contribute themselves.

Lectures are easier to prepare than more interactive resource-based techniques such as case study discussions or problem-based tutorials.

tensions in assessment george brown 1996

Tensions in Assessment(George Brown, 1996)


Fits aims and learning outcomes


Saves time (and effort - eventually)


For understanding and expertise

some examples of innovation in methods of assessment
Some examples of innovation in methods of assessment:



Unseen exams


Annotated Bibliography

Seen exams


Design and create VLE course

‘Posts’ to discussion boards


PBL scenarios



Workbooks, diaries, logs

Reflective Report

Practical tests

Authentic Work-based observations by professionals

Higher Cognitive skills, Transferable skills

Knowledge, Understanding

See also: Integrative Assessment Guide no 3

complex outcomes of learning
Complex outcomes of learning

‘Complex’ outcomes including higher order academic abilities (analysis, critical reasoning) and ‘soft skills’ (teamwork, leadership) are rarely and inconsistently defined

Advanced ‘skilful practice’ is acquired slowly (years?)

Precision & reliability of such assessment is often only attained at the expense of validity

Shift to FORMATIVE assessment

forms modes of assessment







Forms (Modes) of Assessment



so formative assessment
So, formative assessment ….

Helps learning

Provides diagnostic information for students and staff (even if only ‘scores’)

Provides opportunities for Feedback (corrections) and Feedforward (suggestions for improvement ‘for next time’)

Ensures wider coverage of all learning outcomes

speedier feedback
Speedier Feedback

Speeding up feedback accelerates subsequent learning


objective assessments with instant feedback;

exemplar answers

Objective assessment (eg MCQ, matching etc), especially utilising online approaches, form a significant element of an overall assessment strategy; EVS also.

Problem-based learning with ‘ideal’ or exemplar solutions made available (often online)

assessment efficiency
Assessment Efficiency

Synoptic summative assessment

Multiple Choice Questions (MCQ)

More sophisticated Objective Tests

Computer-based (online) administration of assessments

blended assessment between units synoptic approach

Unit 1

Unit 4

Unit 3

Unit 2

Unit 6

Unit 5





Blended assessment between units: Synoptic approach?





assessment efficiency mcqs http www ukcle ac uk resources trns mcqs index html
Assessment Efficiency – MCQs?

What attracts me about using MCQ assessment?

What are my concerns about using MCQ assessment?

assessment efficiency mcqs http www ukcle ac uk resources trns mcqs index html1
Assessment Efficiency – MCQs?

What is an MCQ?

What could I use MCQs for?

What are the features of MCQs?

Does adopting MCQ assessment mean \'dumbing down\'?

Are MCQ scores unrealistically high?

Can students guess their way to success in an MCQ?

assessment efficiency mcqs http www ukcle ac uk resources trns mcqs index html2
Assessment Efficiency – MCQs?

Does using MCQ assessment encourage rote or surface learning?

Can MCQs test oral and written skills?

What should I think about before I design an MCQ test?

How can I write effective MCQs?

How can I provide effective feedback for my MCQs?

What can I learn from the student responses to the MCQs?

what is an mcq design issues http www le ac uk castle resources
What is an MCQ – Design issues

A Multiple Choice Question (MCQ) is one that students are asked to select one ‘answer’ from a given list of options in response to a question ‘stem’

“Which of the following is the currency unit used in Albania?”










what are the features of mcqs
What are the features of MCQs?

Objective marking

a right or a best answer

but scope for crediting more than one right answer or \'near miss\' answers, depending on the marking scheme

Testing is efficient

greater part of syllabus assessed by compulsory questions.

Answers to Questions are easy to score or mark

Especially online or computer-assisted eg DSO, WebCT, Blackboard, QuestionMark.

Scoring uses the entire marking range (of 100% scale)

raises issues where the MCQs are used in summative assessment.

what are the features of mcqs1
What are the features of MCQs?

Results are quantifiable

possible to analyse the level of student achievement in each question and thus identify areas of student difficulty or problematic questions

Feedback can be targeted more effectively

Setting questions is time consuming and challenging

Initial investment is high, but questions can be re-cycled in databanks

Recycling of questions raises ‘quality’ issues for summative assessments

Rank (R) the options (1 - 5)There is an increasing interest in the use of formative assessment in higher education because …

Fewer students submit assignments, reducing staff marking loads.

Students can choose whether or not to complete an assessment assignment.

It allows students to choose when, where and how they complete an assessment assignment.

It provides non-threatening, ‘low stakes’, opportunities for students to learn from their mistakes and feedback provided on their assessed work.

Academic staff need to worry less about the accuracy of their marking because it doesn’t count.

(If correct, R1=5 marks; R2=3 marks ;R3=2 marks; R4=1; R5=0)

evaluate the sentence explaining why mcq tests are used in higher education these days
Evaluate the sentence explaining why MCQ tests are used in higher education these days

MCQ tests are increasingly used in assessment in higher education coursesBECAUSEthey can easily be marked by computer systems and are thus more objective

The assertion and the reason are both correct, and the reason is appropriate or valid.

The assertion and the reason are both correct, but the reason is inappropriate or invalid.

The assertion is correct but the reason is incorrect.

The assertion is incorrect but the reason is correct

Both the assertion and the reason are incorrect

high marks and guessing what to do about it
High Marks and Guessing – What to do about it?

‘An MCQ with 5 options presents a 20% chance of \'guessing\' the correct answer’

‘negative scoring’?

adopting mathematical strategies to \'normalise\' the mark (see Bush, 1999)?

raising the pass mark for the MCQ element of assessment?

making MCQs one component (low weighting) of the assessment strategy?

concentrating on using MCQs in formative situations where the \'result\' is less important than the process?

true false a special case mcq
True/False -a special case MCQ

100 T/F

(Design:1 mark –correct; 0 mark –incorrect)

“Monkey Mark” 50/100 = 50% (Pass?)

100 T/F

(Design: 1 mark correct; -1 mark –incorrect)

Excellent student – 100%

“Monkey Mark” – 0%

mcq negative mark designs
MCQ – Negative mark Designs

3 option MCQ

Unfair: 1 mark –correct; -1 mark –each incorrect distractor

So …….

1 mark correct; -0.5 mark each incorrect

But, easier ….

2 mark correct; -1 mark each incorrect

negative scoring for the whole test
Negative scoring for the whole test

Score = Deserved score + undeserved score (for guessing)

100 MCQ with 5 options; correct option – 4 marks

25 ‘guesses’; 1 in 5 chance (20%)

undeserved score = 20% of 25 x 4 marks

undeserved score = 5 x 4 = 20 marks

Deduction for undeserved score = No. Qs (25-5) x I

Deduction for undeserved score = 20 x 1 = 20

So, adjust Score according to

Score = Deserved + undeserved – deduction undeserved

Score = Deserved +20 – 20

Score = Deserved

what about educated guessing
What about ‘educated guessing’?

Situation: 100 MCQs (5 options)

Student ‘knows’ 50; guesses 25; but ‘educated guesses’ 25 (8 correctly; 17 incorrectly)


Deserved 50 x 4 = 200

Undeserved 5 (20% of 25) x4 – 20xI = 0

Partial 8 x4 -17 x1 = 15

Total 200 + 0 + 15 =215 (out of 400)

If no negative marking ie 0 for incorrect answer)

Total = 200 + (5x4 – 20x0) + (8x4 – 17x0)

Total + 200 + 20 + 32 = 252

other adjustments
Other Adjustments

Order of preference (or ranking) of options


Choose option and assign ‘confidence’ level (1, 2, 3)

If correct marks = 1, 2 or 3

If incorrect marks = 0, -2, -6


X → (x-20) * 5/4 (x = unadjusted mark)

Liberal (more attempts) MCQ

Eg 5 option MCQ

1 attempt – score 4/4 = 100% on Question

2 attempts – score (4-1)/4 = 75 % on Question

3 attempts – score (4-2)/4 = 50% on Question etc

formative online assessment
Formative online assessment

On-line approaches

Deakin Studies Online (DSO)

e.g. Blackboard, Moodle

Assessment Management Systems

e.g. QuestionMark, TRIADS


e.g. Respondus, StudyMate

Techniques for Formative Assessment





on line assessment simple
Self Assessment Exercises

DSO Self Assessment Tool

MCQs within content using

(simple) javascript

On-line Assessment: Simple
on line assessment medium
On-line Assessment: Medium

DSO tests/quizzes

Multiple Choice


Multiple Answer

Matching Pair

Fill in the blanks

Short Answer



What are the “Christian” names for the Islamic prophets Ibrahim, Daviyd, Musa and Isa?

fill in the blanks
Fill in the blanks

Answered Correctly

& Alternative Answers

on line assessment advanced
On-line Assessment: Advanced

Same as Medium Technique


Randomisation of a question’s possible answers

Randomisation of questions

All randomised

Randomised from bank

e.g. any 30 questions from 100

Pre-defined and randomised

e.g. first 10 defined then 30 random questions

on line assessment advanced1
On-line Assessment: Advanced

Same as Medium Technique


  • Timed assessments (“beat the clock”)
  • Timed released
  • Incentive driven e.g.
    • If score >80% release other learning materials
  • Extra support
    • If score <40% release more learning materials
    • + extra test
on line assessment fun


On-line Assessment: Fun

Flash based - can be used within DSO or on standard web pages e.g.

  • Crosswords
  • Pick a Letter
  • Challenge
  • And more…
useful urls

Useful URLs

Approaches to assessment and feedback that foster independent learning

Online assessment Strategies

UK resources to assist MCQ designers


Wimba Create (CourseGenie)


contact us

Contact us

Institute of Teaching and Learning

ITL Support Service


Phone: 5227 8127 (ext. 78127)

Email: [email protected]