Assessment challenge for learners markers
Download
1 / 39

Assessment: Challenge for learners markers - PowerPoint PPT Presentation


  • 114 Views
  • Uploaded on

Assessment: Challenge for learners & markers. Professor Colin Mason. "Assessment is the most powerful tool teachers possess in moulding student effort and learning." Graham Gibbs and Trevor Habeshaw, 1988

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Assessment: Challenge for learners markers' - talen


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Assessment challenge for learners markers

Assessment: Challenge for learners & markers

Professor Colin Mason


Why assess

"Assessment is the most powerful tool teachers possess in moulding student effort and learning."

Graham Gibbs and Trevor Habeshaw, 1988

"If we wish to discover the truth about an educational system, we must look into its assessment procedures .. the spirit and style of student assessment defines the de facto curriculum."

Derek Rowntree, 1987

Why assess?


Pick the best answer why are lectures still used so frequently in higher education
Pick the best answer moulding student effort and learning."Why are lectures still used so frequently in higher education?

So that lecturers can demonstrate how expert they are in their subject.

Because they are a very efficient means of transmitting information to large numbers of students.

To provide a common space where students can be enthused by a lecturer exploring key concepts of their subject.

Students expect them as part of university (rather than school) education and prefer listening to having to contribute themselves.

Lectures are easier to prepare than more interactive resource-based techniques such as case study discussions or problem-based tutorials.


Tensions in assessment george brown 1996

Tensions in Assessment moulding student effort and learning."(George Brown, 1996)

  • EFFECTIVENESS

    Fits aims and learning outcomes

  • EFFICIENCY

    Saves time (and effort - eventually)

  • ENABLEMENT

    For understanding and expertise


Some examples of innovation in methods of assessment
Some examples of innovation in moulding student effort and learning."methods of assessment:

Exams

Coursework

Unseen exams

Presentations

Annotated Bibliography

Seen exams

Reports

Design and create VLE course

‘Posts’ to discussion boards

Essays

PBL scenarios

MCQs

OSCEs

Workbooks, diaries, logs

Reflective Report

Practical tests

Authentic Work-based observations by professionals

Higher Cognitive skills, Transferable skills

Knowledge, Understanding

See also: Integrative Assessment Guide no 3


Complex outcomes of learning
Complex outcomes of learning moulding student effort and learning."

‘Complex’ outcomes including higher order academic abilities (analysis, critical reasoning) and ‘soft skills’ (teamwork, leadership) are rarely and inconsistently defined

Advanced ‘skilful practice’ is acquired slowly (years?)

Precision & reliability of such assessment is often only attained at the expense of validity

Shift to FORMATIVE assessment


Forms modes of assessment

DIAGNOSTIC moulding student effort and learning."

DIAGNOSTIC

FORMATIVE

FORMATIVE

SUMMATIVE

SUMMATIVE

Forms (Modes) of Assessment

‘Reality’

‘Theory’


So formative assessment
So, formative assessment …. moulding student effort and learning."

Helps learning

Provides diagnostic information for students and staff (even if only ‘scores’)

Provides opportunities for Feedback (corrections) and Feedforward (suggestions for improvement ‘for next time’)

Ensures wider coverage of all learning outcomes


Speedier feedback
Speedier Feedback moulding student effort and learning."

Speeding up feedback accelerates subsequent learning

Solutions:

objective assessments with instant feedback;

exemplar answers

Objective assessment (eg MCQ, matching etc), especially utilising online approaches, form a significant element of an overall assessment strategy; EVS also.

Problem-based learning with ‘ideal’ or exemplar solutions made available (often online)


Assessment efficiency
Assessment Efficiency moulding student effort and learning."

Synoptic summative assessment

Multiple Choice Questions (MCQ)

More sophisticated Objective Tests

Computer-based (online) administration of assessments


Blended assessment between units synoptic approach

Unit 1 moulding student effort and learning."

Unit 4

Unit 3

Unit 2

Unit 6

Unit 5

Formative

Formative

Summative

Summative

Blended assessment between units: Synoptic approach?

LATERAL

LONGITUDINAL

TRIMESTER 2

TRIMESTER 1


Assessment efficiency mcqs http www ukcle ac uk resources trns mcqs index html
Assessment Efficiency – MCQs? moulding student effort and learning."http://www.ukcle.ac.uk/resources/trns/mcqs/index.html

What attracts me about using MCQ assessment?

What are my concerns about using MCQ assessment?


Assessment efficiency mcqs http www ukcle ac uk resources trns mcqs index html1
Assessment Efficiency – MCQs? moulding student effort and learning."http://www.ukcle.ac.uk/resources/trns/mcqs/index.html

What is an MCQ?

What could I use MCQs for?

What are the features of MCQs?

Does adopting MCQ assessment mean 'dumbing down'?

Are MCQ scores unrealistically high?

Can students guess their way to success in an MCQ?


Assessment efficiency mcqs http www ukcle ac uk resources trns mcqs index html2
Assessment Efficiency – MCQs? moulding student effort and learning."http://www.ukcle.ac.uk/resources/trns/mcqs/index.html

Does using MCQ assessment encourage rote or surface learning?

Can MCQs test oral and written skills?

What should I think about before I design an MCQ test?

How can I write effective MCQs?

How can I provide effective feedback for my MCQs?

What can I learn from the student responses to the MCQs?


What is an mcq design issues http www le ac uk castle resources
What is an MCQ – Design issues moulding student effort and learning."http://www.le.ac.uk/castle/resources/

A Multiple Choice Question (MCQ) is one that students are asked to select one ‘answer’ from a given list of options in response to a question ‘stem’

“Which of the following is the currency unit used in Albania?”

Dinar

Rouble

Kwacha

Lek

Rupee

Options

‘Answer’

Stem

Distractors


What are the features of mcqs
What are the features of MCQs? moulding student effort and learning."

Objective marking

a right or a best answer

but scope for crediting more than one right answer or 'near miss' answers, depending on the marking scheme

Testing is efficient

greater part of syllabus assessed by compulsory questions.

Answers to Questions are easy to score or mark

Especially online or computer-assisted eg DSO, WebCT, Blackboard, QuestionMark.

Scoring uses the entire marking range (of 100% scale)

raises issues where the MCQs are used in summative assessment.


What are the features of mcqs1
What are the features of MCQs? moulding student effort and learning."

Results are quantifiable

possible to analyse the level of student achievement in each question and thus identify areas of student difficulty or problematic questions

Feedback can be targeted more effectively

Setting questions is time consuming and challenging

Initial investment is high, but questions can be re-cycled in databanks

Recycling of questions raises ‘quality’ issues for summative assessments


Rank (R) the options (1 - 5) moulding student effort and learning."There is an increasing interest in the use of formative assessment in higher education because …

Fewer students submit assignments, reducing staff marking loads.

Students can choose whether or not to complete an assessment assignment.

It allows students to choose when, where and how they complete an assessment assignment.

It provides non-threatening, ‘low stakes’, opportunities for students to learn from their mistakes and feedback provided on their assessed work.

Academic staff need to worry less about the accuracy of their marking because it doesn’t count.

(If correct, R1=5 marks; R2=3 marks ;R3=2 marks; R4=1; R5=0)


Evaluate the sentence explaining why mcq tests are used in higher education these days
Evaluate the sentence explaining why MCQ tests are used in higher education these days

MCQ tests are increasingly used in assessment in higher education coursesBECAUSEthey can easily be marked by computer systems and are thus more objective

The assertion and the reason are both correct, and the reason is appropriate or valid.

The assertion and the reason are both correct, but the reason is inappropriate or invalid.

The assertion is correct but the reason is incorrect.

The assertion is incorrect but the reason is correct

Both the assertion and the reason are incorrect


High marks and guessing what to do about it
High Marks and Guessing higher education these days– What to do about it?

‘An MCQ with 5 options presents a 20% chance of 'guessing' the correct answer’

‘negative scoring’?

adopting mathematical strategies to 'normalise' the mark (see Bush, 1999)?

raising the pass mark for the MCQ element of assessment?

making MCQs one component (low weighting) of the assessment strategy?

concentrating on using MCQs in formative situations where the 'result' is less important than the process?


True false a special case mcq
True/False -a special case MCQ higher education these days

100 T/F

(Design:1 mark –correct; 0 mark –incorrect)

“Monkey Mark” 50/100 = 50% (Pass?)

100 T/F

(Design: 1 mark correct; -1 mark –incorrect)

Excellent student – 100%

“Monkey Mark” – 0%


Mcq negative mark designs
MCQ – Negative mark Designs higher education these days

3 option MCQ

Unfair: 1 mark –correct; -1 mark –each incorrect distractor

So …….

1 mark correct; -0.5 mark each incorrect

But, easier ….

2 mark correct; -1 mark each incorrect


I c n 1 mcq scheme for negative marking
I =C/(n-1) higher education these daysMCQ scheme for negative marking


Negative scoring for the whole test
Negative scoring for the whole test higher education these days

Score = Deserved score + undeserved score (for guessing)

100 MCQ with 5 options; correct option – 4 marks

25 ‘guesses’; 1 in 5 chance (20%)

undeserved score = 20% of 25 x 4 marks

undeserved score = 5 x 4 = 20 marks

Deduction for undeserved score = No. Qs (25-5) x I

Deduction for undeserved score = 20 x 1 = 20

So, adjust Score according to

Score = Deserved + undeserved – deduction undeserved

Score = Deserved +20 – 20

Score = Deserved


What about educated guessing
What about ‘educated guessing’? higher education these days

Situation: 100 MCQs (5 options)

Student ‘knows’ 50; guesses 25; but ‘educated guesses’ 25 (8 correctly; 17 incorrectly)

Scoring:

Deserved 50 x 4 = 200

Undeserved 5 (20% of 25) x4 – 20xI = 0

Partial 8 x4 -17 x1 = 15

Total 200 + 0 + 15 =215 (out of 400)

If no negative marking ie 0 for incorrect answer)

Total = 200 + (5x4 – 20x0) + (8x4 – 17x0)

Total + 200 + 20 + 32 = 252


Other adjustments
Other Adjustments higher education these days

Order of preference (or ranking) of options

Confidence

Choose option and assign ‘confidence’ level (1, 2, 3)

If correct marks = 1, 2 or 3

If incorrect marks = 0, -2, -6

Normalisation

X → (x-20) * 5/4 (x = unadjusted mark)

Liberal (more attempts) MCQ

Eg 5 option MCQ

1 attempt – score 4/4 = 100% on Question

2 attempts – score (4-1)/4 = 75 % on Question

3 attempts – score (4-2)/4 = 50% on Question etc


Formative online assessment
Formative online assessment higher education these days

On-line approaches

Deakin Studies Online (DSO)

e.g. Blackboard, Moodle

Assessment Management Systems

e.g. QuestionMark, TRIADS

Tools

e.g. Respondus, StudyMate

Techniques for Formative Assessment

Simple

Medium

Advanced

Fun


On line assessment simple

Self Assessment Exercises higher education these days

DSO Self Assessment Tool

MCQs within content using

(simple) javascript

On-line Assessment: Simple


On line assessment medium
On-line Assessment: Medium higher education these days

DSO tests/quizzes

Multiple Choice

True/False

Multiple Answer

Matching Pair

Fill in the blanks

Short Answer

Paragraph/Essay


Example
Example higher education these days

What are the “Christian” names for the Islamic prophets Ibrahim, Daviyd, Musa and Isa?


Multiple choice
Multiple Choice higher education these days


Multiple answer
Multiple Answer higher education these days


Fill in the blanks
Fill in the blanks higher education these days

Answered Correctly

& Alternative Answers


Matching pair
Matching Pair higher education these days


On line assessment advanced
On-line Assessment: Advanced higher education these days

Same as Medium Technique

PLUS

Randomisation of a question’s possible answers

Randomisation of questions

All randomised

Randomised from bank

e.g. any 30 questions from 100

Pre-defined and randomised

e.g. first 10 defined then 30 random questions


On line assessment advanced1
On-line Assessment: Advanced higher education these days

Same as Medium Technique

PLUS

  • Timed assessments (“beat the clock”)

  • Timed released

  • Incentive driven e.g.

    • If score >80% release other learning materials

  • Extra support

    • If score <40% release more learning materials

    • + extra test


On line assessment fun

StudyMate higher education these days

On-line Assessment: Fun

Flash based - can be used within DSO or on standard web pages e.g.

  • Crosswords

  • Pick a Letter

  • Challenge

  • And more…


Useful urls

Useful URLs higher education these days

Approaches to assessment and feedback that foster independent learning

http://www.deakin.edu.au/itl/pd/tl-modules/assessment/index.php

Online assessment Strategies

http://www.deakin.edu.au/itl/dso/strategies-teaching/assessment.php

UK resources to assist MCQ designers

http://www.le.ac.uk/castle/index.html

Studymate

http://www.deakin.edu.au/itl/dso/tools/studymate.php

Wimba Create (CourseGenie)

http://www.wimba.com/products/wimba_create/

Votapedia

http://www.votapedia.com/index.php?title=Main_Page


Contact us

Contact us higher education these days

Institute of Teaching and Learning

http://www.deakin.edu.au/itl/

ITL Support Service

http://www.deakin.edu.au/itl/contact/contact-us.php

And,

Phone: 5227 8127 (ext. 78127)

Email: [email protected]


ad