principles of good assessment and standard setting l.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
Principles of Good Assessment and Standard Setting PowerPoint Presentation
Download Presentation
Principles of Good Assessment and Standard Setting

Loading in 2 Seconds...

play fullscreen
1 / 12

Principles of Good Assessment and Standard Setting - PowerPoint PPT Presentation


  • 169 Views
  • Uploaded on

Principles of Good Assessment and Standard Setting. Katharine Boursicot Reader in Medical Education St George’s, University of London. Principles. Examine what is being assessed Review methods/ blueprinting Consider if formats suited to needs Standard setting.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Principles of Good Assessment and Standard Setting' - tiernan


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
principles of good assessment and standard setting

Principles of Good Assessment and Standard Setting

Katharine Boursicot

Reader in Medical Education

St George’s, University of London

principles
Principles
  • Examine what is being assessed
  • Review methods/ blueprinting
  • Consider if formats suited to needs
  • Standard setting
critical questions in assessment
Critical questions in assessment
  • WHY are they doing the assessment?
  • WHAT are they assessing?
  • HOW are they assessing it?
  • HOW WELL is the assessment working?
1 why are they doing the assessment
1. WHY are they doing the assessment?
  • Is its purpose:
    • Formative?
    • Summative?
2 what are they testing
2. WHAT are they testing?
  • Elements of competence
    • Knowledge
      • factual
      • applied: clinical reasoning
    • Skills
      • communication
      • clinical
    • Attitudes
      • professional behaviour

Tomorrow’s Doctors, GMC 2003

3 how are they doing the assessment test formats

Performance assessment in vivo:

Video, WBA eg mini-CEX, DOPs

Does

Performance assessment in vitro:

OSCEs

Knows how

(Clinical) Context based tests:

SBAs, EMQs, SAQ

Knows

Factual tests:

SBAs

Shows how

3. How are they doing the assessment? Test formats

Does

Shows how

Knows how

Knows

4 how well is the assessment working
4. HOW WELL is the assessment working?

Evaluation of assessment systems

  • Is it valid?
  • Is it reliable?
  • Is it doing what it is supposed to be doing?
  • To answer these questions, we have to consider the characteristics of assessment instruments
utility function
Utility function
  • U = Utility
  • R = Reliability
  • V = Validity
  • E = Educational impact
  • A = Acceptability
  • C = Cost
  • W = Weight

U = wrR x wvV x weE x waA x wcC

utility function9

100%

0%

In-training

assessment

EMQ/MCQ

Utility function
  • U = Utility
  • R = Reliability
  • V = Validity
  • E = Educational impact
  • A = Acceptability
  • C = Cost
  • w = Weight

Weight

U = wrR x wvV x weE

utility function10

100%

Overall

assessment

programme

0%

Individual

instrument

Utility function
  • U = Utility
  • R = Reliability
  • V = Validity
  • E = Educational impact
  • A = Acceptability
  • C = Cost

Weight

U = wrR x wvV x weE

setting the pass mark characteristics of credible standards
Setting the pass mark: characteristics of credible standards
  • Are they using a recognised method?
  • Evidence....
  • How are the standards applied?

The method has to be:

  • defensible
  • credible
  • supported by a body of evidence in the literature
  • feasible
  • acceptable to all stakeholders
    • Norcini, J. J. (2003). Setting standards on educational tests. Medical Education, 37, 464-469.
principles of assessment
Principles of Assessment
  • There is no perfect assessment: compromise is always required
    • The compromise depends on the context of the assessment
  • TheQuality of assessments is a matter of the integral assessment programme, rather than of the individual instruments