1 / 10

Performance-based standard setting methods

Performance-based standard setting methods. Katharine Boursicot Trudie Roberts. Performance-based standard setting methods. Borderline group method Contrasting group method Regression based standard method.

overton
Download Presentation

Performance-based standard setting methods

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Performance-based standard setting methods Katharine Boursicot Trudie Roberts

  2. Performance-based standard setting methods • Borderline group method • Contrasting group method • Regression based standard method Kramer A, Muijtjens A, Jansen K, Düsman H, Tan L, van der Vleuten C Comparison of a rational and an empirical standard setting procedure for an OSCE, Medical Education, 2003 Vol 37 Issue 2, Page 132 Kaufman DM, Mann KV, Muijtjens AMM, van der Vleuten CPM. A comparison of standard-setting procedures for an OSCE in undergraduate medical education. Acad Med 2000; 75:267-271.

  3. The examiner’s role in standard setting • Uses the examiner’s clinical expertise to judge the candidate’s performance • Examiner allocates a global judgement based on the candidate’s performance at that station • Remember thelevelof the examination Pass Borderline Fail

  4. Borderline Group Method Test score distribution Checklist 1. Hs shjs sjnhss sjhs sjs sj 2. Ksks sksmsiqopql qlqmq q q qkl 3. Lalka kdm ddkk dlkl dlld 4. Keyw dd e r rrmt tmk 5. Jfjfk dd 6. Hskl;s skj sls ska ak akl ald 7. Hdhhddh shs ahhakk as TOTAL     Borderline score distribution  Pass, Fail, Borderline Passing score

  5. Contrasting groups method Test score distribution Checklist 1. Hs shjs sjnhss sjhs sjs sj 2. Ksks sksmsiqopql qlqmq q q qkl 3. Lalka kdm ddkk dlkl dlld 4. Keyw dd e r rrmt tmk 5. Jfjfk dd 6. Hskl;s skj sls ska ak akl ald 7. Hdhhddh shs ahhakk as TOTAL    Fail Pass   Pass, Fail, Borderline P/B/F Passing score

  6. 1 2 3 4 5 Regression based standard Checklist X= passing score 1. Hs shjs sjnhss sjhs sjs sj 2. Ksks sksmsiqopql qlqmq q q qkl 3. Lalka kdm ddkk dlkl dlld 4. Keyw dd e r rrmt tmk 5. Jfjfk dd 6. Hskl;s skj sls ska ak akl ald 7. Hdhhddh shs ahhakk as TOTAL  Checklist Score  X    Overall rating 1 2 3 4 5 Clear Borderline Clear Excellent Outstanding fail pass 1 = Clear fail 2 = Borderline 3 = Clear pass 4 = Excellent 5 = Outstanding

  7. Performance-based standard setting • Advantages • Utilises the expertise of the examiners • they are observing the performance of the students • they are in a position to make a (global) judgement about the performance • based on • their clinical expertise • expected standards for the level of the test • knowledge of the curriculum/teaching

  8. Borderline/ contrasting/ regression-based methods • Advantages • Large number of examiners set a collective standard while observing the candidates – not just an academic exercise • Reliable: cut-off score based on large sample of judgments • Credible and defensible: based on expert judgment in direct observation

  9. Performance-based standard setting • Disadvantages • Requires large cohort of candidates to achieve enough numbers in the ‘borderline’ group • Passing score not known in advance • Judgments not independent of checklist scoring • Requires expert processing of marks immediately after the exam • Checking of results • Delay in producing results

More Related