1 / 26

MULTI-STATION CLINICAL EXAMINATIONS Win May

MULTI-STATION CLINICAL EXAMINATIONS Win May. MSCE - Advantages. Standardized administration Potentially good reliability Can provide excellent individual feedback from expert clinicians/trained raters. MSCE - Disadvantages. Expensive to prepare and administer

twila
Download Presentation

MULTI-STATION CLINICAL EXAMINATIONS Win May

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. MULTI-STATION CLINICAL EXAMINATIONS Win May ACMD 512

  2. MSCE - Advantages • Standardized administration • Potentially good reliability • Can provide excellent individual feedback from expert clinicians/trained raters ACMD 512

  3. MSCE - Disadvantages • Expensive to prepare and administer • Educational and clinical experts need to work together • Can be logistically difficult to administer ACMD 512

  4. Basic Decisions • Why Evaluate? • Who to Evaluate? • When and Where Evaluate? • What to Evaluate? • What Criteria for Pass/Fail? • Data Collection and Analysis? • What Feedback and Action? ACMD 512

  5. Why Evaluate? • Determine how results to be used • Individual learner evaluation • Formative - diagnostic • Summative - part of grade • Summative - certification • Program evaluation • Decide precision of results needed • Decide who will use results ACMD 512

  6. Who to Evaluate? • Select level of learner and the point in training for the evaluation • Determine how many learners to evaluate • Individual decisions - all learners at selected level • Program decisions - representative sample ACMD 512

  7. When and Where Evaluate? • Time • Amount of testing time per student • Total time needed for testing of all students • Resources • Personnel (faculty, staff, patients) • Monetary resources • Space • How much available • When available ACMD 512

  8. What to Evaluate? Selection of Tasks • List general skill components and specific content which could be tested • Make station/content decisions • Develop test plan ACMD 512

  9. Number of stations per learner • Depends on: • Purpose of test • Time available per testing session • Amount of time per station task • Time set aside for feedback • Who is to rate • Physical layout of testing site • Break needed - raters, students ACMD 512

  10. Example • Time / Learner - 4 Hours (240 minutes) • Time Per Station Task - 15 Minutes • Time for Feedback - 5 Minutes per station • Who Will Rate - Faculty • Physical Layout - 2 minutes in between stations • Break Needed - 15 Minutes Result: 240 Min - 15 Min for break = 225 Min / 22 Min per Station = 10 stations ACMD 512

  11. Other station/content decisions • Stations static or changing : • Static - content can become known • Changing - differential difficulty • Content and skills- should be based on • objectives • common / important problems seen by your learner on graduation/end of residency ACMD 512

  12. Grid for Case Selection ACMD 512

  13. Individual StationDecisions • Determine station content and task(s) • Make sure tasks fit in the time limit • Develop case materials • Locate needed supplies • Train standardized patient • Determine how performance in station will be rated/scored • Dichotomous - Yes/No • Rating Scale ACMD 512

  14. Individual StationDecisions • Determine who will evaluate - faculty or standardized patient • Train evaluators • If feedback is to be provided, train to provide effective feedback ACMD 512

  15. Pass/Fail Decisions • Normative - compared to a group - cut point generally set between 1.5 and 2.0 standard deviations below the mean • Criterion Referenced - compared to a standard ACMD 512

  16. Some criteria to consider • Petrusa (2004) - Minimal acceptable clinical practice • Payne (2008) - Critical items - elements critical for optimal clinical outcomes ACMD 512

  17. Pass/Fail Decisions • Essential Items • A set criteria using a list of essential action items identified by experts • Must pass 4 out of 6 cases ACMD 512

  18. Example: Abdominal Pain Case Summary A 22-year-old female who has been brought to the emergency room by her sister. She has been experiencing stomach pains since yesterday afternoon. The pain started while she was at work and it has been constant ever since. The pain has become progressively more severe. ACMD 512

  19. Example:Abdominal Pain Essential Items * Students are allowed to miss any 1 of the 6 items to pass the case. ACMD 512

  20. Data Collection - Logistics • Coordinator in charge of station development and administration for all stations • Administrative support needs to be obtained to ensure that the test happens as intended ACMD 512

  21. Data Analysis • Data analysis should include • Results by skill scores per station as well as overall score to facilitate effective feedback to learner • Item analysis for each station to facilitate post hoc review ACMD 512

  22. Feedback and Action • Feedback - timely and meaningful feedback to students, administrators, test and station coordinators, and raters. • Action - What happens to students who perform poorly needs to be decided in advance. ACMD 512

  23. Remediation • Video Review • Self-Reflection • Standardized Patient practice • Work with Faculty • Repeat MSCE ACMD 512

  24. Post-Hoc Review • Post-administration review should include: • Reflection on examination process • Subjective and objective review of examination results for purposes of: • Individual student performance • Overall group performance • Station by station results ACMD 512

  25. In summary • Why Evaluate? • Who to Evaluate? • When and Where Evaluate? • What to Evaluate? • What Criteria for Pass/Fail? • Data Collection and Analysis? • What Feedback and Action? ACMD 512

  26. Thank you! ACMD 512

More Related