usmle step 2 clinical skills
Download
Skip this Video
Download Presentation
“USMLE Step 2 Clinical Skills”

Loading in 2 Seconds...

play fullscreen
1 / 32

USMLE Step 2 Clinical Skills - PowerPoint PPT Presentation


  • 285 Views
  • Uploaded on

“USMLE Step 2 Clinical Skills”. Richard E. Hawkins, MD National Board of Medical Examiners. Presented at the 2005 CLEAR Annual Conference September 15-17 Phoenix, Arizona. Presentation Structure . Overview of USMLE Step 2 CS Measures to ensure fairness and consistency.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about ' USMLE Step 2 Clinical Skills ' - mahlah


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
usmle step 2 clinical skills

“USMLE Step 2 Clinical Skills”

Richard E. Hawkins, MD

National Board of Medical Examiners

Presented at the 2005 CLEAR Annual Conference

September 15-17 Phoenix, Arizona

presentation structure
Presentation Structure
  • Overview of USMLE Step 2 CS
  • Measures to ensure fairness and consistency

Presented at the 2005 CLEAR Annual Conference

September 15-17 Phoenix, Arizona

overview of usmle step 2 cs

Overview of USMLE Step 2 CS

Purpose and Delivery Model

Presented at the 2005 CLEAR Annual Conference

September 15-17 Phoenix, Arizona

usmle purpose
USMLE: Purpose
  • Three step examination system primarily designed to support the medical licensing process
  • Jointly sponsored by
    • Federation of State Medical Boards (FSMB)
    • National Board of Medical Examiners (NBME)
  • Step 2 CS: collaboration with ECFMG

Presented at the 2005 CLEAR Annual Conference

September 15-17 Phoenix, Arizona

usmle purpose5
USMLE: Purpose
  • Overall purpose
    • Assess physician\'s ability to apply knowledge, concepts, and principles, and to demonstrate fundamental patient-centered skills that constitute the basis of safe and effective patient care

Presented at the 2005 CLEAR Annual Conference

September 15-17 Phoenix, Arizona

usmle purpose6
USMLE: Purpose
  • Overall purpose
    • Assess physician\'s ability to apply knowledge, concepts, and principles, and to demonstrate fundamental patient-centered skills that constitute the basis of safe and effective patient care

Presented at the 2005 CLEAR Annual Conference

September 15-17 Phoenix, Arizona

usmle purpose7
USMLE: Purpose
  • Patient-centered Skills
    • History taking / Physical examination
    • Communication and Interpersonal Skills
    • Medical Record Documentation
  • Important for safe and effective patient care
    • History/PE  Diagnosis and Management
    • Communication  Patient Health Outcomes
    • Medical Record  Errors and Patient Safety

Presented at the 2005 CLEAR Annual Conference

September 15-17 Phoenix, Arizona

usmle step 2 cs
USMLE Step 2 CS
  • Enhancement to USMLE Step 2
  • USMLE Step 2 – components:
    • Clinical knowledge (CK)
    • Clinical skills (CS)
  • Standard for Step 2 – appropriate for entry into postgraduate training

Presented at the 2005 CLEAR Annual Conference

September 15-17 Phoenix, Arizona

usmle step 2 cs logistics
USMLE Step 2 CS: Logistics
  • Regional delivery model
    • Optimal combination of convenience, cost-efficiency and standardization
  • Five regional test centers across US
  • Projected examine volume – 30,000+ / year
  • Individual center capacity
    • 3 examinations / day (33 examinees); up to 7 days/week

Presented at the 2005 CLEAR Annual Conference

September 15-17 Phoenix, Arizona

requirements of a high stakes performance based examination

Requirements of a High StakesPerformance-based Examination

Ensuring Fairness and Consistency

Presented at the 2005 CLEAR Annual Conference

September 15-17 Phoenix, Arizona

threats to validity and reliability
Threats to Validity and Reliability
  • Content and/or tasks not relevant or realistic
  • Individual test forms vary in content coverage
  • Scoring methods not appropriate for skills tested
  • Inconsistency in SP portrayal and scoring
    • Between cases, across sites, over time
  • Level of difficulty of cases / exams inconsistent
  • Standard setting approach appropriate

Presented at the 2005 CLEAR Annual Conference

September 15-17 Phoenix, Arizona

usmle step 2 cs content
USMLE Step 2 CS: Content
  • Step 2 CS Blueprint
    • Defines content categories
    • In meeting blueprint specifications, each test form provides:
      • Adequate sampling of content domain
      • Comparable content between test forms

Presented at the 2005 CLEAR Annual Conference

September 15-17 Phoenix, Arizona

usmle step 2 cs content13
USMLE Step 2 CS: Content

Blueprint Content Categories:

  • Common and important medical problems / patient presentations
  • Acuity
  • Age
  • Gender
  • Race / ethnicity

Presented at the 2005 CLEAR Annual Conference

September 15-17 Phoenix, Arizona

slide14

Case Content

Cardiovascular

Respiratory Gastrointestinal

Musculoskeletal

Constitutional

Neurological

Psychiatric Genitourinary

Women’s health

Unclassified / multi-system

Case Acuity

Acute

Subacute / Chronic

Test Form

Patient age

Age less than 18

Age 18 – 44

Age 45 – 64

Age 65 +

Patient Gender

Male

Female

Presented at the 2005 CLEAR Annual Conference

September 15-17 Phoenix, Arizona

step 2 cs structure tasks
Step 2 CS: Structure & Tasks
  • 12 patient encounters
  • 15 min. for encounter / 10 min. for patient note
  • Each encounter:
    • Elicit pertinent history,

Perform appropriate physical examination,

Communicate effectively

    • Document:
      • Findings from the history and physical
      • Diagnostic impression / Further work-up

Presented at the 2005 CLEAR Annual Conference

September 15-17 Phoenix, Arizona

step 2 cs score components
Step 2 CS: Score Components
  • Integrated Clinical Encounter (ICE)
    • Data gathering: history and physical exam
    • Patient note
  • Communication / Interpersonal Skills (CIS)
    • Gathering information; sharing information; manner & rapport
  • Spoken English Proficiency (SEP)
    • Listener effort, examinee pronunciation / word choice

Presented at the 2005 CLEAR Annual Conference

September 15-17 Phoenix, Arizona

scoring approach ice
Scoring Approach: ICE
  • Data gathering (Hx / PE)
    • Dichotomous checklists completed by SPs
          • +
  • Patient note
    • Physician raters using holistic methods
          • =
  • Clinical Process + Clinical Outcome Measure

Presented at the 2005 CLEAR Annual Conference

September 15-17 Phoenix, Arizona

case and checklist development
Case and Checklist Development
  • Iterative Process
    • Involves test committees
    • Focus on clinical presentation
    • Checklists “limited”
    • Encounters with SPs
    • Appropriateness of content and difficulty
  • Stepwise progression through pilot and calibration stages
    • Validation / refinement via review of examinee performance data

Presented at the 2005 CLEAR Annual Conference

September 15-17 Phoenix, Arizona

patient note
Patient Note
  • Holistic scoring
  • Rater training
    • General policies
    • Case specific:
      • Consensus development on key features
    • Calibration phase
    • Quality Assurance

Presented at the 2005 CLEAR Annual Conference

September 15-17 Phoenix, Arizona

derivation of communication interpersonal skills cis scale
Derivation of Communication / Interpersonal Skills (CIS) Scale
  • Initial instrument – validated ECFMG scale
  • Reviewed against national consensus recommendations and commonly used scales

Presented at the 2005 CLEAR Annual Conference

September 15-17 Phoenix, Arizona

cis subscales
CIS Subscales
  • Data gathering skills
    • Open-ended questions, transitional statements, not interrupting the patient
  • Information sharing skills
    • Responsiveness to patient questions/concerns, provision of counseling when appropriate, avoidance of jargon
  • Personal manner and rapport
    • Expression of interest in the impact of the illness, concern for patient comfort and modesty

Presented at the 2005 CLEAR Annual Conference

September 15-17 Phoenix, Arizona

step 2 cs scoring
Step 2 CS: Scoring
  • Reporting is Pass or Fail only
  • Examinees must pass all three subcomponents
  • Feedback to examinees
    • Performance report – overall and subcomponent outcomes
    • Failing examinees only – graphical profiles
      • Intended to show relative strengths and weaknesses

Presented at the 2005 CLEAR Annual Conference

September 15-17 Phoenix, Arizona

slide23

Sample PerformanceReport

Presented at the 2005 CLEAR Annual Conference

September 15-17 Phoenix, Arizona

sample performance profile
Sample Performance Profile

Lower Performance

Borderline

Performance

Higher Performance

Integrated Clinical Encounter

Data Gathering

Patient Note

Communication / Interpersonal Skiills

Spoken English Proficiency

XXXXXXXXXXXXX

XXXXXXXXXXXXXXXX

XXXXXXXXXXXXXXXX

XXXXXXXXXXXXXX

XXXXXXXXXXXXX

Presented at the 2005 CLEAR Annual Conference

September 15-17 Phoenix, Arizona

threats to validity and reliability25
Threats to Validity and Reliability
  • Content and/or tasks not relevant or realistic
  • Individual test forms vary in content coverage
  • Scoring methods not appropriate for skills tested
  • Inconsistency in SP portrayal and scoring
    • Between cases, across sites, over time
  • Level of difficulty of cases / exams inconsistent
  • Standard setting approach appropriate

Presented at the 2005 CLEAR Annual Conference

September 15-17 Phoenix, Arizona

ensuring fairness and consistency
Ensuring Fairness and Consistency
  • Rigorous SP (and SP trainer) training
  • Meticulous attention to quality assurance
  • Application of equating procedures

Presented at the 2005 CLEAR Annual Conference

September 15-17 Phoenix, Arizona

sp and sp trainer training
Standardized Patients

Rigorous training – generic and case specific

Electronic delivery (“Ecase”) of case materials

Sign off process for SPs – Criteria : # portrayals, tests

SP Trainers

Adherence to training protocols

“Training academy” for SP trainers

SP and SP Trainer Training

Presented at the 2005 CLEAR Annual Conference

September 15-17 Phoenix, Arizona

quality assurance
Quality Assurance
  • Begins with the SP sign-off process
  • Monitoring procedures and analyses:
    • Qualitative (portrayal and scoring accuracy):
      • Live and video review of SP performances
    • Quantitative
      • Score-based analyses
      • Case level and item level comparisons

Presented at the 2005 CLEAR Annual Conference

September 15-17 Phoenix, Arizona

equating procedures
Equating Procedures
  • Within site
    • SP-case combination
  • Between site
    • Central video review

Data gathering

Communication and Interpersonal Skills

  • Patient note rater – case combination

Presented at the 2005 CLEAR Annual Conference

September 15-17 Phoenix, Arizona

standard setting
Standard Setting
  • USMLE Standard Setting System
    • Committee assigned responsibility to establish and monitor standards
  • Decision-making process
    • Survey of constituent opinion
    • Standard-setting exercises by independent groups
    • Examinee performance data and score reliability

Presented at the 2005 CLEAR Annual Conference

September 15-17 Phoenix, Arizona

conclusions
Conclusions
  • Numerous logistical and psychometric challenges were identified and responses described
  • In order to ensure exam fairness and consistency:
    • Sound test and case development practices
    • Intensive SP and SP trainer training
    • Rigorous quality assurance
    • Well-considered equating and standard setting procedures
  • Supported by ongoing research on reliability and validity

Presented at the 2005 CLEAR Annual Conference

September 15-17 Phoenix, Arizona

speaker contact information
Speaker Contact Information

Richard E. Hawkins, MD

National Board of Medical Examiners

3750 Market Street

Philadelphia, PA 19104

215-590-9204 / Fax 215-590-9440

Email: [email protected]

Websites:

http://www.usmle.org

http://www.nbme.org

Presented at the 2005 CLEAR Annual Conference

September 15-17 Phoenix, Arizona

ad