the role of formal assessment in medical regulation iamra 2010
Download
Skip this Video
Download Presentation
The Role of Formal Assessment in Medical Regulation IAMRA 2010

Loading in 2 Seconds...

play fullscreen
1 / 22

The Role of Formal Assessment in Medical Regulation IAMRA 2010 - PowerPoint PPT Presentation


  • 87 Views
  • Uploaded on

The Role of Formal Assessment in Medical Regulation IAMRA 2010. Stephen I. Schabel MD , FACR Professor of Radiology Medical University of South Carolina Formerly of. SC SBME. USMLEComposite Step3 Step1. FSMB-BOD.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about ' The Role of Formal Assessment in Medical Regulation IAMRA 2010' - katoka


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
the role of formal assessment in medical regulation iamra 2010
The Role of Formal Assessment in Medical Regulation IAMRA 2010

Stephen I. Schabel MD , FACR

Professor of Radiology

Medical University of South Carolina

Formerly of

SC SBME

USMLEComposite

Step3

Step1

FSMB-BOD

ECFMG

NBME

medical regulators
Medical Regulators
  • USA = States – Medical licensure a state function
  • “State funded” – by licensure fees – usually most taken by state government
  • Charge – PROTECT the PUBLIC – from substandard physicians
  • NO resources to produce independent knowledge examinations
  • Therefore have to trust SOMEONE
initial medical licensure in us
Initial Medical Licensure in US
  • Education - M.D. , D.O. ,IMG
  • Examination of Knowledge – USMLE
  • Post Graduate Training – ACGME ( 2-3 Years)
  • External Check – FSMB / State
united states medical licensure examinations history evolution
United States Medical LicensureExaminations - History/Evolution
  • Independent State Licenses – circa 1900
  • Independent State Examinations – FSMB 1912 -reciprocity
  • Medical Education Quality – marked variation – Flexner 1910
  • Post Graduate Training – contact driven – great variation pre ABMS 1933
  • NBME – 1915 – badge of Honor/ excellence (LCME only)
  • ECFMG 1957 – Federal Charter VQE – NBME prepared
  • FLEX – 1960’s – ( NBME prepared ) states used (IMG /US)
  • USMLE – 1994 – single pathway to licensure
why equity of common standards
Why? Equity of Common Standards

1964

Assessed: 20 state medical licensing exams

Findings: Concerns for validity issues and content quality

George Miller, MD

Office of Research in Medical Education

University of Illinois College of Medicine

1965

Reviewed fail rates for state board examinations, 1954-1964

  • 3 states failed no one all 10 years
  • 2 states failed no one 9 out of 10 years
  • 4 states failed no one 8 out of 10 years

Robert Derbyshire, MD

Federation Bulletin, April 1965

united states an egalitarian country
United States – An Egalitarian Country
  • As Americans we want to believe that an individuals effort and ability are the only limitations to eventual success
  • At least by providing a pathway for success by ability we are closer to our ideal
  • In Medicine Individual Ability and Commitment is the only limitation to success
us a country of volunteers
US a Country of Volunteers
  • Self regulation an important principal
  • Professionals historically willing to participate
  • Medical Board Members , FSMB, ECFMG, and USMLE/NBME – “The National Faculty of Medicine”
why mcq s
Why MCQ’s
  • Psychometrically most valid and cost effective ( still expensive) – A Type
  • We are a country of MCQ test takers
  • Other formats are being explored
formal assessments in medicine in united states a national standard
Formal Assessments in Medicine in United States – A National Standard
  • You may think Harvard is better than University of South Carolina – but if a SC student get a +3 SD score – you know they are smart
  • MCAT – Medical College Assessment Test (SAT)
  • USMLE – Unites States Medical Licensure Examination
  • ABMS – American Board of Medical Specialties – certification and MOC
  • SPEX – Special Purpose Examination
  • We have learned to LOVE standardized Tests
abms certification
ABMS Certification
  • No state requires ABMS certification for licensure – all states accept it as a mark of competence
  • Many Hospitals DO require ABMS certification for staff privileges
  • More and More payers require ABMS certification for reimbursement
flex nbme usmle why a single exam we don t want to be sued
FLEX NBME-USMLE – Why a Single Exam We Don’t Want To Be Sued
  • Prior to 1994 IMGs- had to take FLEX - 60% passed
  • Most US LCME students took NBME -99% passed but could take FLEX
  • FSMB was sued – unfair – FLEX must be a harder
  • USMLE - % unchanged
as licensure board residency director patient what can a regulator believe
As Licensure Board/Residency Director/PatientWhat Can a Regulator Believe?
  • What the doctor says or writes?
  • Patient testimonials?
  • The letters or recommendation / Dean’s letter / Program Directors Letter
  • Medical School Transcript
  • License in Another State or Country
  • USMLE – ABMS …..
still needed
Still Needed?
  • Who can you Trust?
  • Every student in an LCME accredited medical school who takes USMLE Step 1 and 2 – has been certified as competent by their medical school’s Dean of Students and faculty as competent – 5% + are judged substandard
  • 100% of candidates for Step 3 of USMLE have an MD degree and 3% + are found inadequate
  • Not the Deans of Students of Medical Schools!
  • Every candidate for ABMS Board Certifying Examinations has been recommended by their program director as competent and many (5-30%) are found inadequate.
  • Not Program Directors!
lake wobegon effect
Lake Wobegon Effect
  • 68% of U. Nebraska Faculty rate themselves in top 25%
  • 87% of Stanford MBA students rate themselves above mean
  • 70% of US college bound students rate themselves above average – 25% in top 1 %
  • 93% of US drivers say their ability in top 50%
medical licensure examinations what would you really like
Medical Licensure ExaminationsWhat Would You Really Like
  • Accurate Measure of Knowledge needed to practice ( as predictor of competence)
  • Quality – secure ,current and correct content , high fidelity
  • Complete but not Excessive – 39 Hours total
  • Inexpensive (US $2700)/ Convenient (CBT- 100’s centers/CK 5 centers)
  • Fair – free of bias toward women, minorities, regions , medical schools , IMGs ( English)
  • Portability/ Universal Acceptance
usmle a model of medical evaluation
USMLE – A Model of Medical Evaluation
  • Single Examination for all M.D.s – open to all Medical Graduates (M.D. and D.O.) – 140,000/year
  • Single Standard - everyone held to the same one
  • Fixed Standard – all could pass or fail
  • Three Steps
usmle 3 steps
USMLE 3 Steps
  • Step 1 – after basic sciences – fundamental scientific knowledge necessary to beginning clinical education – 1 day MCQ CBT
  • Step 2 CK – after clinical rotations Medical school – clinical knowledge necessary to begin supervised Post Graduate training – 1 day MCQ CBT
  • Step 2 CS – clinical skills , communication necessary to begin supervised practice – 1 day multi station OSKE
  • Step 3 – knowledge necessary to begin unsupervised practice ,management – 2 days CBT MCQ
examinations of the 6 competencies
Examinations of the 6 Competencies
  • ACGME-ABMS- Medical school education – accepted 6 competencies of Physicians
  • Patient Care
  • Medical Knowledge
  • Practice based Learning
  • Interpersonal and Communication Skills
  • Professionalism
  • Systems based Practice
  • USMLE MCQs- first 2 – Step 2 CS –communication
  • Others very difficult – evaluation methodologies either unavailable or lesser psychometrically weaker – longer / more expensive
  • Research ongoing
why usmle makes sense for us
Why USMLE Makes Sense For US
  • Economies of scale – states and even regions lack adequate recourses
  • National Exam -attracts National Faculty of Medicine - Schools can examine curriculum ,teaching and student performance and testing strategies
  • Mobile physician and patient populations
why not accept international certifications
Why Not Accept International Certifications
  • Little First Hand Knowledge – Unknown Quality
  • Accept ONE have to accept ALL
  • Medicine Elsewhere Different than US – USMLE in France – 10% of questions answered differently –PPD reactivity and BCG
ad