Statistics of EBO 2009 Examination Symposium EBO “Training for the Trainers” 116 ème Congrès de la Société Française d’Ophtalmologie (SFO). Danny G.P. Mathysen MSc. Biomedical Sciences EBOD Assessment and Executive Officer
Danny G.P. MathysenMSc. Biomedical Sciences
EBOD Assessment and Executive Officer
Antwerp University Hospital, Department of OphthalmologyWilrijkstraat 10, B-2650 Edegem, BelgiumE-mail: [email protected]
Yearly increase of candidates
Many EU countries apply
No significant difference!
No significant differences!
Residents tend to have higher total MCQ scores with lower standard deviations when compared to specialists.
In general there are no statistically significant differences between countries.
EBOD is not a language test!
internal consistencyof EBOD MCQ-test is good
correlation betweenitem and total MCQ score
-1 0 +1
0 58 79 100
NEGATIVE MARKINGAT EBOD 20101
Spread of total test scoreswith negative marking
Spread of total test scoreswithout negative marking
-130 0 260
NEGATIVE MARKINGDOES NOT INFLUENCEPASS RATE!
Synonyms: standard, cutpoint
A pass mark is a special score that serves as boundary between those who perform well enough and those who do not
How to set pass marks? Reaching a consensus rather than obtaining a scientifically correct solution
The purpose of an examination is to select the group of candidates that perform well enough (pass) and to eliminate the group of candidates that do not perform well enough (fail)
In order to achieve this goal, a (limited) number of questions are presented to the candidates
The discriminative power of the examination will depend on the validity of the questions used
Degree of difficulty of questions
Can be assessed by calculating the P-value (i.e. percentage of candidates answering correctly)
Degree of discriminative power of questions
Objective measurement of the degree to which the question is able to discriminate strong from weak candidates
Can be assessed by calculating the Rit/Rir value (correlation of question score to total examination score)
Avoid questions with P-value above 0.90 or below 0.10
Avoid questions with Rit-value below 0.20
“absolute” pass mark (criterion-reference)
expressed as a number (e.g. 70 correct responses) of test questions
expressed as a percentage (e.g. 70 % correct responses) of test questions
“relative” pass mark (norm-reference)
expressed as a number (e.g. 50 best performers) of examinees
expressed as a percentage (e.g. top 20 % performers) of examinees
Validation of questions
NEGATIVE MARKINGDOES NOT INFLUENCETHE PASS RATE !!!
EBOD scores are high!
EBOD scores arecomparable for MCQand Viva Voce!
Residents have higher MCQ and Viva Voce scores with lower standard deviations when compared to specialists.
Success rate of EBOD is much higher ascompared to other medical specialties (60-70 %)
EBOD success rate is quite stable over the years and quite high as the level of candidates usually tends to be good.
18 Residents (out of 220: 8.2%)and 14 specialists (out of 88: 15.9 %)failed at EBOD 2009. As there were 308 candidates the general failure rate was 10.4 %.
Careful monitoring of EBOD
Careful validation of EBOD
Reliable results of EBOD
Stable results of EBOD over the years
Result of careful pre- and post-assessment of EBOD
Publication on EBO website