Assessment

1. Assessment Prof. Tahmina Begum FCPS, MD, M M Ed (UK)

2. Objectives • Functions of assessment • Domains of education • Methods of assessment • Criteria of a good assessment • Advantages & limitations of written methods • Construction of SAQ, MCQ • Construction of OSCE/OSPE • Mention the demerits of traditional oral examination • State the ways and means how the oral examination can be structured • Test construction

3. Assessment To change curricula or instructional methods without changing examination will achieve nothing G E Miller

4. Assessment ~ Learning -Gibbs

5. Why do we assess our students? • To ensure a minimum level of competence • To rank students competitively • To provide feedback to improve students learning (formative) • To evaluate teachers/ course • Tocertify

6. Types of Assessment Formative Summative

7. Types • Formative : attempts to answer the questions ‘How well is the student learning’ ? • Used to monitor progress • Detect learning error & provide feedback to student & tutor

8. Types (cont’) • Summative- (end of course) attempts to answer the question ‘ What has student already learnt’ ? - To ascertain students improvement - To rank student

9. Methods of assessment MethodsInstruments • Written Essay, MCQ, MEQ, SEQ, SAQ, Problem centered/ Case study exam, PMPs • Clinical/Long case, short case, Practical OSCE/ OSPE • Oral Traditional viva/ SOE

10. DOPS,360 degree evaluation, Performance Log, Portfolio Does OSCE, Long & short case Standardized patient Shows how Knows how Clinical scenario based MCQ, Oral, Essay Knows MCQ, Essay, Oral The assessment of clinical skills/ competence/ Performance (Miller GE)

11. Written method Used to test- • knowledge • Application of knowledge

12. Cont’ • Response format • Constructed/ open ended- candidate has to construct answer (fill in or multipage) e.g. Essay, SEQ, MEQ, Short answer, fill ins • Selected/ closed ended, answers are provided in the form of options. E.g. MCQ, PMPs • Stimulus format- Problem centred/ case study exam

13. Cont’ • In the traditional open-ended essay-based assessment, the student is given a topic and instructed to write an extended descriptive narration • The expected reply may run into a few paragraphs (short note) or several pages (long essay)

14. Essay Question Advantage • Ability to write • Power of logic • Original thought • Creativity • Limitations • Low reliability • Subjective marking • Limited coverage

15. Example-Traditional: Write an essay on thyrotoxicosis. Total marks- 25 OR A 46 year old lady presented with loss of weight, palpitation and weakness for 1 year. How will you proceed to diagnose the case? 25

16. Scenario c o m p o n e n t Example: Structure of SEQ A 46-year old female comes to you with a history of generalized weakness, palpitation, and excessive sweating for last 1 year. In spite of good appetite she has lost 10 kg of weight during this period. 1. What other information will you take to obtain for history? 2. What additional physical signs will you look for in this patient? 3. What is the most likely diagnosis? 4. Mention 4 investigations you will advise to support your diagnosis? 5. Outline the steps in the management of this patient. 06 06 04 04 06

17. Short Answer Question (SAQ)

18. Types • Completion type: • Incomplete statement • Candidate supply word, term, symbol etc. • Example- • Hemophilia is due to def. of factor …………… • Define pulse …………………………………………….. • Draw/ label the intrinsic pathway of blood clotting • Name the Vit. K dependent clotting factors

19. Open type: The statement should be worded clearly either in the form of direct question or following a short scenario 1. a) Enumerate 5 important causes of significant jaundice within 24 hours of life 2.5 b) Outline the investigation plan with expected findings to reach a diagnosis 2.5

20. Direct question Q1. a) Name 4 Gm negative bacilli 2.0 b) How do you differentiate endotoxin from exotoxin 3.0 Q 2. a) What is DIC? 1.0 b) Explain in short the mechanism of DIC 4.0

21. Q 1. • Classify diuretics 2.0 • Which diuretic will you use in a patient with heart failure and why 3.0 Q 2. • For bone age determination, X- ray of left wrist joint is advised. Why? Justify your answer 2.5 • How many bones will you expect in a 4 year old boy? 2.5

22. A 14 year old boy has been suffering from high grade fever for 1 month. On examination the boy was found severely pale with purpuric spots all over the body but there was no hepato-splenomegaly. 1. What is your clinical diagnosis? 2 2. Justify your answer 3

23. Reasoning • A 50 year old smoker has been suffering from high blood pressure for 1 year. He is diabetic for last 5 years. • What antihypertensive drug will you use to control BP? Justify your answer. 3.0 • What will happen if BP is not controlled? 2.0

24. Construction of Multiple Choice Questions (MCQ)

25. Multiple choice Questions (MCQs) are fixed responseObjective assessment • Widely used in under & post-graduate exam

26. Why we use MCQs? • Purely objective in scoring & testing • Broad coverage of content/ topics • Efficient use of time • High reliability • Test large no. of candidate • Ease of analysis • Ease of marking - manually/ computer • Banking

27. Limitations • Allow for guessing • Difficult & time consuming to construct • Extended response • Creative thinking can not be • Writing ability assessed • Longer reading time required • Success depends on distracters

28. Types of MCQ • Simple true false • Multiple true-false type • Single best answer • Matching type • Multiple true-false completion type • Assertion and reason • Extended Matching

29. Multiple True-false type • Candidate choose more than one response from a list of possible answers/ options • Options are related to single topic

30. Example Microcytic hypochromic anaemia is found in: • Acute haemorrage • Folate deficiency • Iron deficiency Key- C,D, E • Sideroblastic anaemia • Thalassaemia

31. Following organisms are considered atypical • Chlamydia • Diphtheria • Klebsiella Key- a, e • Mycobacteriam • Mycoplasma

32. Components of MCQ (Item) • Stem- present a problem/ a clinical scenario or a incomplete statement • Lead in- short question that the candidate is asked to answer • Option- 5 options should all be possible answer to the question • Correct answer is KEY, others are distracters

33. PLAN the question Recall Understanding • The learning level Application Clinical judgment problem solving

34. Options (key & distracters) • Similar in terms of grammar, length & complexity • Homogenous or fall into the same category e.g. signs, diagnosis, risk factors, tests, treatment etc. • The best No. is- 3 to 5 • Arrange the list of options systematically (chronological, alphabetical, numerical)

35. Other points need to consider Many DO’s & DO NOT’s

36. DO’s • Clear and concise • Reading time minimum • Specify unit • Use only plausible and logical distracters • Minimum negatively phrased statements in stem, options • Spell out acronyms, abbreviations

37. DO’s • Accurate in grammar & sentence • Continuous with the stem • Use familiar words • Within the syllabus • Important & useful aspect • Link options to each other (e.g. all diagnosis, tests, treatment)

38. DO NOT’s/ Avoid • Too open /ambiguous term e.g. can / possible/may/sometimes/common/ often/seldom-likely to be true • Too absolute term e.g Never / always- tend to be false • Should not contain clueto answer • Avoid uncommon terminology & abbreviation • Avoid double negative

39. Avoid • Implausible item • More than one concept • Incorrect information • Too long statement • Opposite statement in one branch • Avoid clues to correct answer • All or none of the above

40. Avoid Do not use the question-”which of the following statements are correct?” The question is unfocused

41. SBA: Factual Recall of Knowledge The candidate repeats previously learned material by recalling facts, terms, and basic concepts Example: A branch of which cranial nerve supplies the vocal cords ? A. Optic B. Trochlear C. Abducens D. Vestibulocochlear E. Vagus

42. Scenario:A 5 year old girl has been brought to OPD for short stature Lead in: Select the hormones that are likely to be deficient here • Options: • Glucocorticoid • Growth hormone • Insulin Key : b, d, e • Oestrogen • Thyroxin

43. Scenario: A 8 year old boy developed respiratory distress who had puffiness of the face, scanty high colored urine and hypertension Lead in: Select the diuretic that should be administered for initial management in this case • Options: • Ameloride • Frusemide • Mannitol Key: b • Spiranolactone • Thiazide

44. SBA: Pathology Item • A 72-year-old man comes to the physician complaining of the sudden onset of knee pain that began a week ago. He describes the pain as sharp and severe and located in the joint. The knee is swollen, warm to the touch, and the skin over it is red. Joint fluid is aspirated, examined using polarized light microscopy, and shows the following: • What is the most likely diagnosis for the patient’s knee pain? • A. Gout • B. Osteoarthritis • C. Pseudogout • D. Rheumatoid arthritis • E. Traumatic effusion

45. The jugular foramen • The accessory nerve passes • Transmits inf. Petrosal sinus • Is a foramen in occipital bone • Is about 0.5 cm in diameter • Lies lateral to hypoglossal nerve ×a –not gramatically continuous √ a) Gives passage to accessory nerve

46. Avoid using ambiguous/ imprecise/ absolute term Q 1.Microscopic exam of an area of inflammation: a)May reveal eosinophilic accumulation b) Sometimes shows lymphoid follicles Q. The possible side effects of digoxin are- • Arrythmia • Nausea • Headache • ----------- × ambiguous / too open term ‘may’ / ‘sometimes’ /’possible’ have different meaning for different person

47. Cont’ • Haemophillia never occur in female • Hepatomegaly is always found in PEM ×Avoid absolute term ‘‘never’/ always’ the answer is usually false • Malarial parasite often found in blood film during febrile phase × ambiguous term ‘often’

48. The following are notfound inthyrotoxicosis • Weight gain • Heat intolerance • Reduced appetite • Sinus tachycardia • Goitre with bruit × Avoid negative in stem & if possible in item

49. OObjective SStructured CClinical EExamination

50. OSCE / OSPE • Clinical competencies of medical students are assessed uniformly & objectively