PREPARING EFFECTIVE MULTIPLE CHOICE EXAMINATION QUESTIONS

1. PREPARING EFFECTIVE MULTIPLE CHOICE EXAMINATION QUESTIONS PATRICK DUFF, M.D.

2. LEARNING OBJECTIVES • Describe the levels of complexity of an examination question • Describe the types of examination questions • Describe desirable and undesirable features of examination questions

3. KEY REQUIREMENTS FOR AN EXAMINATION

4. COMPLEXITY OF EXAMINATION QUESTIONS

5. TYPES OF EXAMINATION QUESTIONS

6. TYPES OF EXAMINATION QUESTIONS

7. EXAMPLERECALL • Which of the following is the most important risk factor for squamous cell carcinoma of the lung? A. Age > 50 B. Diabetes C. Smoking D. Obesity

8. EXAMPLEINTERPRETATION • A 24 year old nulliparous woman at 16 weeks gestation had the following values on her serum analyte screening test: AFP …below the mean HCG…above the mean Estriol…below the mean Inhibin…above the mean

9. EXAMPLE INTERPRETATION • Which of the following chromosome abnormalities is her baby most likely to have? A. Triploidy B. Trisomy 21 C. 45, X D. Trisomy 18

10. EXAMPLEINTERPRETATION • A term neonate has hepatosplenomegaly, thrombocytopenia, and the diffuse rash noted in the photograph.

11. EXAMPLEINTERPRETATION • Which congenital infection is most likely? • A. Parvovirus • B. CMV • C. HIV • D. Mumps

12. EXAMPLEPROBLEM SOLVING • A 22 year old male has a 48 hour history of malaise, fever (39 deg C), shaking chills, and cough productive of rust-colored sputum. On physical examination, there is dullness to percussion in the lower left lung field, and tubular breath sounds can be heard in this area.

13. EXAMPLEPROBLEM SOLVING • Which of the following is most appropriate for treatment of this patient? A. Isoniazid B. Fluconazole C. Ceftriaxone D. Acyclovir

14. APPROPRIATE MIX OF EXAMINATION QUESTIONS

15. GUIDELINES FOR AN EFFECTIVE TYPE-A QUESTION • Determine a single clear objective prior to writing the question • Have only one objective per question • Make sure the question is at an appropriate level of difficulty for the target audience

16. GUIDELINES FOR AN EFFECTIVE TYPE-A QUESTION • Correct response should be obvious to the well-informed examinee before the answers are read • If the examinee has to read all of the answers before deciding, the question is probably a K-type “disguised” as an A-type

17. TYPE A QUESTIONGOOD EXAMPLE • Which of the following is the most common complication of multiple pregnancy? A. Gestational diabetes B. Placenta previa C. Postpartum hemorrhage D. Preterm delivery

18. BEWARE THE PSEUDO-A QUESTION • All of the following statements about preterm delivery are true except • The frequency is 10 %. • It can be prevented by use of progesterone. • It is more common in whites than in African-Americans. • It may result in neonatal RDS.

19. GUIDELINES FOR AN EFFECTIVE TYPE-A QUESTION • Include sufficient information in the stem, but avoid unnecessary verbiage • Be certain the question has a clear, unambiguous answer

20. GUIDELINES FOR AN EFFECTIVE TYPE-A QUESTION • Include 4 to 5 possible answers • Be certain the incorrect answers are reasonably plausible

21. TYPE A QUESTIONBAD EXAMPLE • What is the risk of pregnancy loss after a second trimester amniocentesis? A. 0.5% B. 10 % C. 25 % D. 50 %

22. GUIDELINES FOR AN EFFECTIVE TYPE-A QUESTION • Do not use “none of the above” or “all of the above” as distractors • Confusing • Rewards guessing

23. GUIDELINES FOR AN EFFECTIVE TYPE-A QUESTION • Avoid clues in the responses such as answers of unequal length or repetition of key words

24. GUIDELINES FOR AN EFFECTIVE TYPE-A QUESTION • Avoid compound answers, ie, answers that include multiple actions

25. TYPE A QUESTIONBAD EXAMPLE • Which of the following is the most appropriate treatment for community-acquired pnemonia? • A. Clindamycin • B. Vancomycin • C. Valacyclovir • D. Administer oxygen, check blood gases, and administer levofloxacin

26. GUIDELINES FOR AN EFFECTIVE TYPE-A QUESTION • Be wary of “except” type questions, especially when a “double negative” is present

27. GUIDELINES FOR AN EFFECTIVE TYPE-A QUESTION • In certain instances, an “except” type question may be appropriate and may accurately assess the breadth of an examinee’s knowledge

28. TYPE A QUESTIONGOOD EXAMPLE • All of the following are risk factors for gestational diabetes except A. Obesity B. Positive family history C. Age < 25 y D. History of a macrosomic child

29. GUIDELINES FOR AN EFFECTIVE TYPE-A QUESTION • Be certain that illustrative materials are clear and that they reproduce well • Vary the correct response among the letters a - d

30. GUIDELINES FOR AN EFFECTIVE TYPE-B QUESTION • Clearly identify the nature of the items in column A and column B • Include at least 1 or 2 unused distractors

31. GUIDELINES FOR AN EFFECTIVE TYPE-B QUESTION • Indicate if answers in one of the columns may be used only once, more than once, or not at all • Be certain the matching items are reasonably related to one another

32. GUIDELINES FOR AN EFFECTIVE TYPE-B QUESTION • Matching items are particularly useful for testing interpretation skills • X-ray • ECG • Sonogram • Histology

33. TYPE B QUESTIONGOOD EXAMPLE

34. TYPE B QUESTIONBAD EXAMPLE

35. CONCLUSIONS • Use only type-A and type-B questions • Minimize recall questions and maximize questions that test interpretive and problem-solving skills

36. CONCLUSIONS • Be certain that each question has a single and specific objective • Be certain the questions can be answered in the time allotted