Performance characteristic of computer based Extended matching items and One Best Answer: A pilot st...
Download
1 / 13

Lunelle L Pienaar & Gregory Doyle Education Development Unit, - PowerPoint PPT Presentation


  • 95 Views
  • Uploaded on

Performance characteristic of computer based Extended matching items and One Best Answer: A pilot study . Lunelle L Pienaar & Gregory Doyle Education Development Unit, Faculty of Health Sciences, University of Cape Town. Background .

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about ' Lunelle L Pienaar & Gregory Doyle Education Development Unit, ' - terri


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript

Performance characteristic of computer based Extended matching items and One Best Answer: A pilot study

Lunelle L Pienaar & Gregory Doyle

Education Development Unit,

Faculty of Health Sciences, University of Cape Town


Background
Background matching items and One Best Answer: A pilot study

  • Multiple choice format remain one method of assessment that is most commonly used to test theoretical knowledge in medical education (Palmer EJ and Devitt PG. 2007, Mavis et al, 2001).

  • Increasingly computer based assessment is used more within the FHS UCT-student numbers, drive increase IT use


Mcq t ypes
MCQ matching items and One Best Answer: A pilot study Types

  • R-type (OBA)

    • Traditionally used more often and is written with question in the form of a clinical case scenario with 4 or 5 option, questions are thought to test recall of knowledge.

  • A-type (EMI)

    • The questions are written with a theme and questions are constructed around the focus theme (Beullens et al, 2002, Palmer and Devitt, 2007).


  • Why emi
    Why EMI matching items and One Best Answer: A pilot study

    • EMIs is considered a better format to test problem solving and integration of knowledge( Swanson et al, 2008, Swanson et al, 2006).

    • EMI has demonstrated better discrimination between students as well as more difficulty compared to OBA (McCoubrie2004).


    Study question
    study question matching items and One Best Answer: A pilot study

    • What is the performance of in house test administered to medical students during one of their clinical rotations in 5th year.

      • What is the test item difficulty and discrimination of EMIs and OBA used in summative assessment.

      • Is there any differences between item difficulty and discrimination between the EMI and OBA.


    Methods
    Methods matching items and One Best Answer: A pilot study

    • The questions were administered to all 5th year students at the end of their clinical block.

    • We evaluated OBA items and extended matching items that were developed by the 5th year course lecturer and evaluated as appropriate for testing by the course convener(not the researcher).

    • The item difficulty and discrimination for the EMIs and OBA was determined


    Results
    Results matching items and One Best Answer: A pilot study

    • Results were from the end of block test with 5 groups of students. The same questions were administered to students all 5 groups of students.

    • The 193 students answered a computer based test containing five EMI sets with 34 items and 16 (OBA)-Type MCQs.

    • The OBA -had 4 options with each of the 5 EMIs sets had 9 options with 9 questions.


    Distribution of test item difficulty
    Distribution of Test item Difficulty matching items and One Best Answer: A pilot study


    Distribution of test item discrimination
    Distribution of test item discrimination matching items and One Best Answer: A pilot study


    Mean test item difficulty and discrimination
    Mean Test Item Difficulty and Discrimination matching items and One Best Answer: A pilot study


    Conclusion
    Conclusion matching items and One Best Answer: A pilot study

    • The study did not show any statistical significance between the 2 types of MCQs –

      • small sample of test items.

      • Possible reason for this may also be that writing OBA format were used more often in the past.

    • Both EMI and OBA test items wide range of item difficulties

    • The results raises 2 issues

      • when questions are reused to select the best performing items.

      • evaluation of test post administration is important to determine how to improve quality of the assessment items.


    References
    References matching items and One Best Answer: A pilot study

    • Beullens J, Van Damme B, Jaspaert H, Janssen PJ. 2002. Are extended-matching multiple-choice items appropriate for a final test in medical education? Medical Teacher 24 (4):390-5.

    • Mavis BE, Cole BL, Hoppe RB. 2001 A survey of student assessment in U.S. medical schools: the balance of breadth versus fidelity. Teaching and Learning in Medicine, 13 (2), 74-79.

    • McCourbrie P. 2004. improving the fairness of multiple- choice questions: a literature review. Medical Teacher, vol 26. 8 (709-712)

    • Palmer EJ and Devitt PG. 2007. Assessment of higher order cognitive skills in undergraduate education: modified essay or multiple choice questions? Research paper. BMC Med Educ; 7: 49.

    • Swanson, D.B., Holtzman, K.Z., Allbee, K. 2008. Measurement characteristics of content-parallel single-best-answer and extended-matching questions in relation to number and source of options. Journal of the Association of American Colleges, 83(10 Suppl):S21-4

    • Swanson, D.B., Holtzman, K.Z., Allbee, K., Clauser, B.E. 2006. Psychometric characteristics and response times for content-parallel extended-matching and one-best-answer items in relation to number of options. Journal of the Association of American Colleges. Abstract 81(10 Suppl):S52-5


    http://edusum.edublogs.org/category/thoughts-on-assessment/ matching items and One Best Answer: A pilot study


    ad