faculty.ksu.sa

faculty.ksu.sa PowerPoint PPT Presentation


  • 205 Views
  • Updated On :
  • Presentation posted in: General

Related searches for faculty.ksu.sa

Download Presentation

faculty.ksu.sa

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


1. 1 RELIABILITY, VALIDITY, AND FEASIBILITY OF THE OBJECTIVE STRUCTURED CLINICAL EXAMINATION (OSCE) FOR ASSESSING THE CLINICAL SKILLS OF FINAL YEAR SURGICAL CLERKSHIP

2. 2

3. 3

4. 4 AIMS To present our experience with the 1st OSCE conducted for the final year surgical clerkship in May 2005. To discuss its reliability and validity. To discuss issues related to its feasibility.

5. 5

6. 6 METHODS…continue Stability of the OSCE was measured by Cronbach’s alpha, Internal Consistency by Carmine’s theta using the BMDP® statistical software. Credibility and comprehensiveness of the OSCE were judged by faculty and students. Concurrent validity comparing OSCE and MCQs results was measured by Pearson’s correlation.

7. 7 RESULTS OSCE Stations reliability (Day 1) Table 1: Description, stability, and internal consistency of day 1 - OSCE stations ________________________________________________________________________ Station Type Cronbach’s Alpha Carmine’s Theta (Stability) (Internal Consistency) ________________________________________________________________________ History Taking and Communication Skills 1. Biliary colic 0.6719* 0.8595** 2. Breast cancer 0.6703 0.7617 3. Lower urinary tract syndrome 0.6385 0.7288 4. Hematuria 0.6701 0.7956 5. Peripheral vascular occlusion 0.7001 0.7760 6. Polytrauma (standardized patient) 0.6289 0.6620 7. Bowel obstruction 0.6764 0.6453 8. Inguinal hernia 0.6287 0.8879 9. Diabetic foot 0.6597 0.6440 Total History 0.6608 0.7512 ________________________________________________________________________ Physical Examination 10. Subcutaneous lipoma 0.6709 0.8586 11. Undescended testis (Child) 0.6785 0.8234 12. Multinodular goiter 0.6879 0.8292 13. Peripheral vascular occlusion 0.6604 0.7925 14. Incisional hernia 0.6851 0.6830 15. Appendicular mass 0.7048 0.7663 16. Parotid tumor 0.6537 0.6620 17. Thyroglossal duct cyst 0.6666 0.7528 Total Physical Examination 0.6760 0.7710 ________________________________________________________________________ Problem Solving Skills 18. Intestinal obstruction X-rays (Neonate) 0.6654 0.6834 19. Child with nephrostomy tube 0.6287 0.7303 20. Subdural hematoma CT films 0.6938 0.9091 21. Cleft lip and palate picture 0.6959 0.8649 22. Post-operative bile leak case 0.6684 0.8687 23. Renal stones intravenous urogram 0.6718 0.7263 24. Diverticular disease barium enema 0.6429 0.8749 Total Problem Solving 0.6667 0.8082 _______________________________________________________________________ Overall OSCE Stations 0.68 0.79 * Overall OSCE stability coefficient with this station removed ** Station’s coefficient to the sum of all stations’ coefficient correlation

8. 8 Table 2: Description, stability, and internal consistency of day 2 - OSCE stations ________________________________________________________________________ Station Type Cronbach’s Alpha Carmine’s Theta (Stability) (Internal Consistency) ________________________________________________________________________ History Taking and Communication Skills 1. Renal colic 0.7646* 0.7464** 2. Bowel obstruction 0.7894 0.8325 3. Lower urinary tract syndrome 0.7814 0.8028 4. Inguinal hernia 0.7733 0.6979 5. Dysphagia (standardized patient) 0.7895 0.7263 Total History 0.7796 0.7610 ________________________________________________________________________ Physical Examination 6. Abdominal examination 0.7968 0.8081 7. Multinodular goiter 0.7845 0.8064 8. Incisional hernia 0.7735 0.7056 9. Breast cancer 0.7810 0.7824 10. Subcutaneous lipoma 0.7729 0.7747 11. Diabetic Foot 0.7813 0.5686 12. Polytrauma (standardized patient) 0.7823 0.6734 13. Thyroglossal duct cyst 0.7771 0.6426 Total Physical Examination 0.6760 0.7710 ________________________________________________________________________ Problem Solving Skills 14. Congenital diaphragmatic hernia X-rays (Neonate) 0.7768 0.6749 15. Colostomy case 0.7657 0.7246 16. Nephrostomy tube case 0.7729 0.8498 17. Brain tumor CT films 0.7730 0.8484 18. Undescended testis (child) 0.7689 0.8145 19. Postoperative bile leak (drain) 0.7719 0.7826 20. Urinary bladder tumor (cystogram) 0.7759 0.7419 21. ERCP film 0.7830 0.6551 22. Burn pictures 0.7795 0.8077 Total Problem Solving 0.7742 0.7666 ________________________________________________________________________ Overall OSCE Stations 0.79 0.81 * Overall OSCE stability coefficient with this station removed ** Station’s coefficient to the sum of all stations’ coefficient correlation (Item total correlation) RESULTS OSCE Stations reliability (Day 2)

9. 9 VALIDITY Comprehensiveness and credibility of the OSCE were judged as good by majority of staff and students. Concurrent validity was determined by Pearson’s correlation with the written one best answer MCQ’s (P=0.5). Predictive and construct validities were not addressed in this study.

10. 10 FEASIBILITY Very labor-intensive Inadequate management Some patients were exhausted, others were uncooperative, and some have left the exam. Lack of use of standardized patients. Lack of proper organization.

11. 11 DISCUSSION Since its introduction by Professor Harden and colleagues in 1979, OSCE’s have developed into different forms depending on the desired competencies to be tested. We tested combination of competencies involving history taking, communication skills, physical examination, and problem solving skills. OSCE preparation including staff commitment, blue print, stations development, etc.. is of paramount importance.

12. 12 DISCUSSION….continue Measuring reliability and validity are keys to successful examination. To achieve high reliabilities OSCE has to be longer than is often practical (Newbel; 2004). We achieved moderate reliabilities (i.e. r=0.68 and 0.79) for 2.7 hrs. OSCE. High reliabilities have been achieved by some authors with shorter OSCE’s (Cohen et al. 1990, Stillman et al. 1991, others).

13. 13 Validity is the hallmark of OSCE’s despite the arguments that encounters are short (Stillman, et al, Newbel, et al). The comprehensiveness of our OSCE was obvious as it covers wide range of clinical and problem solving skills. The credibility was also judged by the majority of staff and students to be adequate (i.e 5 minutes encounter). Reported concurrent validity are generally low (Shakun, et al, 1990; Merrick, et al; 2000). However, moderate and high correlations of OSCE’s with written tests, have been also reported (Gerrow, et al; 2003, Matsell, et al; 1991). Our OSCE reported moderate correlation (P.c.=0.5) with the written one best answer MCQ’s. DISCUSSION….continue

14. 14 Feasibility is the most critiqued aspect of OSCE’s as being costly and labour-intensive (Reznick et al. 1993; Barman 2005). We have experienced some difficulties on our first OSCE, which were mostly related to lack of proper organization, use of real patients, and exhaustive preparation. However, these issues were significantly improved with repetitions and experience. DISCUSSION….continue

15. 15 CONCLUSIONS Staff commitment and adequate preparation are keys for successful OSCE’s. Clinical competence encompasses different aspects that no single examination can cover with high degrees of reliability and validity. Well-designed OSCE can asses most of these aspects with variable degrees of reliability and validity. Feasibility of OSCE’s can be further improved with repetitions, experience, and use of standardized patients.

16. 16

  • Login