1 / 33

Rafael Blanco

Evaluation of learning. Rafael Blanco. Lewisham University Hospital London. How do we grade our trainees?. Beginners. Advance. Intermediate. 1. scope and duration of training; 2. institutional organization; 3. program director and faculty; 4. facilities and resources;

darcie
Download Presentation

Rafael Blanco

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Evaluation of learning Rafael Blanco Lewisham University Hospital London

  2. How do we grade our trainees? Beginners Advance Intermediate

  3. 1. scope and duration of training; 2. institutional organization; 3. program director and faculty; 4. facilities and resources; 5. educational program; 6. scholarly activity; 7. consultant skills; 8. evaluation; 9. board certification. Current training is not sufficient to allow proficiency by the end of residency for a given block. Chelly et al. Demonstrated a lack of confidence in PNBs by residents.

  4. To reach proficiency…when? 90%? • Rosenblatt et al. • Only 50% of the residents were able to perform the interscalene block without supervision after seven to nine blocks • with more than 15 attempts, 87.5% of residents were autonomously successful

  5. The relation between theoretical and practical knowledge “explicit formal knowledge” “practical aspects” Cognitive elements Psychomotor elements Affective elements personal skills Interpersonal skills Communication

  6. Learning Manual Skills in Anaesthesiology: Is There a Recommended Number of Cases or Anaesthetic Procedures? Anaesthesia & Analgesia Volume 86(3), March 1998, pp 635-639Conrad, Christoph; Schupfer, Guido; Wietlisbach, Markus;Gerber, Helmut

  7. Tracking the early acquisition of skills by traineesAnaesthesia (C) 2001 Association of Anaesthetists of Great Britain & IrelandVolume 56(10), October 2001, pp 995-998Harrison, M. J. Period 1 year 100 medical students and 19 trainees No previous experience intubating

  8. Tracking the early acquisition of skills by trainees • Advantages: No requirement to perform an x number of procedures No requirement for a particular level of success • Disadvantages: Trainees collected the data 4 Different institutions Students were attached in 6 batched over the year

  9. What are the markers of expertise? • Flexibility • Confidence

  10. How do we assess trainees? Continuous assessment Logbooks Cusum

  11. CUSUM

  12. If success move along “x” axis If failure move along “y” axis 27/3/07 20/3/07 2/3/07 29/3/07 20/3/07 6/3/07 20/3/07 29/3/07 12/3/07 16/3/07 22/3/07 17/3/07 22/3/07 18/3/07 22/3/07 22/3/07 18/3/07 22/3/07 18/3/07 27/3/07 18/3/07 27/3/07 18/3/07 27/3/07 18/3/07

  13. Cusum is a target value set for the level of performance when the block is successful 1= success 0= failure Three successes, follow by a failure and 2 more successes

  14. The Construction of Learning Curves for Basic Skills in Anesthetic Procedures Anesthesia & Analgesia Economics,Education and Health Systems Research Volume 95(2), August 2002, pp 411-416 de Oliveira Filho, Getulio Rodrigues MD There was a wide inter individual variability in the number of procedures before attaining acceptable failure rates. This suggests that performance should be followed on at individual bases

  15. The Construction of Learning Curves for Basic Skills in Anaesthetic Procedures (Spinal) Succesful identification SAE first attempt in 569 procedures(82.70%) A second interspace used in 119 procedures (17.29%)

  16. The Construction of Learning Curves for Basic Skills in Anaesthetic Procedures (Epidural) Successful identification epidural space at interspace first chosen occurred in 275 patients (79.94%) A second interspace was used in the remaining 69 cases ( 20.05%)

  17. Cusum analysis is a useful tool to assess resident proficiency at insertion of labour epidurals Canadian Journal of Anaesthesia 2003 / 50: 7 / pp 694-698 Viren N. Naik ,* Isabella Devito, Stephen H. Halpern

  18. But… No data

  19. Define success CUSUM Axillary block with nerve stimulator

  20. Designing a CUSUM charting The statistical way

  21. Upward CUSUM At the nth procedure Cn= max (0, Cn-1 +Xn- k ) At 0 procedures the value of CUSUM is 0 Cn>h implies unacceptable performance

  22. Values Cn= max (0, Cn-1 +Xn- k ) • Xn is 0 or 1 for a binary procedure 1= failure 0= success • K is the reference value • H is the decision interval

  23. IC and OC h is determined by specifying the in-control (IC) average run of length (ARL)of a CUSUM chart Type I error (alpha) IC-ARL is the average number of consecutive procedures required for a CUSUM chart to cross a decision interval despite performing at acceptable level 1-Type II error (beta) or 1-false negative error OC-ARL is the average number of procedures performed before the CUSUM chart signals when an individual is performing at unacceptable level

  24. Let’s revise:How to calculate the values? • CUSUM starts at 0 • Define acceptable and unacceptable failure rates • Desire magnitude of alfa and beta errors ( Type I and Type 2) • Calculate ho and h1

  25. Advantages of CUSUM charts • It works for individual and group observations • Design to detect small shifts in performance • We specify the target value • It makes explicit the trade-off between sensitivity and false alarm • It is objective and with visual appeal

  26. In summary • A structured regional anesthesia rotation • A complete re-evaluation of the anesthetic training( compulsory R.A. blocks) • Guidelines • A training curriculum • A group of proper trainers in R.A. • Numbers • Determination

  27. Thanks

More Related