1 / 10

Validation of the Manitoba Practice Assessment Program CME Congress: June 1, 2012

Validation of the Manitoba Practice Assessment Program CME Congress: June 1, 2012. MPAP Program Team. Jose Francois MD, Associate Dean Marilyn Singer MD, Director Jillian Horton MD, Assistant Director Brenda Stutsky PhD, Program Advisor Debbie Lemkey, Program Assistant

jemima
Download Presentation

Validation of the Manitoba Practice Assessment Program CME Congress: June 1, 2012

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Validation of the Manitoba Practice Assessment Program CME Congress: June 1, 2012

  2. MPAP Program Team Jose Francois MD, Associate Dean Marilyn Singer MD, Director Jillian Horton MD, Assistant Director Brenda Stutsky PhD, Program Advisor Debbie Lemkey, Program Assistant Robert Renaud PhD, Dept. Ed. Admin., Foundations & Psychology Conflict of Interest: Nothing to disclose: Pilot study self-funded

  3. Background/Purpose • MPAP developed based on need from the College of Physicians and Surgeons of Manitoba (CPSM) • Involves the assessment of practicing physicians on the CPSM conditional register • The purpose of the pilot study was to test selected MPAP processes and examine the preliminary reliability and validity of the practice assessment tools

  4. Method • Mixed methods design • 6 Stages: • Training of assessors • Selection of family physicians and 1st year family medicine residents • Self-assessment process and 360 degree surveys • Onsite assessment • Collation of data and feedback to the physician candidates • Final debriefing session

  5. Results: Assessor Online Training (N = 8) Length of time to complete the self-directed online learning activities: • Mean 7.13 hrs. (Range of 4-9 hrs.) Enjoyed online format: • Yes 75% • It was okay 25% • No 0% After completing the online activities, assessors believed they had a good understanding of the various assessment strategies/processes: • Yes 100% • No 0%

  6. Results: Assessor Face-to-Face Training (N = 12) I am ready to begin to assess physicians: • Yes, and I do not require additional training: 50% • Yes, but I will need to shadow/buddy an experienced assessor or participate in a pilot/mock assessment: 50% • No, I will need additional guidance and training: 0% I recommend that the Assessor Workshop be modified: • Yes: 0% • No: 100%

  7. Results: Comparison of Mean Scores between Family Physicians and Residents & Correlation to PAR

  8. Results: Debriefing Session • Overall, what did you like about the MPAP Process? • Interprofessional collaboration, training process, usability and variety of tools • Overall, what could improve? • Office scheduling, patient selection, access to EMR/charts, use of EMR for data, space, length of time between training and onsite assessment • How did you feel during the assessment process? • Assessors: uncomfortable, anxious → confident over time • Candidates: nervous, pre-occupied with other roles, “like a resident again” • If you were to provide guidance to physicians undergoing the assessment process, what would you tell them? • Relax, process allows your strengths to shine, encouraging comments - want them to succeed

  9. Conclusions • Blended learning was effective • Provided valuable training experience for assessors in a low-stakes environment • Limited generalizability given small sample size • Validity and reliability and of tools is promising • Continue to work on validity and reliability • Need only minor changes to the process Copy of Presentation (Follow MPAP links):www.umanitoba.ca/cpd

More Related