EM Milestones and Resource Development

1. EM Milestones andResource Development CORD 2013

2. EM Milestones andResource Development • Putting Competency Based Education into Practice – Doug • Intro to the EM Milestones Wiki – Joint Milestones Task Force • When you get home, get on the Wiki and see what works for you • Not a finished product but a starting point

3. ACGME Milestones; putting CBME into context Douglas Char, MD FACEP FAAEM CORD Academic Assembly March 2013

4. A Brave new world… • We are at a tipping point in competency-based medical education (CBME) — and it’s only taken 40 years since the competency conversation first appeared in the medical literature! • In case you were not aboard the CBME train as it left the station, this concept is an integrated framework for education, in which specific behavioral outcomes (competencies) drive both medical school curricula and individual advancement, rather than the current driving forces of time (four years of medical school) and process (clerkships of specific length). • Carol Aschenbrener – Chief Medical Officer, AAMC http://wingofzock.org/2012/09/25/competency-based-medical-education-the-time-is-now/

5. Competency Based Medical Education • Traditional medical education presumes that all students are ready to graduate once they have completed a set number of years of study and passed the required assessments, • There is a growing interest in tailoring the length as well as the content of medical education to individual aptitudes. • “People learn in different ways and at different speeds,” • “As early as 1932, reports emerged saying that it is not enough to stuff students’ heads with information • Stakeholder no longer accepting residents as independent actors, they expect physicians to function as leaders and participants in team-oriented care. https://www.aamc.org/newsroom/reporter/april11/184286/competency-based_medical_education.html

6. Competency-Based Medical Education • It is a curricular concept designed to provide the skills physicians need, rather than solely a large, prefabricated collection of knowledge. • A medical school or residency program using competency-based medical education defines a set of skills or competencies based on societal and patient needs, such as medical knowledge, patient care, or communications approaches, and then develops ways to teach that content across a range of courses and settings. The competency-based approach still includes scientific knowledge, but in the broader context of a physician’s tasks as a healer. https://www.aamc.org/newsroom/reporter/april11/184286/competency-based_medical_education.html

7. CBME implementation isn't easy • We are wrestling with it just like everyone else. The challenge is not so much accepting the concept, which we think is great, but figuring out how to make it work. Where do we teach? How do we evaluate performance? How do we remediate students who have not met requirements?”Thomas Pellegrino - EVMS • How to define competencies, and how to assess performance are perhaps the two most significant concerns about competency-based medical education. Peter Katsufrakis – NBME • “we’ve been wrestling with this question for decades,”M. Brownell Anderson - AAMC

8. CBME Challenges • Implementing competency-based training in postgraduate medical education poses many challenges. Making this transition requires change at virtually all levels of postgraduate training. • Key components of this change include; • Development of valid and reliable assessment tools such as work-based assessment using direct observation, • Frequent formative feedback • Learner self-directed assessment; • Active involvement of the learner in the educational process; • Intensive faculty development that addresses curricular design and the assessment of competency Iobst. Teach Med 2010; 32: 651–656

10. Bloom’s Taxonomy 1956 Anderson’s revision 2000 Cognitive (Knowledge) Affective (Attitude) Psychomotor (Skills)

11. Milestone Outcomes • Final milestones will provide meaningful data on the performance that graduates must achieve before entering unsupervised practice (graduate) • Initial milestones for entering residents will add a performance- based vocabulary to conversations with medical schools about graduates’ preparedness for supervised practice (residency)

12. Miller’s Pyramid of Clinical Competencies Norcini BMJ 2003:326(5):753-755 Miller GE. The assessment of clinical skills/competence/performance. Acad Med 1990:S63-7.

13. “Work-based Assessment”

14. Assessment drives learning • Many people argue that this statement is incorrect and that the curriculum is the key in any clinical course. • In reality, students feel overloaded by work and respond by studying only for the parts of the course that are assessed. • To promote learning, assessment should be educational and formative—students should learn from tests and receive feedback on which to build their knowledge and skills • Pragmatically, assessment is the most appropriate engine on which to harness the curriculum. V Wass. Lancet 2001; 357: 945–49

15. Assessing “Does” • Trained Observers • Common understanding of expectations • Sensitive “eye” to key elements • Consistent evaluation of a given level of performance • Minimum number of quality observations • Assessment based on 7-9 observations felt to be valid and reliable • Interpreter/Synthesizer Experts • Clinical Competency Committee

16. Narratives vs Numbers • Numbers produce range restriction • Narratives are easily understood by faculty and produce data without range restriction • Natural to how we teach and provide feedback • Goalis to create verbal pictures • 4 cm laceration right arm vs

18. Monkey see, monkey do: a critique of the competency model in GME • Danger here is that rather than engaging a total practicum to which other forms of learning discourse bring their insights, a limited professional education is based upon an inappropriate epistemology of competency • Tendency to limit the reflection, intuition, experience and higher order competence necessary for expert, holistic or well developed practice • Martin Talbot, Med Educ 2004; 38: 587–592

19. What does this all mean? • If your are feeling overwhelmed and confused by all this new jargon – you are not alone • Your faculty are looking to you for answers! • Nobody has all the answers so stop waiting for the Holy Grail? • Better to join the legion of PDs working to define it • There is no way to sort out the milestones without getting “dirty” – expect to make mistakes • Assessment is suppose to drive curriculum (this is a game changer) • Resistance if futile, give in and drink the kool aid • Reduce your stress, it’s going to happen!

20. Intro to EM Milestones Wiki Kevin Biese, MD, MAT

21. EM Milestones Wiki

24. JMTF – Work so far • Intro to the Wiki – Christina • Care Based Milestones – Moshe • Systems Based Milestones – Mary Jo • Procedural Milestones – Jenna • Milestones Workbook - Rodney

25. Intro to Wiki Resources Christina Shenvi, MD, PhD

26. WiiFM

32. But ultimately, check back to…

33. Accessing the wiki • Emmilestones.pbworks.org • You do not need a login • We hope this will be a helpful resource

34. JMTF – Care-Based Milestones Moshe Weizberg, MD, FACEP

35. THANK YOU • Committee • Nestor Rodriguez • Jason Seamon

36. EVALUATIONS • How do you want to divide your milestones • What do you want your evaluation questions to look like • How do you label each level • Where do you want your comment boxes

37. HOW DO YOU WANT TO DIVIDE YOUR MILESTONES • All milestones on every evaluation • Divide among various rotations • Divide among faculty • Divide by day of the month • Hit all core competencies • CCCs evaluate all milestones based on evals

38. PRO’S AND CON’S

39. WHAT DO YOU WANT YOUR EVALS TO LOOK LIKE • Yes/No/N/A • Mix up elements from various milestones • Mirror the milestone pages

44. PRO’S AND CON’S

45. LESSONS LEARNED • Many faculty gave all residents level 5 • Faculty bias based on label of each level (PGY) • Faculty education • Resident education

47. JMTF - Systems-Based Milestones Mary Jo Wagner, MD