How do we get the best specialists?

1. How do we get the best specialists? Professor Charlotte Ringsted, MD, MScHPE, PHD BMO Chair in Health Professions Education Research Director and Scientist, The Wilson Centre Department of Anesthesia University of Toronto and The University Health Network

2. Scientists • Promote creative synergies betweendiverse theoretical perspectives,and between theory and practice

3. Overview • Competency-, outcome-based education • Framework and Conception • Clinical training • Curriculum design

4. Frameworks of competence • CanMEDS roles • Medical Expert • Health Advocate • Communicator • Collaborator • Manager • Scholar • Professional • ACGME competencies • Medical knowledge • Patient care • Interpersonal and communication skills • System based practice • Practice based learning and improvement • Professionalism

5. EFPO project, 1992 Undergraduate education, Ontario, society’s needs, eight roles CanMEDs project, 2000 Postgraduate education, RCPSC, entire Canada, seven roles The seven roles Medical expert DK Communicator Health advocate Collaborator Manager Scholar Professional Whole person

6. Canada and Denmark – Red and white; Neighbours; Hans Island 3 persons per km2 125 persons per km2 No 3

7. North America • “Assessment rich area” • National exams • Flooded by psychometricians • Heavy focus on reliability of tests and exams • Strong tradition of cognitive psychology and behaviourism

8. MCQ SAQ ESSAY CEX OSCE ITER WBA ITER Portfolio B Hodges 2013 Specialist training as Sausage Factory Focus: Assessment and exams

9. “Assessment free area” Focus on training programsand evaluation of education No specialist exams andno psychometricians “To emphasize the educational purpose of training, comprehensive formative evaluation is suggested as alternative to specialist examinations.” Karle, Nystrup ME1995 Denmark

10. National Courses Seminars Reading Simulation Clinical training Logbooks Rotations Programs Trainees’ evaluation of quality of program Supervisor Appraisal meetings B Hodges/C Ringsted 2013 Specialist training as Sausage Factory Focus: Training and Evaluation

11. PGME 1991 NBH rules, guidelines Goals and objectives Specialist societies Speciality courses Clinical programmes Training posts CRE and supervisor Appraisal meetings (3) Trainees’ evaluation No exams PGME reform 2001 NBH rules, guidelines Goals and objectives CanMEDS framework Plus ’general’ courses* Clinical programmes Training posts CRE and supervisor Appraisal meetings (3) Trainees’ evaluation In-training assessment DK reform: C/OBE and ITA

12. WBA, In-training assessment, Anaesthesiology Ringsted et al. Med Teach 2003 Clinical skills assessments (12) Observation in vivo / vitro • Cusum scoring • Logbook on experience • Learning portfolio 1 st year training Longitudinal assessments Assessment based on practice data and written reflective assignments/reports • Communication skills (1) • Management/collaboration (2) • Academic competence (3)

13. Factors related to value of ITA Ringsted et al. ME 2004, Med Teach 2003, ASS 2003 • The link to practice • Help in structuring teaching, training and learning • Outcomes clear, monitoring progress, identify problems • Coupling of theory to practice • Used as licence to practice rather than end-of-training assessment • The effect on learning • Should include a challenge to the learner • ‘We all learn more’ • Assessors’ attitudes • Enthusiasm and rigour

14. ITA-programs and psychometrics A challenge to psychoanalyse this Schuwirth & v.d. Vleuten ME 2006 A plea for new psychometric methods

15. Future of Medical Education in CanadaToward a Competency-Based Approach

16. Time Long DM, Acad Med 2000 Competency-based residency training – Reducing time from 3 years to 1½ year

17. CanMEDS 2015 project • Hybrid of Time and Competence • In-training WPB assessment • EPAs and Milestones • Focus on “Intrinsic Roles” • Patient safety and inter-professional collaboration • Graded responsibility

18. A call for systems-based education • Outcome-based curricula • Milestones, graded responsibility • Systems/society orientation • Teamwork within and across professions and institutions

19. Current practice • Focus on individuals • Point-in-time sampling • Standardization Future • Focus on teams • Longitudinal WBA • Subjective, collective

20. Clinical trainingA matter of curriculum design

21. The concrete taskthe near team • Patient consultation, ward round, amb., operation, diagn. investigation • The system context andthe broader team • Primary, secundary, and tertiary sector and interplay withinand between these plus other stakeholders • The wider context andthe general perspective • The speciality/society, the profession, the region, the state, the society

22. Specialist training • Experience and exposure • Time and volume • Professional development • Deliberation

23. Oct 2013

24. Significant correlation between scores and complication rate

25. Experience – number of procedures and years of practice

26. Experience and exposure Curriculum design • Logbook of experience • Help in designing the composition of the training program • Ensure breadth and depth in experience and exposure

27. Experience is not enough Debilerate practice Guest et al, 2001, Coles 2002, Andersson, 2004 • Critical appraisal and reconstruction of practice - instruction, monitoring, feedback and discussions, and opportunities to improve performance repeatedly

28. Professional judgment • Not so much about finding the “right” answer but rather what is “best” in the situation. Coles 2002 • Ability to manage ambiguous problems, tolerate uncertainty and make decisions with limited information. Epstein and Hundert JAMA 2002

29. Routine experts vs. Adaptive experts Innovative dimension ’Adaptive experts’ Expertise Performance Efficiency dimension ’Routine experts’ Most of us Experience Schwartz et al. 2004 Ericsson, Guest et al., Choudhry et al. 2005

30. Self- regulation of learning and performance Zimmerman 2011 • Self-regulated learning and performance • Forethought • Adaptation • Evaluation • Characteristics • Motivation, proactive goal setting, strategic learning style, monitoring, adaptation, modelling learning environment, self-efficacy, assistance-seeking, - practice, practice, practice

31. Thoracic surgeons – why and how did they learn a new procedure? • Video Assisted Thoracoscopic Surgery • New technique introduced in late 90’s • Henrik Jessen Hansen & René Horsleben Petersen • Jensen et al. studied why and how experts learn a new procedure • Interviews in 2011 with ten VATS experts/local pioneers

32. Model – Experts learning VATS Self- directed learning Society- based coaching ”I didn’tlearn it – I taught it myself” Motivation Incentive Self realisation Social contagion Self-regulation of learning and performance Quality Of care Systems-regulation of learning and performance Self- efficacy Social competition Monitoring outcomes Jensen et al. 2012 Paper in progress

33. Self – and system regulationJensen et al 2012 Self – regulation • Build on prior knowledge and skills of anatomy, disease, techniques, equipment • Highly creative in developing technique (’towel cover’) • Step-by-step approach, Zone of ProximalDevelopment – time, elements, size and place • Monitor patient outcome System – regulation • Organiational doubts and concerns; personal recognition • Finances, available equipment • Time constraints (the ’list’), co-workers (the team) • Expectations of patients and co-specialties

34. The concrete taskthe near team • Patient consultation, ward round, amb., operation, diagn. investigation • The system context andthe broader team • Primary, secundary, and tertiary sector and interplay withinand between these plus other stakeholders • The wider context andthe general perspective • The speciality/society, the profession, the region, the state, the society

35. Person-Task-Context TASK Simple ... complicated Part … Whole CONTEXT Alone … Team Complexity Uncertainty Performance PERSON Novice ... Advanced Knowledge, skills, experience

36. Situated learning • Legitimate Peripheral Participation • Single task • Simple situation • Basic procedures • Working context • Multi-professional teams • New procedures Advanced Novice

37. Professional develomentDreyfus, Epstein & Hundert TASK Complex Atypical CONTEXT Complex systems Independent Supervisingothers Year 4-5 PERSON Proficient Expert TASK Complicated Typical TASK Simple Single CONTEXT Larger teams Distant supervision Year 2-3 CONTEXT Small teams Close supervision Year 1 PERSON Competent PERSON Adv. beginner Novice

38. The concrete taskthe near team • Patient consultation, ward round, amb., operation, diagn. investigation • The system context andthe broader team • Primary, secundary, and tertiary sector and interplay withinand between these plus other stakeholders • The wider context andthe general perspective • The speciality/society, the profession, the region, the state, the society

39. Integrating roles at 3 layers (EPAs)

40. Roles at 3 levels (Milestones)

41. Summary and conclusion

42. Competence? • Competency = specific capability • ”Reflects expectations that are expressible in measurable behaviour; uses criterion standards for judging; informs learners and others about expectations” Albanese ME 2008 • Competence= holistic overall capacity • ”The habitual and judicious use of communication, knowledge, technical skills, clinical reasoning, emotions, values, and reflection in daily practice for the benefit of the individual and the community being served” Epstein and Hundert JAMA 2002

43. Outcome goals Clearly defined standards of performance Checklists Competence cards Rating forms Quality of product Efficiency dimension Process goals Training as Preparation for Future Learning (PFL) Approach to the task Deliberation, reflection Adaptation to situation Critical re-construction Innovative dimension Future directions – the goals? Schwartz 2004,2005 Coles 2002 Harden 1999

44. Curriculum design Appropriate level of difficulty Plan and structure of the experience Instruction and feedback LEARNING Repetition and correction of errors Questions and dialogue Critical appraisal of practice EDUCATION

45. Curriculum design • Careful and thoughtful planning of experience • Grade the tasks and responsibilities, acknowledge the contextual issues of learning • Coach • Stimulate innovative dimension and meta-cognition – as preparation for future learning • Critical appraisal of practice – own and general • Using paper assignments and students as resource Med Educ 2011

46. Thank you for your attention ??????

47. Challenge in postgraduate education Work- based Postgraduate education • Learn from managing cases • Learn how to manage cases • Reflect in and on practice Does Does Can Can School- based Knows Knows Undergraduate education

48. Cultural dimensions • Individualism • ‘I’ vs. ‘We’ thinking • Power distance • Acceptance of hierarchies

49. Cultural dimensions • Masculinity/Femininity • Competition, ‘Be the best’, rewards for success • Uncertainty avoidance • Control of future, rules, principles, guidelines

50. AssessmentEPAs and milestones