Accreditation Review Meeting: University of Toronto

1. University of TorontoPre-Survey Meeting withDepartment / Clinical Chairs Date: September 21, 2012 Time: 10:45 a.m. to 12:15 p.m. Room: Queen’s Park BallroomPark Hyatt Hotel

2. Objectives of the Meeting To review the: • Accreditation Process • New Categories of Accreditation • Standards of Accreditation • Pilot Accreditation Process • Role of the: • Program director • Department / division chairs • Residents • Program administrators

3. Accreditation • Is a process to: • Improve the quality of postgraduate medical education • Provide a means of objective assessment of residency programs for the purpose of Royal College accreditation • Assist program directors in reviewing conduct of their program • Based on Standards

4. The Accreditation Process • Based on General and Specific Standards • Based on Competency Framework • On-site regular surveys • Peer-review • Input from specialists • Categories of Accreditation

5. Pilot Accreditation Process The University of Toronto is one of three universities participating in a pilot accreditation process! • Details for the pilot process will be discussed later in presentation

6. Six Year Survey Cycle 1 6 Monitoring 5 2 3 4 Internal Reviews

7. Process for Pre-Survey Questionnaires University Questionnaires Specialty Committee Questionnaires Royal College Comments Questionnaires & Comments Comments Program Director Surveyor

8. Role of the Specialty Committee • Prescribe requirements for specialty education • Program standards • Objectives of training • Specialty training requirements • Examination processes • FITER • Evaluates program resources, structure and content for each accreditation review • Recommends a category of accreditation to the Accreditation Committee

9. Composition of a Specialty Committee • Voting Members (chair + 5) • Canada-wide representation • Ex-Officio Members • Chairs of exam boards • National Specialty Society (NSS) • Corresponding Members • ALL program directors

10. The Survey Team • Chair - Dr. Kamal Rungta • Responsible for general conduct of survey • Deputy chair – Dr. AnuragSaxena • Visits teaching sites / hospitals • Surveyors • Resident representatives – CAIR • Regulatory authorities representative – FMRAC • Teaching hospital representative– ACAHO

11. Information Given to Surveyors • Revised questionnaire (PSQ) and appendices • Completed by program • Program-specific Standards (OTR/STR/SSA) • Report of last regular survey • Reports of mandated Royal College reviews since last regular survey, if applicable • Specialty Committee comments • Also sent to PGD / PD prior to visit • Exam results for last six years

12. The Survey Schedule • Document review (30 min) • Residency Program Committee minutes • Resident assessment files • Meetings with: • Program director (75 min) • Department chair (30 min) • Residents (per group of 20 - 60 min) • Teaching staff (60 min) • Residency Program Committee (60 min)

13. Meeting Overview • Program director • Overall view of program • Address each Standard • Time & support • Department chair • Support for program • Concerns regarding program • Resources available to program • Research environment • Teaching faculty • Involvement with residents • Communication with program director

14. Meeting with ALL Residents • Topics to discuss with residents • Objectives • Educational experiences • Service /education balance • Increasing professional responsibility • Academic program / protected time • Supervision • Assessments of resident performance • Evaluation of program / assessment of faculty • Career counseling • Educational environment • Safety

15. The Recommendation • Survey team discussion • Evening following review • Feedback to program director • Exit meeting with surveyor • Morning after review • 07:30 – 07:45 at the Park Hyatt Hotel • Survey team recommendation • Category of accreditation • Strengths & challenges

16. Categories of Accreditation New terminology • Revised and approved by the Royal College, CFPC and CMQ in June 2012.

17. Categories of Accreditation Accredited program • Follow-up: • Next regular survey • Progress report within 12-18 months (Accreditation Committee) • Internal review within 24 months • External review within 24 months Accredited program on notice of intent to withdraw accreditation • Follow-up: • External review conducted within 24 months

18. Categories of Accreditation Definitions • Accredited program with follow-up at next regular survey • Program demonstrates acceptable compliance with standards.

19. Categories of Accreditation Definitions • Accredited program with follow-up byCollege-mandated internal review • Major issues identified in more than one Standard • Internal review of program required and conducted by University • Internal review due within 24 months

20. Categories of Accreditation Definitions • Accredited program with follow-up by external review • Major issues identified in more than one Standard AND concerns - • are specialty-specific and best evaluated by a reviewer from the discipline, OR • have been persistent, OR • are strongly influenced by non-educational issues and can best be evaluated by a reviewer from outside the University • External review conducted within 24 months • College appoints a 2-3 member review team • Same format as regular survey

21. Categories of Accreditation Definitions • Accredited program onnotice of intent to withdraw accreditation • Major and/or continuing non-compliance with one or more Standards which calls into question the educational environment and/or integrity of the program • External review conducted by 3 people (2 specialists + 1 resident) within 24 months • At the time of the review, the program will be required to show why accreditation should not be withdrawn.

22. After the Survey survey team Reports specialty committee royal college university Report & Response Reports Responses Recommendation Reports & Responses accreditation committee

23. The Accreditation Committee • Chair + 16 members • Ex-officio voting members (6) • Collège des médecins du Québec (1) • Medical Schools (2) • Resident Associations (2) • Regulatory Authorities (1) • Observers (9) • Collège des médecins du Québec (1) • Resident Associations (2) • College of Family Physicians of Canada (1) • Regulatory Authorities (1) • Teaching Hospitals (1) • Resident Matching Service (1) • Accreditation Council for Graduate Medical Education (2)

24. Information Available to the Accreditation Committee • All pre-survey documentation available to surveyor • Survey report • Program response • Specialty Committee recommendation • History of the program

25. The Accreditation Committee • Decisions • Accreditation Committee meeting • October 2013 • Dean & postgraduate dean attend • Sent to • University • Specialty Committee • Appeal process is available

26. General Standards of Accreditation “A” Standards • Apply to University, specifically the PGME office “B” Standards • Apply to EACH residency program • Updated January 2011

27. “A” Standards Standards for University & Education Sites A1 University Structure A2 Sites for Postgraduate Medical Education A3 Liaison between University and Participating Institutions

28. “B” Standards Standards for EACH residency program B1 Administrative Structure B2 Goals & Objectives B3 Structure and Organization of the Program B4 Resources B5 Clinical, Academic & Scholarly Content of the Program B6 Assessment of Resident Performance

29. B1 – Administrative Structure There must be an appropriate administrative structure for each residency program. • Qualifications of, and support for program director • Membership = resident(s) + faculty • Responsibilities • Operation of program • Program & resident evaluations • Appeal process • Selection of candidates • Process for teaching & evaluating competencies • Research

30. B1 – Administrative Structure“Pitfalls” • Program director autocratic • Residency Program Committee dysfunctional • Unclear Terms of Reference (membership, tasks and responsibilities) • Agenda and minutes poorly structured • Poor attendance • Department chair unduly influential • RPC is conducted as part of a Dept/Div meeting • No resident voice

31. B2 – Goals and Objectives There must be a clearly worded statement outlining the Goals & Objectives of the residency program. • Rotation-specific • Address all CanMEDS Roles • Functional / used in: • Planning • Resident evaluation • Distributed to residents & faculty

32. B2 – Goals & Objectives“Pitfalls” • Missing CanMEDS roles in overall structure • Okay to have rotations in which all CanMEDS roles may not apply (research, certain electives) • Goals and objectives not used by faculty/residents • Goals and objectives dysfunctional – does not inform evaluation • Goals and objectives not reviewed regularly

33. B3 – Structure & Organization There must be an organized program of rotations and other educational experiences to cover the educational requirements of the specialty. • Increasing professional responsibility • Senior residency • Service responsibilities, service / education balance • Resident supervision • Clearly defined role of each site / rotation • Educational environment

34. B3 – Structure & Organization “Pitfalls” • Graded responsibility absent • Service/education imbalance • Service provision by residents should have a defined educational component including evaluation • Educational environment poor

35. B4 - Resources There must be sufficient resources – Specialty-specific components as identified by the Specialty Committee. • Number of teaching faculty • Number of variety of patients and operative procedures • Technical resources • Resident complement • Ambulatory/ emergency /community resources/experiences

36. B4 – Resources “Pitfalls” • Insufficient faculty for teaching/ supervision • Insufficient clinical/technical resources • Infrastructure inadequate

37. B5 – Clinical, Academic & Scholarly Content of Program The clinical, academic and scholarly content of the program must prepare residents to fulfill all Roles of the specialist. • Educational program • Curriculum / structure • Content specific areas defined by Specialty Committee • CanMEDS Roles • Teaching of the individual competencies • Resident / faculty participation in conferences

38. B5 – Clinical, Academic & Scholarly Content of Program “Pitfalls” • Organized academic curriculum lacking or entirely resident driven • Poor attendance by residents and faculty • Teaching of essential CanMEDS roles missing • Role modelling is the only teaching modality

39. B6 – Assessment of Resident Performance There must be mechanisms in place to ensure the systematic collection and interpretation of evaluation data on each resident. • Assessment must be - • Regular, timely, formal • Face-to-face • Based on objectives • Include multiple evaluation techniques

40. B6 – Evaluation of Resident Performance “Pitfalls” • Mechanism to monitor, promote, remediate residents lacking • Formative feedback not provided and/or documented • Evaluations not timely (particularly when serious concerns identified), not face to face • Summative evaluation (ITER) inconsistent with formative feedback, unclearly documents concerns/ challenges

41. Learning Environment What are the processes in place to resolve problems / issues? Appropriate faculty / resident interaction and communication must take place in an open and collegial atmosphere so that a free discussion of the strengths and challenges of the program can occur without hindrance.

42. Pilot Accreditation Process Scheduled from April 7 to 12, 2013 • PGME and teaching sites – A Standards • Residency programs – B Standards

43. Pilot Accreditation Process ALL residency programs • Complete PSQ • Undergo a review, either by • On-site survey, or • PSQ/documentation review, and input from various stakeholders Process varies depending on group • Mandated for on-site survey • Eligible for exemption from on-site survey • Selected for on-site survey

44. Programs Mandated for On-site Survey Scheduled for On-site Reviewin April 2013 Criteria • Core specialties • General Surgery, Internal Medicine, Obstetrics & Gynecology Pediatrics, Psychiatry • Palliative Medicine • Conjoint Royal College/CFPC program • Program Status • Not on full approval since last regular survey • New program which has not had a mandated internal review conducted

45. Process for Programs Mandated for On-site Review Process remains the same • PSQ Review • Specialty Committee • On-site survey by surveyor • Survey team recommendation • Survey report • Specialty Committee • Final decision by Accreditation Committee • Meeting in October 2013 • Dean & postgraduate dean attend

46. Programs Eligible forExemption from On-site Review Criteria • Program on full approval since last regular on-site survey

47. Process for Programs Eligible for Exemption • PSQ and documentation review • Accreditation Committee reviewer • Specialty Committee • Recommendations to exempt • Accreditation Committee reviewer • Specialty Committee • Postgraduate dean • Resident organization (CAIR) • Steering Committee (AC) Decision • Review of recommendations • Exempted: on-site survey not required • Not exempted: program scheduled for on-site survey in April • Selected program (random) • University notified in January 2013

48. Contact Information at theRoyal College accred@royalcollege.ca 613-730-6202 Office of Education Margaret Kennedy Assistant Director Accreditation & Liaison Educational Standards Unit Lise Dupéré Manager Sylvie Lavoie Survey Coordinator