Applications of Performance Assessment

1. Applications of Performance Assessment Dr Alison Reid Medical Director NSW Medical Board 10 December 2007

2. Professional Regulation & Governance • Should focus on health professionals’ performance & fitness to practiserather than just their qualifications & experience.

4. Public Expectations • Health professionals are assessed regularly, like pilots. • Assessments are comprehensive and reliable. • Results are acted upon appropriately.

5. The Irony of Assessment • Pre-tertiary education • University • Postgraduate training • Employed practice • Independent practice } Assessments galore Hardly anything NOTHING!

6. The language of performance assessment

7. Competence • The possession of knowledge and skill. • i.e. What the health professional knows, and knows how to do.

8. Performance • The application of knowledge and skill. • i.e. What the health professional actually does for real patients in a real setting.

9. Miller’s framework Performance Competence

10. A cricket analogy

11. Competence v Performance • It is certain that an incompetent health professional will perform poorly. • However, a competent health professional cannot be guaranteed to perform well.

12. Competence v Performance • Experience • System factors • Distracters

13. Professional performance 2.5% ≈ 700 doctors in NSW low high - 2 sd +2 sd performance standard

14. Myth #1 • Attendance at Continuing Professional Development sessions ensures competence.

15. Myth #2 • Health professionals: • know what education they need, • know when they need it, • immediately acquire it when they need it, • always benefit from it.

16. Approaches to assessing performance • Routine: • for all health professionals in relation to registration / employment • (aimed at improving quality, NOT finding bad apples.)

17. Approaches to assessing performance contd. • Opportunistic: • expressions of concern • complaints • in response to unsatisfactory routine assessment

18. Choosing assessment tools • Demonstrates performance • Do-able • Defensible

19. Assessment Tools • Assessment validity increases: • with more than one assessor • with multiple assessment tools • in the health professional’s natural habitat

20. Assessment Domains • Clinical practice • Communication • Collaboration • Management practices • Personal development • Education / training • Professional behaviour • Personal health / wellbeing • etc

21. Assessment tool kit • Performance appraisal interview • CPD / portfolios • Tests of knowledge, eg MCQs • OSCE / Case based oral / CSR • Record review • Outcome audits • Interviews with colleagues / patients • Observed practice (Mini CEX)

22. Miller’s framework Performance Competence

23. Clinical interview / Case based oral / OSCE

24. Performance Competence Case-based oral / OSCE

25. Interview with colleagues/patients

26. Performance Competence Interview with colleagues/patients

27. Observed practice

28. Performance Competence Observed practice

29. Assessment tools • No single tool fully addresses performance in every domain. • Aim for overlap along the competence / performance spectrum. • The more tools the better.

30. NSW Medical BoardPerformance Assessment Program • In place since 2000 • Opportunistic • ~ 30 full PAs / year • Early intervention / remediation • Peer assessment • Environmental validity

31. Record review Simulation Observed practice Clinical interview Knowledge knows knows how shows how does Competence Performance

32. Supplementary assessment • Focused assessment • Medical assessment • Neuropsychometric testing • Simulator

33. Common assessment triggers • Multiple complaints • History with Board • Multi-faceted concerns • Isolation • Age

35. Discussion Topics • Can a well governed healthcare system afford NOT to do Performance Assessment? • Where should the principles of Performance Assessment be applied? • What assessment tools would be applicable? • What barriers would need to be overcome? • How can students be prepared for career-long assessment?