Workplace-based Assessment (WBA) – Beyond the numbers

1. Workplace-based Assessment (WBA) – Beyond the numbers James Kwan Associate Professor, Emergency Medicine and Medical Education

2. Acknowledgements • A/Prof Lukas Kairitis • Dr Karuna Keat • Professor Jenny Reath • Professor Wendy Hu • Members Year 3/5 Committee

3. Outline • Competence vs. Performance • Overview WBA at UWS • Assessor judgements on performance • Delivering Feedback

4. Definitions • Competence • “Ability to do something successfully” (Oxford Dictionaries, 2012) • Competence in medicine • Ability to perform a specific clinical task • Requires integration of competencies • Measurable in terms of observable behaviour • Context specific • Process of growth and development

5. Professional authenticity Miller’s pyramid Non- standardised assessment (emerging) Does Shows how Standardised Assessment (mostly established) Knows how Knows Miller GE. The assessment of clinical skills/competence/performance. Academic Medicine (Supplement) 1990; 65: S63-S7.

6. Goals of WBA • Make a judgement on a student’s performance of a specific task in the clinical setting • Provide guidance to the student through effective feedback to improve future performance of the task

7. Examples • Encounter based • Case Discussion (Long case in the workplace) • Mini-CEX (direct observation clinical/procedural skill) • Global • Clinical Attachment Assessment • Multi-Source Feedback (360) • Shift Feedback Form

8. Year 3 Mini-CEX • Student (or assessor) selects a patient • Student performs a focused clinical task e.g. history taking, physical examination, performing a procedural skill, counseling a patient • Assessor directly observes the encounter • Assessor rates the student’s performance in a range of domains and provides feedback • No longer than 30 min including feedback

9. Year 3 Mini-CEX

10. Challenges • Competence is developmental vs. normative (not satisfactory, borderline, satisfactory, good and excellent) • Assessors do not read the back page – use their own internal rubric • Reluctance to rate a student as “not satisfactory” – “Failure to fail” • Feedback is often limited and not as helpful as it could be • A “tick box” exercise resulting in students chasing a number

11. Crossley, J., Johnson, G., Booth, J., & Wade, W. (2011). Good questions, good answers: construct alignment improves the performance of workplace-based assessment scales. Med Educ, 45(6), 560-569.

12. Construct aligned scales • Judgement on extent to which a supervisor would trust a trainee (or student) to perform a task independently • Descriptors reflect the need for close or more distant supervision • Different contexts will have a different way of construing “independence” • Greater reliability with fewer assessments • 3 vs. 6 Mini-CEXs required to achieve a Generalisability coefficient 0.7

13. Domains of assessment

14. Construct aligned Mini-CEX

15. Providing effective feedback

16. Effective Feedback • Characteristics • Perceived need for change • Technique • Creating an action plan • Coaching

17. Delivering Feedback • Enable learner reaction • Encourage self assessment • Provide an assessment of strengths and weaknesses • Develop an action plan • Document the feedback

18. Year 5 Emergency Medicine Shift Feedback Form

19. Summary • Competence is developmental • Construct aligned scales may help align both assessor judgements on performance and the development of competence by the student • Delivering effective feedback is an important part of the assessment process to guide student learning • Action plan and coaching

20. Questions? j.kwan@uws.edu.au