1 / 53

Proposal for Assessment of Clinical Placement Andrew Kilgour

Proposal for Assessment of Clinical Placement Andrew Kilgour. Factors involved in design. Transition table Alignment of University standards and professional accreditation body standards REA/competency review Educational literature Input from the profession locally. Transition Table.

judson
Download Presentation

Proposal for Assessment of Clinical Placement Andrew Kilgour

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Proposal for Assessment of Clinical PlacementAndrew Kilgour

  2. Factors involved in design • Transition table • Alignment of University standards and professional accreditation body standards • REA/competency review • Educational literature • Input from the profession locally

  3. Transition Table

  4. Transition table

  5. Original transition table excellent for progression from one year of program to the next • Expanded version used for progression from one placement block to the next • Based on original, but extra levels inserted to define expectations/achievement levels for each block

  6. Using the expanded transition table to assess student performance Assessment of a student’s clinical performance is OBSERVATIONAL ASSESSMENT “Assessment of clinical performance during training ..... range(s) from informal observations of students in clinical settings to very formal (and sometimes complex) systems of data gathering from multiple raters about the performance of health professions students in actual clinical settings, with real patients...” Downing (2009)- p 6

  7. Downing (2009) states that “...no more than seven quality rating categories should be used on observational assessment instruments.” (p 209) Williams, Klamen & McGaghie (2003) agree: “Five to seven quality ratings are optimal.... Additional quality rating categories add no performance information and just complicate the task for raters.” Downing, S. M. (2009). Assessment in health professions education: Routledge. Williams, R. G., Klamen, D. A., & McGaghie, W. C. (2003). SPECIAL ARTICLE: Cognitive, Social and Environmental Sources of Bias in Clinical Performance Ratings. Teaching and Learning in Medicine, 15(4), 270-292. doi: 10.1207/S15328015TLM1504_11

  8. Standards based assessment of clinical performance Assessment is the making of judgments about how students’ work meets appropriate standards (Boud & Dochy 2010) Assessment of student achievements is judged against consistent national and international standards that are subject to continuing dialogue, review and justification within disciplinary and professional communities (Boud & Dochy 2010) In addition to knowing about standards, students need to compare their actual levels of performance with these standards (Sadler 1989) Teachers must be prepared to assess based on judgments about the quality of work, not based on wanting to reward effort or improvement (Sadler 2005) Boud, D., & Dochy, F. (2010). Assessment 2020. Seven propositions for assessment reform in higher education. status: published. Sadler *, D. R. (2005). Interpretations of criteria‐based assessment and grading in higher education. Assessment & Evaluation in Higher Education, 30(2), 175-194. doi: 10.1080/0260293042000264262 Sadler, D. R. (1989). Formative assessment and the design of instructional systems. Instructional Science, 18(2), 119-144. doi: 10.1007/BF00117714

  9. Application of standards-based assessment The performance of the student is measured against the standard expected in each of the six categories: Practice knowledge; Degree of independence; Time efficiency; Professional/clinical reasoning; Task ability; Interpersonal capabilities

  10. The standard expected of students in each of these categories varies from one placement block to the next, but the layout and format of the assessment form is the same for each block Williams, Klamen and McGaghie (2003) emphasise the importance of keeping rating instruments short, stating: “Adding more items only boosts rater time and makes the task harder.” Williams, R. G., Klamen, D. A., & McGaghie, W. C. (2003). SPECIAL ARTICLE: Cognitive, Social and Environmental Sources of Bias in Clinical Performance Ratings. Teaching and Learning in Medicine, 15(4), 270-292. doi: 10.1207/S15328015TLM1504_11

  11. Global assessment Williams, Klamen and McGaghie (2003) also suggest that a global item should be included in the assessment form used. In the transition table, the global item is referred to as “general description of attainment” These have also been expanded – see next slide Williams, R. G., Klamen, D. A., & McGaghie, W. C. (2003). SPECIAL ARTICLE: Cognitive, Social and Environmental Sources of Bias in Clinical Performance Ratings. Teaching and Learning in Medicine, 15(4), 270-292. doi: 10.1207/S15328015TLM1504_11

  12. Global items at each placement block

  13. How global items appear on the assessment form Global Description of Expected Performance for Yr 1 Placement Block: Capable of interacting in the workplace in a professional manner. Demonstrates limited clinical skills under close supervision. Please write your comments regarding the student’s performance below: ________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________ Signed ______________________ Date _______________________

  14. Alignment of University standards and professional accreditation body standards

  15. Work done so far......... • A lot of work has already been done mapping the “Professional and Practice-Based Education Standards” to the “AIR Competency Based Assessment” and the “ANZSNM Competency Based Standards” • (See MRS_Clinical_Subject_Mapping_2Feb.docx)

  16. In order to track a student’s progress in meeting these standards, a longitudinal documentation is required • How do we do this?

  17. The e-portfolio... • “In health professions education, a portfolio is a collection of evidence documenting progress, accomplishments and achievements over time...... portfolios provide a means to assess competencies such as self-directed learning, which are demonstrated over the course of months or years. Portfolios also comprise a vehicle for the longitudinal, multi-method, multi-source assessment of learner achievement.” Downing (2009) p 287 Downing, S. M. (2009). Assessment in health professions education: Routledge

  18. How we use portfolios... • “...portfolios are best used as part of a comprehensive assessment system that can triangulate on learner competence.” (Melville, Rees, Brookfield, & Anderson, 2004) • The formative and summative functions of the portfolio should be separated, and learners should be allowed to select “best work” or “best evidence” to submit (pinsky & Fryer-Edwards, 2004) Melville, C., Rees, M., Brookfield, D., & Anderson, J. (2004). Portfolios for assessment of paediatric specialist registrars. Medical Education, 38(10), 1117-1125. Pinsky, L. E. M. D., & Fryer-edwards, K. (2004). Diving for PERLS. Journal of General Internal Medicine, 19(5), 582-587.

  19. CSU Portfolio • Communications and Interactions (CI)1. Demonstrate ethical, respectful, supportive and culturally competent communication and interaction consistent with professional codes of practice. 2. Demonstrate proficient and professional communication, through a variety of delivery media/modes to specialist and non specialist audiences. 3. Demonstrate teamwork abilities, leadership, collegiality, conflict management and professional conventions at the level of an emerging professional. • Professional Judgement (PJ)1. Demonstrate critical and creative decision making and problem solving that is context relevant. 2. Make work-related decisions that are aligned with professional values, standards and ethics and address legal requirements. 3. Demonstrate accountability by being able to report and articulate the basis for professional decisions and actions. • Profession Competence and Work readiness (WR)1. Demonstrate the discipline-specific technical capabilities of a beginning practitioner or professional. 2. Integrate discipline, practical and social knowledge and skills in contemporary professional practice. 3. Demonstrate an understanding of legal and ethical requirements and the boundaries in which to work. 4. Recognise and respond appropriately to unsafe practice. 5. Demonstrate an ability to plan and manage workloads. • Professionalism and Citizenship (PC)

  20. Professionalism and Citizenship (PC) • 1. Demonstrate commitment, and an ability to undertake life-long learning through reflection, self-evaluation and self-improvement2. Exhibit qualities and behaviours consistent with professional values informed by social justice, global citizenship, Indigenous and cultural competencies and inclusion principles. 3. Explain how practice is informed by knowledge of continuous quality improvement, sustainability and global trends in practice. • Information Literacy (IL)1. Demonstrate an ability to critique new information and determine its relevance to a given situation. 2. Demonstrate efficacy in the use of information and communication technologies as part of: a) learning b) professional practice.

  21. REA/Competency Review

  22. Existing REA.... • Either a student is competent in an examination or they are not- little merit in giving them a score for different facets of an examination • Competency should be a high standard. It is a mastery process, and if a student doesn’t achieve it the first time, they are free to try again • Old forms use outdated terminology (ie “film type”)

  23. Where Do REAs Fit Into Assessment Scheme?

  24. Alternate Model • The model can be adapted to include a “gateway” into Year 4 • This is considered essential to ensure that our Yr 4 students are at the standard expected in the workforce

  25. The proposed assessment schedule meets the criteria.......... • Triangulation: e-portfolio is run parallel to clinical supervisor assessment and individual examination category assessment (REAs) • Formative and summative functions separated: reflective journal entries formative, supervisor assessments summative. Entered in separate “assets” in e-portfolio • Tied together in capstone assessment

  26. Educational Literature

  27. Holistic Knowledge Integration • “By concentrating on discrete pieces of knowledge and skill, there is a distinct danger of only dealing with fairly superficial aspects of professional practice, whilst ignoring the holistic way in which such knowledge and skill is integrated and coordinated in actual professional practice.” (Hager & Gonczi, 1991) • We need to look at the big picture of student performance, rather than assessing each little aspect of performance in detail Hager, P., & Gonczi, A. (1991). Competency‐based standards: a boon for continuing professional education? Studies in Continuing Education, 13(1), 24-40. doi: 10.1080/0158037910130103

  28. The Reason for Standards-based Assessment....... • Students deserve to have work graded strictly according to quality, without comparison to other students, and without their academic history being considered. Students deserve to know the bases on which judgments about their work are made (Sadler 2009) Sadler, D. R. (2009). Grade integrity and the representation of academic achievement. Studies in Higher Education, 34(7), 807-826. doi: 10.1080/03075070802706553

  29. Judgment of Professional Competence • Professional competence involves being able to do high-level tasks consistently well. When it comes to grading, we need to arrive at judgments which are or would be consistent with peers even when there are no independent sources of data to substantiate those professional judgments (Sadler 2011) Sadler, D. R. (2011). Academic freedom, achievement standards and professional identity. Quality in Higher Education, 17(1), 85-100. doi: 10.1080/13538322.2011.554639

  30. Assessment Should Facilitate Students Monitoring Their Own Learning • Even when teachers provide students with valid and reliable judgments about the quality of their work, improvements do not necessarily follow. Assessments should develop the ability of students to exercise executive control over their own productive activities, and eventually to become independent and fully self-monitoring(Sadler 1989) Sadler, D. R. (1989). Formative assessment and the design of instructional systems. Instructional Science, 18(2), 119-144. doi: 10.1007/BF00117714

  31. The Role of Reflective Writing in Assessment • “The process of learning is a process of students devising learning strategies to solve the challenges their motives have defined for them” (Wilson & Fowler 2005) • Reflective writing facilitates students devising their own learning strategies Wilson *, K., & Fowler, J. (2005). Assessing the impact of learning environments on students' approaches to learning: comparing conventional and action learning designs. Assessment & Evaluation in Higher Education, 30(1), 87-101. doi: 10.1080/0260293042003251770

  32. Integration of Formal and Informal Learning • Work-related programs need to be flexible and responsive to the circumstances of the learner and of the work setting. They must not focus on a narrow competency-based approach, rather be more open to the learning derived from the student experience. The design of the assessments must allow both formal and informal learning to be integrated in ways which will demonstrate their graduate qualities (Clements & Cord, 2011) Clements, M. D., & Cord, B. A. (2011). Assessment guiding learning: developing graduate qualities in an experiential learning programme. Assessment & Evaluation in Higher Education, 38(1), 114-124. doi: 10.1080/02602938.2011.609314

  33. Learning as a Process • Traditional education has focused on content as the object of learning. When we view learning as a process, we focus on learning as the creation of circumstances that favour acquisition of ideas by the mind. Many capabilities and abilities that are necessary for competence are developed by the view of learning as a process. These include the capacity to analyse a situation, the ability to plan ahead, and the ability to gather pertinent information. These processes make learning processes more likely to be effective (Hager 2004) Hager, P. (2004). The competence affair, or why vocational education and training urgently needs a new understanding of learning. Journal of Vocational Education & Training, 56(3), 409-433. doi: 10.1080/13636820400200262

  34. Input From The Profession Locally

  35. Why Invite Input From the Profession? • “Stakeholder involvement in the design of assessment programs not only promotes input of creative ideas, but also ensures a certain fitness for practice. It can give stakeholders a sense of ownership of the program, thereby gaining their support, without which goals can remain elusive” (Dijkstra, Vleuten & Schuwirth, 2010) Dijkstra, J., Vleuten, C. P. M., & Schuwirth, L. W. T. (2010). A new framework for designing programmes of assessment. Advances in Health Sciences Education, 15(3), 379-393. doi: 10.1007/s10459-009-9205-z

  36. How Information Was Gathered.... • Survey, whose link was published by the AIR in 2012: • https://www.surveymonkey.com/s/VBH6YHD • 103 responses nationwide

  37. Who Responded?

  38. What They Said....

  39. Comments on What Should Be Assessed • I think an importance should be placed on CPD at a earlier stage in Radiography it may impact behaviours at a later stage • With the influx of international students, perhaps clinical communication skills appropriate or inappropriate • Attitude and professionalism • Ability to accept feedback and improve • Social skills are vitally important and too many students have absolutely none! should be emphasised more • Ability to work as part of a team. Lack of teamwork can impact on clinical competency and patient outcomes • Communication - with patients AND staff. Extremely important

  40. What we are proposing..........

  41. CSU will be introducing the new assessment form for 2B placement, commencing 24 June • We have already started the reflective journals, using pebblepad, and are getting some excellent results • We propose that Sydney begin the trial of the new form with the 2.2 placement, commencing in July

More Related