1 / 64

CREATING A CULTURE OF ASSESSMENT

CREATING A CULTURE OF ASSESSMENT. Closing the Loop. Dean Kenneth B. Roberts, Ph.D. Heidi M. Anderson, Ph.D. Eleanora Bird, M.S. Jeff Cain, M.S. Office of Education Innovation University of Kentucky College of Pharmacy. Outline. Literature “Creating a Culture of Assessment”

Download Presentation

CREATING A CULTURE OF ASSESSMENT

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. CREATING A CULTURE OF ASSESSMENT Closing the Loop Dean Kenneth B. Roberts, Ph.D. Heidi M. Anderson, Ph.D. Eleanora Bird, M.S. Jeff Cain, M.S. Office of Education Innovation University of Kentucky College of Pharmacy

  2. Outline • Literature “Creating a Culture of Assessment” • UKCOP Case Study • Created an office to handle the College assessment efforts • Faculty development and involvement • Student involvement • Documenting current assessment efforts • Future Directions at UKCOP

  3. Creatinga Culture of Assessment 3

  4. Culture of Assessment • Leadership must be ‘on board’ • Mission, resources, use results for improvement • Involve all stakeholders • Faculty • Students • Alumni

  5. Culture of Assessment • Make assessment relevant • Course reviews • Surveys, rubrics • Focus on teaching and learning rather than assessment • Focus conversations not on assessment, but on helping students learn

  6. Culture of Assessment • Provide opportunities to learn about assessment • Minimize the burden of assessment on faculty and students • Provide centralized leadership, coordination and support for assessment • Provide resources • Keep paperwork to a minimal

  7. Culture of Assessment • Start small, Set up clear expectations and be flexible • Celebrate and reward assessment efforts • Give faculty and staff written recognition • Encourage leaders to commend publicly • Provide special funding • Honor faculty and staff who obtain external grants that will strengthen assessment

  8. Culture of Assessment • Use assessment results appropriately • Keep faculty, students and staff informed • Don’t penalize faculty whose assessments results are less than positive • Don’t expect everyone to get on board

  9. (UKCOP) 9

  10. Vision • Office of Education Innovation (OEI) was developed to assist faculty • Teaching • Learning • ASSESSMENT • Technology • Scholarship of Teaching Today

  11. Mission • The mission of the “Office of Education Innovation” (OEI) is to provide assistance to faculty in developing and evaluating the professional pharmacy and graduate programs within the University of Kentucky, College of Pharmacy (UKCOP).

  12. Community • Dean’s Leadership Team • Office of Academic Affairs • Office of Education Innovation (4 FTEs) • Assistant Dean • Assessment Coordinator • Information Technology Officer • Administrative Assistant • Faculty • Students • Other Participants TEAM

  13. www.mc.uky.edu/Pharmacy/oei

  14. Who the office works with? • The “Office of Education Innovation” (OEI) works closely with: • UKCOP Course Coordinators/Directors • UKCOP Individual Faculty • UKCOP Departments • UKCOP Students • External Constituents to College

  15. OEI Activities and Services • Curriculum • Assessment • Instructional Technology • Faculty Development (Enrichment) • Scholarship of Teaching Today

  16. UKCOP Faculty Development: How, What and Why of Assessment? 16

  17. Why do assessment? • Accountability • ACPE Accreditation • University Review • Curriculum committee requirements • Enhance student’s educational development • Decision-making

  18. Accreditation • July 2000 accreditation standards require assessment activities to be aligned with mission and goals: • achievement indicators, • formative and summative formats, • outcomes assessment, and • systematic assessment plan.

  19. Used for Improvement Ongoing Use Multiple Measures Assessment ProcessDriven by Mission Reassess

  20. What is the Assessment Process? • “Systematic collection, review and use of information about educational programs undertaken for the purpose of improving student learning and development” • Palumbo and Banta, 1999, p. 3

  21. Institutional Academic CURRICULUM COURSES Teaching-learning Methods Assessment Methods Individual STUDENTS: Learning Performance Attitudes Behaviors Skills Satisfaction Levels of Assessment

  22. Institutional UNIV REVIEW ACCREDITATION Individual Faculty SELF Teaching Portfolio Peer Observation Scholarly Productivity Levels of Assessment

  23. Student Student Learning Outcomes Performance Faculty Contribution to teaching Scholarly productivity National visibility Program Graduation rates Licensure pass/fail Curriculum Courses Teaching-Learning Methods What is Assessed?

  24. Difference: Student Assessment vs. Program Assessment • Student Assessment is a teaching process designed to improve learning • Program Assessment is an institutional process designed to improve the program

  25. Student Assessment • Carefully define your student learning outcomes • If you don’t know what you’re looking for, you will have a difficult time measuring it. • Outcome statements should describe student learning or behavior • Outcome: “the result”

  26. Student Assessment • Start simple. Choose a limited number of outcomes to measure at first. • Begin modestly so that the data collected can help to define and refresh accomplishments

  27. Faculty Involvement 27

  28. Faculty Involvement • Faculty learning about assessment: • Website developed to keep them abreast of activities • Faculty participation on committees: • Assessment committee • Curriculum committee

  29. Faculty Involvement • Self-study reports • Curricular purposes • Curricular Outcomes • Course Review (online submission) • Course Evaluation (online) • Survey development • PS Graduate Program • Requested peer evaluation of teaching

  30. UKCOP General Outcomes • Foundation Knowledge • Critical Thinking • Information Management • Communications • Professionalism • Social Interaction, Citizenship, Leadership • Life-long Learning

  31. UKCOP Professional Outcomes • Patient Care • Medication Distribution • Human Resources Management and Operations • Research • Teaching

  32. Faculty Involvement • Focus on Teaching/Learning • Generate the appropriate assessment questions • Determine appropriate ways to measure outcomes • Decide on when to assess the students • Prepare and utilize the reports

  33. Faculty Involvement • Developing Questions to Ask: • What do we want students to know, and do? • How will we know they are learning what we hope they will? • What is the effect of our curricula, courses, teaching, styles and educational tools? • Did our changes bring about the effect we desired?

  34. Student Involvement 34

  35. Student Involvement • Incoming orientation: • Explain the College’s assessment, rationale relating to accreditation standards, role of OEI • Review IRB protocol and informed consent • Assessments throughout their 4 years • Anonymous (code for tracking) • Reports in aggregate fashion

  36. Student Participation • Curriculum Committee • Student Liaison Committee (SLC) • meets 3x per semester • Assessment Committee • Course • Performance Assessments • Other • Accreditation Visit, Program Review

  37. UKCOPCase Study:Documenting Current Effort 37

  38. Documenting Current Efforts Purpose • Determine what assessments were already taking place within the College • Develop an assessment inventory list and appropriate questions about student learning

  39. Goal: Develop a Complete Student Profile • Self-assessment of outcome abilities • Attitudes towards peer/self assessment • Performance records from courses • CAPP results (integrated abilities) • Student portfolios • Yearly interviews • Post-graduation surveys

  40. Three Phases • Phase 1: Needs Analysis • Phase 2: Recommendations and Plan Development • Phase 3: Align Assessment Plan, Outcome Indicators

  41. Needs Analysis Research • Literature review was conducted to determine types of assessment data that might be useful to gather for the program’s assessment functions • Student records • Program evaluations • Student performance and competencies

  42. Needs Analysis Data Audit • Class of 2002-2005: Student and educational outcomes records • Data obtained from various offices and files within the college: • Demographic • Entry-exam scores • Learning styles • GPAs

  43. Needs Analysis Data Audit • Data portfolios for several student cohorts were examined • Faculty • Other sources: • Institutional Research Office • Boards of Pharmacy • State/local pharmacy associations

  44. Data Audit Report

  45. Needs Analysis Data Audit • Success • External IR officer requested to use the college audit form as a model for the campus • Data are being collected from all audiences in a wide variety of venues • Some data are being disseminated in meaningful way

  46. Lessons Learned from Data Audit • Amount & breadth of information was considerable • Gaps/Duplication • Some areas require more or different data

  47. Lessons Learned from Data Audit • Length and frequency of surveys • Faculty engagement • Incorporate findings into College assessment plan • Reassess

  48. Phase 2 Recommendations about research questions to assessment committee • I-E-O data management model used to classify student and program data (Astin, 1993) I=inputs E=environments O=outcomes

  49. Phase 2 A. Content Analysis B. Educational Impact Which educational experiences/environments maximize student learning during the first year of pharmacy school?

  50. Phase 2 Ho: Structured Clinical Instructive Models (SCIMs), prior to students participating in EPPE, statistically improve student critical thinking skills. Ho: Videotaped counseling sessions statistically improve student communication skills during the first year of pharmacy school.

More Related