"Assess This": Lessons Learned from a 12-Month Access Assessment Project

1. "Assess This": Lessons Learned from a 12-Month Access Assessment Project Emily Quinn, M.Ed.Tyler Johnson, M.Ed.Joshua Mason, M.Ed. Disability Resource Center University of Tennessee at Chattanooga

2. Conference Inclusion Statement We ask you to join us in creating a culture that reflects… Access and Inclusion and Civility and Respect …this week and in all aspects of our organization.

3. Welcome!

4. Why Assessment? • "Let's get better." • Understand the students with whom you work • Find out what you do well and what you need to improve • Challenge the vulnerability that occurs when you're stagnant  • Acknowledge patterns in utilization of resources • Review the effectiveness of your programs and services • Communicate in universal language with administration and colleagues in other departments • Collect and share data to help frame disability culture on campus

5. Overview & Learning Objectives Goal 1: Participants will identify challenges associated with and strategies for implementation of assessment in disability services.  Action Plan: Presenters will explain and show examples of a framework for engaging in assessment and discuss with participants Goal 2: Participants will reflect on current practices and strategies for assessment implementation in their own higher education setting. Action Plan: Presenters will explain process for developing an assessment plan and participants will work through a template plan. 

6. Activity Step One: Learning Objectives • What is a critical element of your work right now?  • What do you want to know? • Take a minute to draft one or two measurable learning objectives. 

7. Our Assessment Experience

8. Project Overview • Structure & Timeline • A 'team within the team' responsible for assessment guidance • Leadership member, DRC coordinator, & Mosaic coordinator • CAS (Council for Advancement of Standards in Higher Education)  • July 2018 – July 2019 • Monthly education, workshop, and assessment review with full staff • Assessment team coordinated projects, but all staff were involved in consultation and implementation • Department programming funds contingent upon assessment plan (not outcome)

9. Administrative Goals • "Let's get better." • Improve individual staff competency  • Improve services for students • Tell our story powerfully and more frequently • Understand perceptions and expectations for our department • Align department goals, division goals, and UTC goals

10. Administrative Challenges • Building assessment base knowledge for a full team • Developing and implementing assessment takes administrative time • “Buy in” from a full team with a range of responsibilities, goals, and opinions • Last minute and reactive assessment plans are hard to manage

11. Disability Resource Center Assessment Report

12. DRC: Assessing Operations • Accommodate: students served, disability categories, accommodations used • Uses: Funding, advertising, internal training, departmental collaboration, external training for faculty, staff and students • Next steps: • Use of current data for position alignment and funding • Assess timeliness of accommodation provision • Assess quality of accommodation provision

13. DRC: Assessing Operations2018/2019 Academic Year Disability Frequency

14. DRC: Assessing Operations • 2018/2019 Academic Year Disability Frequency • Data analysis yielded increase of students requesting accommodations due to impacts mental health diagnoses • Information used to initiate collaboration and internal training with UTC Counseling • Next steps: training for UTC faculty and staff 

15. DRC: Assessing Academic Coaching • Qualtrics: students served, frequency of meetings, topics covered, & campus resource referrals • Uses: Funding for additional staff positions due to demand, internal and external training, collaboration with similar university service providers • Next steps: • Assess efficacy of academic coaching and redundancies/partnerships with similar university supports

16. DRC: Assessing Academic CoachingSpring 2019 Topics

17. DRC: Assessing Academic Coaching • Spring 2019 Topics: • Data analysis yielded access coordinators utilizing large number of hours for academic coaching • Data utilized to secure funding for additional academic coaching position • Next steps: training regarding high frequency topics such as college life balance

18. DRC: Assessing Programming • Paper: Attendance, Learning Outcomes • Uses: Funding, allocation of resources, efficacy of programming • Next steps: • Consider different programming modalities, revising high-attendance programming, create assessments for future remote-programming 

19. DRC: Assessing ProgrammingAttendance Data Example: Spring Open House • Have you ever attended a DRC or Mosaic event before today (58 attendees, 42 responses)?

20. DRC: Assessing Programming • 2018/2019 Academic Year Program Attendance • DRC Open House was most highly attended program; technology tutorial sessions yielded low attendance throughout academic year • Information used to change resource allocation and modalities to online tutorials for technology and integration during DRC Open House programs • Next steps: create assessments for online tutorials

21. Mosaic Program Assessment Report

22. Mosaic & Assessment • Primary Format: • Utilized online platforms like Qualtrics and QuestionPro that help create the assessment.  • Simple paper surveys. • Have alternate versions available 

23. Mosaic & Assessment • Primary Uses: • Coaching Tracking & Effectiveness • Mosaic specific coaching effectiveness • Campus partnerships and collaboration • Programming Effectiveness • External Programming • Professional Development • Process Effectiveness • Understanding better whether current processes are sustainable and useful

24. Next Steps • Create a more sustainable system for assessment • Making sure that staff has access and understands process to initiate assessment • Stable systems allow for more streamlined efficiency • Present findings to staff for external use and understanding • Utilize and report findings to University and beyond 

25. Activity Step Two: Methods • What are some options you can consider to gather information? • Compare and contrast the methods you can utilize to gather information about your topic.

26. Lessons Learned

27. Leadership • You don't have to be an assessment expert to do assessment! • When in doubt, choose a quick and easy-to-access method for collecting feedback. • Expect an ego-check when you review and analyze information. • Assessment is ongoing; it's not a one-time project.

28. DRC Coordinator • Electronic databases yield a wealth of valuable data • Communicating importance of data trends of student populations is challenging but vital • University-wide funding models align with data-driven programming • Proactive, thoughtful approaches to consideration of learning outcomes and assessment of programming is essential

29. Program Coordinator • There was more to learn from assessment than just what was going well or going poorly.  • It is necessary to create a system of assessment that the whole office can participate in and buy in to.  • It is not an overnight change. It takes time to effectively assess a program. 

30. Strategies & Resources Create, Analyze, and & Tell Your Story

31. Create • Start with a simple question or thought.  • Create learning objectives that are measurable and reportable. • Decide on a method or platform to use (or a combination) • Keep a quick template ready to edit and use • Identify potential models from which you can pull language, methods, and base questions

32. Analyze • Analyze using what is known and easily ready: Excel, Question Pro, PowerPoint, Google Sheets, etc.... • Built in analytics – training at your institution? • Reference the learning/program objectives you created as you review the results 

33. Tell Your Story • Share outcomes with all department staff members • Quick, easy-to-remember data points can be pulled into many conversations  • Everyone's on the same page, telling the same story • Publish key results from assessment projects to website and/or newsletter • Ask for what you need using the details that communicate the necessity of the request (funding, staffing, etc.)

34. Assessment Design Workshop

35. Activity Step 3: Design Your Assessment • Review your learning outcomes • Choose a method • Draft a short assessment  • Share with a partner for feedback

36. Session Evaluation Please see session moderator for paper evaluation form or complete the evaluation online.