Assessing for Accountability and Improvement: Telling YOUR Story

1. Data, Decisions, & Telling YOUR Story • Gina M. Vanacore, • Executive Director, Housing & Residence Life, University of North Texas • HERE IN THOUGHT : Lindsey Fields, M.S., M.Ed., LPC • Assistant Director for Residence Life

2. Why Do We Assess? Accountability Institutional and Programmatic Improvement The relentless pursuit of perfection through innovation drives continuous improvement.

3. About You: Raise your hand if… Time in our field is less than three years ? Originally from Texas ? First generation college student ? You have a master’s degree ? Prior assessment experience ?

4. Take A Moment Please draw a Residence Life staff member conducting assessment activities: • What is this person doing? • If there were posters on the walls describing the work being done what would they say? • What items are on this person’s desk?

5. Why do we see this in that way? How does that vision differ from: • Your images of describing what we do to interested parties • How we visualize staff telling stories about the student experience • How you communicate your achievements

6. Iterative Model Does your Program, Department, and Division have a Strategic Assessment Plan? If not, what expectations are put on your Program, Department, or Division about how, when, and how often to communicate assessment results?

8. Dreams and data are interconnected • Every student has a dream and through our efforts we either help them achieve it or put a roadblock in their path • Data are just numbers unless residence life staff can use it to close loops, make new meaning and improve our programming and services • SO….I believe effort in assessment is a vital component to helping more of our students succeed and vital to that is telling our story in a • Compelling • Interesting • Accurate • Persuasive ADD MORE WORDS HERE …….. manner

9. Stories are how we build empathy

10. Tell Our Story, Build Empathy, Shape Our Culture

11. Where is the story telling in the assessment cycle ?

12. DESIGN THINKINGdschool.standford.edu From Stanford d.School: dschool.standford.edu

13. DESIGNED ASSESSMENT THINKING Lesley D’Souza, 2017

14. Assessment 10 Years Ago and Now • Learning Outcome Oriented • Indirect and Direct Evidence-Based • Technologically Expanding • Complex Foci Depending on Goals/Needs • Increasing Study Complexity and Analysis • Bottom-Up and Top-Down Culture • Master’s level expertise the norm, increasingly Doctoral level expertise needed • Increasing staff sizes • NILOA, SAAL, ASSESS, NASPA and ACPA growth • Satisfaction-Oriented • Indirect Evidence-Based • Technologically Limited • Justification-Focused • Methodological Simplicity • Bottom-Up Culture • Some Master’s level expertise needed • One person shops (usually part of shared duties) • Isolated professional opportunities

15. 7 Key Areas of Assessment • Cultural - Is your department ready to assess? • National Standards – How do you compare to your peers? What does CAS state? • Mission– Is your mission relevant? Current? • Users – Who actually uses your service? How do they match the larger population? • Needs – What do your residents need? What are you offering? Is it still relevant? • Cost Effectiveness – Do residents perceive value? Are you using resources wisely? • Learning Outcomes – What are residents learning? How do you know? Do you integrate learning outcomes into programming?

16. What are some common assessment methods you have heard of or used?

17. Our Assessment Toolbox • Pre and post tests • Marketing/brand encounters • Institutional data mining • Social media analysis • Environmental scans • Student art opportunities • Campus poster review • Direct evidence (ID card scanning and data capture) • Staff initiated rubrics • Student self-scored rubrics • Online surveys • Paper-based surveys • Focus groups • National studies • Historical document review • Secondary data analysis • Reflective journals • One minute essays • Photo analysis • Process cartoons • Expense categorization

18. Resources for Understanding Data •  National Center for Higher Education Management Systems  • http://www.higheredinfo.org/resources.php • National Surveys • Institutional Resources • Institutional Research Office • Graduate programs • Qualtrics or Survey Dashboard

19. Assessment from Problem to Report How Might You Frame Out an Approach? Source: Schuh, J. H., Upcraft, M. L. & Associates. (2001)

20. Step 10: Determine the implications of the study for policy and practice We suggest you rely on your findings and present how your findings impact the campus or program area. • What are the implications of this study? • What approaches to solving the problem should be considered in light of this study? • What policies and procedures need to be created or overhauled? • What is the call for action for the UNIT?

21. Step 11: Report the results effectively • Understand we must package the findings carefully so that it motivates change • How a study is distributed and formatted may be more important than the results found • Segment your audience and tailor specific reports to specific audiences • Make sure the results get into the right hands of people who can make a difference

22. Communicate Data Findings TEACH They know I’m paying attention to it (data), that I care about it, that I talk to them about it. Monitoring data and responding to it can be a powerful culture change on campus.

23. Sharing YOUR Story TEACH • Inside your department • Outside your department, within your institution • Outside your institution Examples on the following slides may or may not be representative of actual data/results from the University of North Texas

24. Other Institutional Practices – Ohio State

25. Other Institutional Practices – Southern Illinois

29. Longitudinal Data Highlighting your own growth

31. What stood out to you about these schools and how they translate data into action?

32. Elements of Storytelling From www.forbes.com: Data Storytelling: The Essential Data Science Skill Everyone Needs, 2016

33. COMBINED Elements of Storytelling From www.forbes.com: Data Storytelling: The Essential Data Science Skill Everyone Needs, 2016

34. Executive Summaries (cont.) • Standard length – 2 pages • Should be written in jargon-free language • Content can be used for: • annual reports • performance evaluations • Resumes • Conference proposals • Grants from NASPA, ACPA, etc. • reaccreditation • website content • print marketing • speeches

36. Assess YOUR Level of Sharing YOUR Story • Assessment data and analysis is regularly communicated to campus partners and other relevant educational partners. • Assessment data and analysis is regularly communicated to the professional staff members and educators who design and execute the program/initiative being assessed. • Assessment data and analysis is regularly communicated to student staff members who execute the program/initiative being assessed. • Assessment data is regularly analyzed and used to enhance the effectiveness of the program/initiative as a part of a commitment to continuous improvement.

37. Assess YOUR Level of Sharing YOUR Story • A review process is developed whereby the entire program/initiative is reviewed by educational experts who provide guidance on its improvement. • A review team incorporates a diverse set of internal and external voices – including students, staff, faculty, campus partners, and educational experts. • Suggestions for improvement made through a review process are acted upon and incorporated into future planning.

38. Concluding Thoughts

39. Why Assessment Takes Partners • This work builds community from a data-driven perspective • This work helps inform what students need from you the most • This work provides your Department with a platform to talk about itself • This work can help you become more marketable in the higher-ed workspace • This work can make all the difference in the world to our students – remember we are in the dreams business

40. One final drawing: Draw another Housing staff member conducting assessment? • What is this person now doing? • What items are on this person’s desk? • If there were posters on the walls describing the work being done what would they now say?

41. I love to talk about assessment, research and my job….You can reach me at gina.vanacore@unt.edu940 565 2282