Assessing Student Learning Outcomes in College Union Programs and Services

1. Assessing Student Learning Outcomes in College Union Programs and Services Association of College Unions International Region 2 Conference SUNY Oneonta November 8, 2008

2. Presenter Patricia Francis, Associate Provost for Institutional Assessment and Effectiveness SUNY Oneonta

3. Session Topics • Why student learning outcomes in Student Affairs? • Assessment “basics” • Developing a unit assessment plan, with a focus on objectives and measuring outcomes • Good assessment practices in college unions – Review and examples • Hands-on opportunity to develop objectives/ measures

4. Session Learning Objectives for Participants • To learn about the newly emerging focus on direct student learning outcomes in student affairs divisions in American colleges and universities • To examine examples of these outcomes, with an emphasis on those from college union programs and services • To develop outcomes specific to their areas of responsibility

5. Student Learning Outcomes in Student Affairs: Rationale • Recent transformation in higher education • From inputs to outcomes • Heightened emphasis on personal development as primary goal of undergraduate education • Recognition that co-curricular environment is as important as the classroom • Important question: How do student development professionals intentionally create conditions that enhance student learning and personal development?

6. Important Assumptions Underlying a “Learning-Oriented Student Affairs Division” (from ACPA, 1996) • Characteristics of a college education person range from complex cognitive skills to the ability to apply knowledge to practical problems to a “coherent integrated sense of identity” • The concepts of learning, personal development, and student development are intertwined and inseparable • Experiences in and outside the classroom – and on and off campus – contribute to student learning and development • Learning and personal development occur through transactions between students and their environments broadly defined

7. And Last, But Not Least “Student affairs professionals are educators who share responsibility with faculty, academic administrators, other staff, and students themselves for creating the conditions under which students are likely to expend time and energy in educational-purposeful activities.”

8. Some Assessment “Basics” • Establishing congruence among institutional goals, programmatic and unit objectives, and assessments • Using a variety of measures, both quantitative and qualitative, in search of convergence • Using existing data sources as much as possible • Assessment as an ongoing, iterative process • Assessment as an opportunity for continuous dialogue, professional development and, most important, improving student services

9. Assessment’s Four Steps • Setting objectives: “What you say you do” • Objective mapping: “How you do what you say you do” • Assessment: “How you know you are doing what you say you do” • “Closing the loop”: “What you do next based on results” • Assessment without #4 = Waste of time!

10. Developing a Unit Assessment Plan: Focus on Objectives and Measures

11. Questions to Ask at the Beginning • Does the division/unit mission statement explicitly address student learning and personal development as primary objectives? • Do staff understand, agree with, and perform in ways congruent with this mission? • How can staff be more intentional about promoting student learning/development (and still provide needed services)? • What measures should be in place to demonstrate effectiveness and inform practice?

12. Overall, Unit Objectives Would Reflect: Institutional effectiveness performance indicators Documentation of all services and programs offered Consideration of all constituents served Tracking of use of services (and by whom) Student satisfaction with services/programs Direct impact of services/programs on students

13. Writing Good Objectives

14. Components of a Good Objective • Who is the target? • What domain of student development is the target (behavioral, cognitive, attitudinal)? • What activity is expected to lead to change? • What change is expected?

15. But Other Things to Keep in Mind Do you have a program/activity in place to bring about outcome? Can desired change in students be measured? How will you know you were successful? Do external standards apply (i.e., in case of external accreditation/certification)?

16. Objective Writing Rule #1: Focus on Student, Not Program Good example: “The employee effectively utilizes the eTime system to punch in/out and to review personal records.” Poor example: “The College Union will emphasize to student employees the importance of reporting to work on time and keeping personal records.”

17. Objective Writing Rule #2: Use Concrete Language and Action Verbs Good example: “Employees will demonstrate a good working knowledge of customer service principles and display these principles in their daily work.” Poor example: “Our objective is to enhance students’ sense of and commitment to customer service.”

18. Objective Writing Rule #3: Focus on Results, Not Process Good example: “Employees will demonstrate clear and accurate knowledge regarding the College’s policies and procedures for student employment.” Poor example: “Students will attend workshop that focuses on college policies and procedures related to student employment.”

19. Measuring Outcomes

20. Basic Principles • Measurement techniques must be rigorous, since unreliable data are of minimal value • Best to use variety of quantitative and qualitative measures • Quantitative easier, but not often as rich • Qualitative often more informative, but require check on scoring (e.g., rubrics) • Indirect measures have value, but should never be sole indicator of effectiveness • Rely as much as possible on existing data sources

21. Types of Information to Include • Survey data • SUNY SOS • NSSE • Local surveys of student satisfaction/ perceptions • National benchmarking data (e.g., ACUI) • Performance-based data • Focus groups 1. Normative/benchmark information whenever possible (external sources, own performance over time) 2. Locally collected data

22. From Measures to Criteria • Assessment criteria reflect your expectations about student performance (i.e., how you know you were successful) • Set criteria at reasonable but challenging levels • Often take these forms: • “90% of students will …..” • “80% of students will score at least 70% on…”

23. Authentic, Performance-Based Assessment: The Value of Rubrics • Rubrics provide reliable means of rating student performance in more qualitative way • Steps in developing rubrics • Use entire staff to help develop and be as specific as possible in differentiating between performance levels • Use existing rubrics as guide • Pilot test to assure scoring is reliable

24. Rubric Example: Leadership Skills (Teamwork) (Univ. of W. Florida)

25. Student Learning Objectives in College Union Programs Review and Examples

26. San Jose State University As a result of development opportunities, training and work experience, student employees will be able to: • demonstrate effective written and verbal communication skills; • improve their time management skills; • enhance their problem solving skills; • develop planning and teamwork skills; and • demonstrate their understanding of ensuring a safe work environment.

27. Davidson College Student employees will demonstrate the ability to: • work with others to provide services and produce programs for the greater community; • coordinate major events such as Spring Frolics, including the planning and handling of all aspects of event production; • manage large-group challenges and handle complaints; and • work independently with minimal staff supervision.

28. Your Turn!

29. Assessing Student Learning Outcomes in College Union Programs and Services Association of College Unions International Region 2 Conference SUNY Oneonta November 8, 2008