Assessment 101: An Overview

1. Assessment 101: An Overview Graduate Assistant Professional Development Program September 27, 2011 Amy Franklin, Director, Planning & Assessment Division of Student Affairs & Enrollment Management Northern Illinois University

2. Ask Yourself . . . • What decision have you made about a program you were a part of recently? • What evidence did you use to inform that decision? • What was it that you were trying to influence about your program when making that decision with the stated evidence? Adapted from Bresciani 2010

3. And the Answer Is . . . • If you used the evidence (data) you got to make a decision, you were doing assessment! • Doing assessment is about documenting the evidence we use to make decisions • Documenting evidence • helps our departments stay focused • provides proof to our stakeholders, and • allows us to manage our knowledge more effectively

4. Learning Outcomes • Explain the role of assessment • Explain the 3 M’s of planning & assessment • List several tools that could be used to measure program learning objectives • Practice constructingmeaningful, measurable, & manageable student learning outcomes

5. Assessment “Assessment is any effort to gather, analyze, and interpret evidence which describes institutional, departmental, divisional, or agency effectiveness” Upcraft and Schuh (1996) p. 18

6. Questions Assessment Answers • What are we trying to do and why are we doing it? • What do we expect the student to know or to do as a result of our program? • How well are we doing it? • How do we know? • How do we use the information to improve? • Does that work? Bresciani, M.J. (2002)

7. Planning & Assessment Cycle Identify dept. goals, pgm obj’s & learning outcomes Plan assessment method(s) Strategic Plan Mission Goals Objectives Implement program Use results Compile & review data Collect data

8. MMM Is the learning and the assessment: MEANINGFUL? MEASURABLE? MANAGEABLE? Simon, D.M. (2006)

9. A strategy is . . . “ a plan, method, or series of maneuvers or stratagems for obtaining a specific goal or result. ..” www.dictionary.com

10. Methods & Strategies • Accreditation • Benchmarking • Focus Groups • Internal/External Review • Interviews • Needs Assessment • Observation • Other • Pre-test/Post-test • Program Evaluation • Published test/survey • Reflective Journals • Satisfaction Survey • Retrospective Pre-Test • Student GPAs • Student Retention • Tracking

11. Pre Assessment Questions Why are we doing this assessment? What will we assess? How will we assess? Who will assess? How will the results be analyzed? How will the results be communicated/to whom? Upcraft and Schuh, 1996

12. Once we have the answers to those 6 questions, we can proceed with our selected strategy

13. Frequency Tracking, utilization, contacts Identifies who is benefiting from our programs and services Helps identify who is NOT using programs and services

14. Needs Assessment Provides information about students’ needs Focus groups Interviews Surveys Open forums More methods = richer data

15. Satisfaction Assessment May or may not be related to “good educational practice” Static vs. Active measures secret shoppers, national vs. local surveys, interviews, document review, observation More methods = richer data (Upcraft & Schuh, 1996)

16. Focus: Student Learning “The involvement of student affairs professionals in the assessment of student learning and development is critical—and the need for your involvement is urgent. As new student affairs professionals, you can leverage your inherent passion for high-quality, holistic student learning and development as well as strengthen the credibility of the issues you raise by helping your organization implement systematic assessment.” Bresciani, M.J. (2011)

17. Student LEARNING Outcomes Adapted from A.Barsch & D. Simon (2005) by A. Franklin Student learning outcomes encompass a wide range of student attributes and abilities, both cognitive, affective, and psychomotor which measure how students’ college experiences have supported their personal or academic development

18. TYPES: Student Learning Outcomes Cognitive: Demonstration of specific knowledge or abilities Affective: Emotions, values, appreciations, enthusiasms, motivations & attitudes Psychomotor: physical skills, coordination, movement Adapted from A. Barsch & D. Simon (2005) by A. Franklin; D. Clark (2007)

19. Learning Outcomes Phrases Simon, D.M. (2006) • Students will develop . . . • Students will learn . . . • Students will be able to demonstrate. . . • Students will be able to identify. . .

20. Learning Outcomes Phrases • Students will compare. . . • Students will evaluate. . . • Students will differentiate. . . • Students will apply. . .

21. Sample Learning Outcomes NIU Cares’ Day participants will describe how their participation influenced their thinking about volunteerism. By the end of the training, Community Advisors will be able to correctly determine when to contact supervisors regarding a sensitive student issue.

22. Activity Writing Student Learning Outcomes

23. ABCD Model Zerwas, S., 2006 A = Audience B = Behavior C = Condition D = Degree

24. SLO Equation A (Audience) • B (Behavior) • C (Condition) • D (Degree) SLO (Student Learning Outcome) A + B + C + D = SLO

25. Condition • In a 1-on-1 meeting • While assisting with a department- sponsored event • At the student organizational orientation meeting • At the end of the semester

26. Degree • How much? • When? • How many? • How fast?

27. Sample SLO #1 AudienceBehaviorConditionDegree By the end of the training, all participantswill be able to correctly determinewhen to contact their supervisor regarding a sensitive student issue.

28. Sample SLO #2 AudienceBehaviorConditionDegree Three-fourths of students who join the Emerging Leaders program will identify two specific examples of their personal leadership style when completing a reflective journal at the end of the school year • Components? • Other measurement strategies (methods)?

29. Data Collection Qualitative: Answers WHY? Looks for meaning(e.g. interview, focus group) Quantitative: Answers HOW MANY? Looks for statistical relationships (e.g. % #’s, etc.) Upcraft and Schuh (1996), pp. 63-70 Measurement strategy depends on project purpose

30. Methods • Accreditation • Benchmarking • Focus Groups • Internal/External Review • Interviews • Needs Assessment • Observation • Retrospective Pre-Test • Other • Pre-test/Post-test • Program Evaluation • Published test/survey • Reflective Journals • Satisfaction Survey • Student GPAs • Student Retention • Tracking

31. Methods Survey Question: When you experience a sensitive student issue, under what circumstances MUST you call your supervisor? Role Play with Checklist Training participants given scenario, trainer reviews actions based on checklist of criteria

32. You have the evidence . . . . . . . .now what? Survey Results 75% of participants correctly identify when they must call their supervisor for sensitive student issues Role Play Results All participants in role play meet at least 8 of 10 criterion on checklist

33. Other Evidence? What other evidence do you collect? What do you do with it?

34. Practice In groups of 2 or 3: Given a specific program or service of your choice, determine what you might assess and complete the ABCD worksheet.

35. Sample Project Activity • Think of a current project or program that you would like to assess. • Review the Pre-Assessment Questions: • Why are we doing this assessment? • What will we assess? • How will we assess? • Who will assess? • How will the results be analyzed? • How will the results be communicated and to whom?

36. Remember . . . • Learning about and applying effective assessment practice is a process! • You will get some of it right and some of it wrong, and you will learn from your experiences • The point is to learn to use whatever you discover to improve your work and students’ experiences at NIU • Good assessment is: • MEANINGFUL • MANAGEABLE & • MEASURABLE

37. Learning Outcomes Review • Explain the role of assessment • Explain the 3 M’s of planning & assessment • List several tools that could be used to measure program learning objectives • Practice constructingmeaningful, measurable, & manageable student learning outcomes

38. Discussion & Questions

39. Resources & References • Barsch, A. and Simon, D., “Partners in Planning & Performance, Assessment and Student Learning Outcomes,” Student Affairs Cabinet Workshop, Northern Illinois University, January 31, 2005. • Bresciani, M. (2011). Making assessment meaningful: What new Student Affairs professionals and those new to assessment need to know. National Institute for Learning Outcomes Assessment. [Online]. Available from http://www.learningoutcomesassessment.org/documents/ABStudentAffairs.pdf • Bresciani, M.J. The first steps toward identifying student learning. Faculty Development & Instructional Design Center Workshop, Northern Illinois University, October 15, 2010. • Bresciani, M.J. (2002, December 9). Outcomes assessment in student affairs: Moving beyond satisfaction to student learning and development. Retrieved January 30, 2006 from National Association of Student Personnel Administrators, NetResults Website: http://www.naspa.org/membership/mem/nr/PrinterFriendlyu.cfm?id=884 • Clark, D. (2007), Learning Domains or Bloom’s Taxonomy. Retrieved November 19, 2007 from http://nwlink.com/~donclark/hrd/bloom.html • Council for Higher Education Accreditation (2007). CHEA HEA Update. Retrieved September 26, 2007 from Web site:http://www.chea.org/Government/HEAUpdate/CHEA_HEA40.html • Dictionary.com Unabridged (v 1.1). (n.d.). Retrieved February 16, 2007, from Dictionary.com website: http://dictionary.reference.com/browse/strategy • Learning Reconsidered (January 2004). Retrieved September 20, 2007 from http://www.cedu.niu.edu/cahe/Docs/ExtStandardsDocs/LearningReconsidered04.pdf • Spendolini, M. (1992). The Benchmarking Book. New York: AMACOM Books. Retrieved July 17, 2006 from http://www.netlibrary.com/Reader/strategy • U.S. Department of Education.  (2006).  A test of leadership: Charting the future of U.S. higher education.  Retrieved September 26, 2007 http://www.ed.gov/about/bdscomm/list/hiedfuture/reports/final-report.pdf • Upcraft, M.L., & Schuh, J.H. (1996). Assessment in student affairs: A guide for practitioners. San Francisco: Jossey-Bass Publishers.

40. Additional Resources • Assessment Pages at NIU Division of Student Affairs & Enrollment Management: http://www.stuaff.niu.edu/stuaff/PlanningAssessment/index.shtml • NIU Student Affairs Strategic Plan & Enrollment Management Strategic Plans http://www.stuaff.niu.edu/stuaff/PlanningAssessment/strategic%20planning/index.shtml • Toolkit: http://www.niu.edu/assessment/Toolkit/ • American Association of Higher Education Principles of Good Practice for Assessing Student Learning: http://web.uconn.edu/assessment/docs/resources/AAHE_Principles_of_Good_Practice.pdf • Internet Resources for Higher Education Outcomes Assessment, Compiled by North Carolina State University: http://www2.acs.ncsu.edu/UPA/assmt/resource.htm • Schuh, J. H. and associates (2009). Assessment methods for student affairs. San Francisco: Jossey-Bass Publishers.