DEREK HERRMANN & RYAN SMITH Illinois State University University Assessment Services

2. IT TAKES TWO:A Systematic, Effective, & Practical Process for Integrating Assessment & Program Review DEREK HERRMANN & RYAN SMITH Illinois State University University Assessment Services Association for Institutional Research, New Orleans, LA, 6-5-2012

3. About ISU • DRU • Total enrollment: 21,080 • 35.5% of new students were transfer students • 82.7% of degrees granted were baccalaureate degrees

4. About ISU • Retention rate: 85.1% • Graduation rate: 69.8% • Student to Faculty ratio: 19:1 • Organizational structure • UAS is part of Academic Affairs • IR is part of Finance and Planning

5. About ISU • Illinois Board of Higher Education (IBHE) • All academic degree programs required to have assessment plans on file • All academic programs required to undergo a periodic and systematic program review

6. The Problem 1. Shifting views of higher education “Higher education is one of the greatest hopes for intellectual and civic progress in this country. Yet, it is seen by many as part of the problem rather than the solution.” - C. Palomba & T. Banta, Assessment Essentials, 1999

7. The Problem 2. Increasing Accountability Higher Learning Commission In the 11 pages of Criteria in the proposed pathways model, assessment is mentioned 19 times, with Teaching and Assessing for Learning as its own criteria. IBHE - C. Palomba & T. Banta, Assessment Essentials, 1999

8. The Problem 3. Changing World

9. The Problem 4. Assumptions about evaluation and assessment “Assessment projects are simply getting in the way of good teaching… We spend far too much time and energy in trying to assess learning, an unattainable goal….I love teaching, however. It is a great pleasure to share my enthusiasm for (my discipline) and I absolutely love the chance to pick the minds of my bright students.” - B. Moore, Washington State Board of Community & Technical Colleges, Using Assessment to Improve Instruction (video), http://www.youtube.com/watch?v=BZ3USs16J3Y

10. The Problem

11. How Colleges Are Organized

12. How They Are in Reality • Random People • Random People

13. Basic Assessment Model Education, training, & experience, life: health, spouses/partners, kids, family, friends, pets, etc. Accreditation, IBHE/ICCB, State of Illinois, Strategic plans, Tech. plans, enrollment mgmt. plans, facilities plans, financial plans, environmental scan data, IR data Governance, Budgets, Resources, Boards, Politics, Org. culture, good and bad bosses, etc.

14. Failure of Decision-making Mission Failure of decision-making or making it meaningful Process is the outcome

15. Advantages

16. The Problem How does one design processes that… • Engage faculty and staff? • Can be used for true improvement and change in today’s society? • Make information meaningful for decision-making? • Meets accountability standards? • Are genuine, not just transparent?

17. Disclaimer

18. At ISU • 8-Year cycle that includes • Process for the Review of Academic Assessment Plans (PRAAP) • Self-study • Program Review • Annual Update • Many across campus involved in these processes

19. At ISU • Year 1 – PRAAP • Chairs/program directors asked to provide current assessment plans • Assessment Advisory Council (AAC) reviews them using a rubric and completes a feedback form

20. The PRAAP Rubric

21. At ISU • Year 1 – PRAAP • Chairs asked to provide current assessment plans • Assessment Advisory Council (AAC) reviews them using a rubric and completes a feedback form • UAS staff meets with chairs/program directors to discuss the feedback

22. At ISU • Year 2 – Self-Study • Faculty and staff complete self-study for program review • Assessment plan is used to collect data

23. At ISU • Year 3 – Program Review • Academic Planning Committee examines Program Review documents • Often programs submit follow-up reports based on assessment plans

24. At ISU • Years 4-8 – Annual Update • Chairs asked to complete a short form concerning their assessment activities • Questions based on Higher Learning Commission’s (HLC) Fundamental Questions for Conversations Student Learning

25. The Annual Update Form • Please provide a brief summary of the previous year’s data which was collected related to your outcomes. • What efforts were used during the past year to involve faculty and constituencies in assessment processes?

26. The Annual Update Form • Please outline what action plans have been developed based upon the assessment process. • Please provide any additional comments or information. • Have changes or improvements been made to the Academic Assessment Plan in the previous year?

27. At ISU • Years 4-8 – Annual Update • If applicable, an updated assessment plan is sent to UAS • Collection of forms provided to programs to be used in the next cycle

28. Advantages 1. Assessment is a separate from program review, but still leads the process. • A culture of inquiry over a culture of evidence. • One is about performance, the other about learning.

29. Advantages A focus on what results mean, instead of what they are. - Jane Chapman, Heartland Community College, It’s Eleven O’Clock: Do You Know Where Your Students Are?, 15th Annual Comm. College Assessment Fair, Bloomington, IL, 2011

30. Advantages 2. Value people’s time, resources, energy, history, thoughts, and opinions. • Attitudes, beliefs, and engagement with assessment are just as important as processes. • Faculty have control through the self-study and review process. • Assessment & program review aren’t used to micro-manage or fix a problem that is not perceived to exist. • Ideas, people, and dialog lead – technologyfollows.

31. Advantages

32. Advantages 3. There is consensus and agreement about the process, and it is used to make decisions. - It’s more important to be genuine, rather than transparent. - Assessment and program review are never goals – they’re processes.

33. Continuous Improvement Nothing is ever perfect, but it can be healthy. Incremental changes to rubric.

34. Questions?Comments?

35. For More Information http://assessment.illinoisstate.edu/program/resources.shtml http://provost.illinoisstate.edu/academic/program-review.shtml