Identifying Your Role in Outcomes-Based Assessment Program Review

1. Identifying Your Role in Outcomes-Based Assessment Program Review Marilee J. Bresciani, Ph.D. Professor, Postsecondary Education and Co-Director of the Center for Educational Leadership, Innovation, and Policy San Diego State University 3590 Camino Del Rio North San Diego, California, U.S.A. 619-594-8318 Marilee.Bresciani@mail.sdsu.edu

2. Presentation Overview • Overview of Outcomes-Based Assessment (OBA) Program Review • Importance of OBAPR • Importance of Assessing Student Learning • Astin I-E-O • Questions to Consider • Questions Bresciani, M.J.

3. Ask Yourself These Questions • What decision did you make about your program last year? • What evidence did you use to inform that decision? • What was it that you were trying to influence about your program when making that decision with the stated evidence? Bresciani, M.J.

4. That is Outcomes-Based Assessment (Bresciani, 2006) • Most people do capitalize on their innate intellectually curiosity to find out what works • Most people just don’t articulate their intended end results (e.g., outcomes) ahead of time • Most people don’t document the decisions made based on their results • Most people don’t follow up later to see if their decisions made the intended improvement Bresciani, M.J.

5. The Assessment Cycle(Bresciani, 2006) • The key questions… • What are we trying to do and why? or • What is my program supposed to accomplish? or • What do I want students to be able to do and/or know as a result of my course/workshop/orientation/program? • How well are we doing it? • How do we know? • How do we use the information to improve or celebrate successes? • Do the improvements we make contribute to our intended end results? Bresciani, M.J.

6. The IterativeSystematicOBPR Cycle(Bresciani, M.J.) Gather Data Interpret Evidence Mission/Purposes Goals Outcomes -Values influence every aspect of this cycle. Implement Methods to Deliver Outcomes (Action Planning) and Methods to Gather Data Document decisions to improve programs; enhance student learning and development; inform institutional decision- making, planning, budgeting, policy, public accountability Strategic Planning/ Inputs/Capacity External Review Bresciani, M.J.

7. The Purpose • Outcomes-Based assessment does not exist for assessment’s sake • It is taking what most of us already do, and making it systematic • Its purpose is to reflect on the end result of doing - - how well are we accomplishing that which we say we are? Bresciani, M.J.

8. The Purpose, Cont. • Strategic and Action Planning are incorporated into it • It is intended to inform decisions for improvement and resource re-allocation • It helps you link what you do to institutional strategic initiatives and performance indicators Bresciani, M.J.

9. The Purpose, Cont. • Makes the purpose of our programs and services clear to students, faculty, parents and other constituents • Promotes student responsibility for learning • Promotes opportunities for collaboration • Outcomes-Based Assessment is not research Bresciani, M.J.

10. Importance of Assessing Student Learning • Demonstrates contributions to institutional mission and goals • And contributions to institutional priorities • Assists in informing prioritization of your time as well as other resources

11. Outcomes-Based Assessment can be Implemented at Multiple Levels Decisions are made at multiple levels - some decisions reside only at one level Bresciani, M.J.

12. Given what OBAPR is, how do you perceive your role in the process? Bresciani, M.J.

13. Why the increased emphasis in OBAPR? Bresciani, M.J.

14. Drivers of Outcomes-Based Assessment (Ewell, 2003) • To improve the underperforming student • Competency Movement in Business and Industry • International Trade Agreements • Bologna Declaration of 1999 Bresciani, M.J.

15. Drivers, Cont. • Government Conversation, 1985 • The Higher Education Re-authorization Act Testimonies in USA, 2002 and 2006 • Response to NCLB Legislation • Regional Accreditation – flexibility • CRAC – 2003, 2004 • Both documents focus on student learning Bresciani, M.J.

16. Drivers, Cont. National Commission on the Future of Higher Education • Demand for Public Information about Performance • Transparency of outcomes and results • Comparable measures of quality • Demonstration of value-added of the entire educational experience Bresciani, M.J.

17. Drivers, Cont. • Accountability requirements handed back to states • Performance indicators • Discipline Standards could be designed by disciplines • AACU Essential Outcomes/Degree Frameworks and Expected Levels of Learning • Economic State of the Country Bresciani, M.J.

18. Astin’s I-E-O • Inputs - Pre-Assessment • Environment –Formative Assessment • Outcomes – Summative Assessment Bresciani, M.J.

19. The Focus on Assessing Student Learning “The concepts of learning, personal development, and student development are inextricably intertwined and inseparable.” – The Student Learning Imperative

20. Some Questions about Student Learning and Development(Bresciani, Moore Gardner, & Hickmott, 2009) • What do you expect your students to know and be able to do by the end of their education at your institution? And how is your program designed to contribute to that expected learning? • What do you do in your programs to promote the kinds of learning and development that your institution seeks? Bresciani, M.J.

21. Some More Questions (Bresciani, Moore Gardner, & Hickmott, 2009) • How do I influence student learning? • Is it through the education of my colleagues? • Of faculty? • Of parents? • Of community participants? Bresciani, M.J.

22. Reflection Questions (Bresciani, Moore Gardner, & Hickmott, 2009) • How are you directly or indirectly contributing to student learning? • How are you directly or indirectly supporting student learning? • How are you directly or indirectly interfering with student learning? Bresciani, M.J.

23. The Ideal for Student Learning Establish collaborations between academic and student affairs to • facilitate student learning • facilitate student engagement and socio-academic integration • evaluate professional development for faculty and staff about effective learning environments Bresciani, Zelna, & Anderson, 2004 Bresciani, M.J.

24. Now, how do you perceive your role in OBAPR?

25. Ideas for Your Involvement • Pre-assessment design • Cognitive • Developmental • Testing the efficacy of placement scores • Incorporating the developmental aspects into design of placement • Working with advisors to see how placement scores are used Bresciani, M.J.

26. Ideas, Cont. • Working with Academic and Student Affairs to evaluate the efficacy of all assessment instruments and determine what other measures may be utilized to evaluate student learning and development. Bresciani, M.J.

27. Ideas, Cont. • Determining which comparable measures of learning are appropriate for your organization to use • Assisting with the development of new comparable measures of learning and testing their efficacy • Rubrics Bresciani, M.J.

28. What Else? Bresciani, M.J.

29. Questions to Consider • What is your vision for your role in OBAPR? • With whom do you need to communicate your vision? • How might you need to re-organize your office to meet your emerging role in OBAPR? Bresciani, M.J.

30. Questions, Cont. • What resources do you need in order to carry out your vision? • Who do you need to collaborate with within your • Institution • State • Nation? Bresciani, M.J.

31. Questions, Cont. In order to make your vision a reality, • What do you need to do immediately when you return to your campus? • What do you need to do three months from now? • What about a year from now? Bresciani, M.J.

32. Resources • Each Other • NASPA, ACPA, and AACU • University Planning and Analysis (UPA) Assessment website • http://www2.acs.ncsu.edu/UPA/assmt/ Bresciani, M.J.

33. Questions?

34. One Minute Evaluation • What is the most valuable lesson that you learned from this workshop? • What is one question that you still have? • What do you think is the next step that your division/program needs to take in order to implement systematic program assessment? Bresciani, M.J.

35. References • Astin, A. W. (1993). What matters in college: Four critical years revisited. San Francisco: Jossey-Bass. • Bresciani, M.J. (September, 2002). The relationship between outcomes, measurement. and decisions for continuous improvement. National Association for Student Personnel Administrators, Inc NetResults E-Zine. http://www.naspa.org/netresults/index.cfm • Bresciani, M.J., Zelna, C.L., and Anderson, J.A. (2004). Techniques for Assessing Student Learning and Development in Academic and Student Support Services. Washington D.C.:NASPA.

36. References, Cont. • Bresciani, MJ.(2006). Outcomes-Based Undergraduate Academic Program Review: A Compilation of Institutional Good Practices. Sterling, VA: Stylus Publishing. • Bresciani, M. J., Gardner, M. M., & Hickmott, J. (2010). Demonstrating student success in student affairs. Sterling, VA: Stylus Publishing. • Palomba, C.A. and Banta, T.W. (1999). Assessment essentials: Planning, implementing and improving assessment in Higher Education. San Francisco: Jossey-Bass.