Using Program/Unit Review to Facilitate Change

1. Using Program/Unit Review to Facilitate Change SACS Annual Meeting December 6, 2004 Terri M. Manning, EdD Kathy Drumm, DBA, CPA Central Piedmont Community College Charlotte, North Carolina

2. Some history…. • CPCC is the largest community college in both the Carolinas with approximately 60,000 enrolled students. • We have over 100 instructional program areas including for-credit, continuing education and literacy • We have almost 50 administrative and student affairs units

3. Some history…. • When we started this process (1998), the College had done nothing truly “IE related” since the last SACS visit in 1992 when they received a recommendation for most must statements in section III. • The IR office staff had rolled over and all institutional memory was gone. • We had less than 5 years to put every program and unit through a review process.

4. Why this Process? • The evaluation of teaching and educational effectiveness is a top priority for most institutions • However, the evaluation of business, administrative and student services units is often undervalued and overlooked • Evaluating the effectiveness of services (not just customer satisfaction) makes good business sense

5. Institutional Effectiveness and the Southern Association of College and Schools • At the heart of SACS’ philosophy is the concept of institutional effectiveness (IE) • IE involves a process of planning, evaluation, assessment and use of results • Under the new core requirements and comprehensive standards, there are two “mandates” that specifically address program or unit reviews

6. Under the New Core Requirements and Comprehensive Standards Core Requirement 2.5 states “The institution engages in ongoing, integrated, and institution-wide research-based planning and evaluation processes that incorporate a systematic review of programs and services that (a) results in continuing improvement, and (b) demonstrates that the institution is effectively accomplishing its mission.” Comprehensive Standard 3.3.1 States “The institution identifies expected outcomes for its educational programs and its administrative and educational support services; assesses whether it achieves these outcomes; and provides evidence of improvement based on analysis of those results.”

7. IE at Central Piedmont • CPCC established an IE committee in 1998 and an IE website in 2000 (http://inside.cpcc.edu/IE) (this website gets an average of 61 hits a day – most outside the college) • Committee members were from various instructional departments, student services and administrative/business offices • The committee established an IE plan for the college

8. Visual of the IE Plan

9. Four Major Overlapping Pieces of the IE Plan • Annual goal setting and follow-up required by all units (Strategic Plan) • Program/unit review on a 3-5 year cycle • College assessment plan involving surveys, assessment and data analysis • The evaluation of general education

10. Developing the Process • We wanted to create a meaningful program/unit review process • We wanted programs to complete the process having learned something valuable (not a document to set on a shelf) • We wanted the process to be “outcome-based” to stand the test of time

11. Developing the Process • The first review process was created for instructional programs • During the 1998-99 year, 18 programs were reviewed (approximately 100 programs over five years) • The perception at the beginning was that “this is just another academic exercise” • But the results were very different than any review process previously completed

12. Organizational Stages of Outcome Evaluation Stage 5 Acceptance & adaptation Challenge & competition Catalyst - Proactive Stage 4 Depression - exhaustion Compliance - Passive reactive Stage 3 Bargaining - no time/no money Seek outside sources Stage 2 Anger and antagonism Resistant & Reactive Disbelief & Denial Paralysis - Passive resistance Stage 1

13. Instructional Program Review • The review process contained five sections: • I. The Program Profile • II. Program Content • III. Student Learning Outcomes • IV. Need for Change • V. Future Issues

14. Instructional Program Review • Tasks accomplished through review: • Defined their program • Detailed faculty and staff (credentials, prof. dev, accomplishments) • Unit mission or purpose • Link unit mission to the college mission • Defined the content of their program • Defined the population served • Set outcome objectives • Performed some means of assessment • Analyzed results • Determined strengths and weaknesses • Created strategies for improvement • Determined future needs (including curricular change, needed resources, staff and space)

15. Instructional Program Review • Rules for the process included: • Involvement on the part of all faculty in the department under review (not just one person). It is recommended that the program begin with a brief faculty retreat to discuss and divide tasks. • All programs must use at least one external committee (advisory groups are fine) to provide feedback to programs. • All programs must utilize feedback from students.

16. What We Learned…. • The first group was dragged kicking and screaming through the process • Faculty had very little time for the details • Faculty had trouble identifying “student learning outcomes” • The results produced great marketing materials • Once they saw the results, faculty embraced the process

17. Identifying Outcomes • We focused on two types of measures: • Outcomes • Program Outcomes • Student Learning Outcomes • Administrative Objectives

18. Administrative Objectives • Many units do not directly serve students or they want results within their units that are not truly outcomes. • They want to improve services or approach an old problem in a new way. • They want to become more efficient and effective. • They will set administrative objectives.

19. My Administrative Objectives 1. 80% of faculty/staff responding to the faculty/staff survey will perceive that Planning and Research responds quickly to their requests for data. 2. 80% of faculty/staff responding to the survey will perceive that Planning and Research makes a significant contribution to the College. 3. 80% of faculty/staff responding to the survey will perceive that Planning and Research contributes to the effectiveness of CPCC. 4. 80% of faculty/staff responding to the survey will indicate that Planning and Research produces enough reports to meet the planning and information needs of faculty and staff.

20. Outcomes • "Outcomes" are benefits for people: changes in knowledge, values, position, skills, behavior or status. More simply stated, outcomes are typically what service providers hope recipients achieve once they complete a program or receive services. This is not the “what” but the “why” of education. • Student learning outcomes are outcomes related to the learning that takes place in the classroom. We measure improvements in writing, speaking, understanding the scientific method, etc. • Program outcomes are the benefits to a student who receives an associate degree in Nursing or completes a certificate in Network Administration? Typical outcomes might be licensure exam scores, job placement, etc. • Outcome objectives are just objectives that relate to the identified outcomes.

21. Program Outcome Model INPUTS ACTIVITIES OUTPUTS ResourcesServicesProducts or Results of Activities Staff Education (classes) Numbers served Buildings Services FTE (input next year) Facilities Counseling # Classes taught State funds Student activities # Students recruited FTE # Who participated Constraints Laws State regulations

22. Program Outcomes Model INPUTS > > > Benefits for People *New knowledge *Increased skills *Changes in values *Modified behavior *Improved condition *Altered status *New opportunities ACTIVITIES OUTPUTS OUTCOMES

23. We Had to do Training on How to Set Objectives • There’s no magic number • e.g. 80% or 90% • What is reasonable? • What can you afford? • What realistically can your staff accomplish? • May need to benchmark (e.g. enrollment growth) • What percent shows you’re not committed and what percent shows you’re naïve?

24. How to Set Objectives • Examples: • Fifty percent of students will be able to communicate effectively in writing (complete the writing exam with a grade of 60 [D] or better) • By the end of the spring term, 95% of faculty and staff will have completed 20 contact hours of professional development (workshops, college courses, conferences, onsite trainings, etc.)

25. More Realistic • Seventy percent of students will be able to communicate effectively in writing (complete the writing exam with a grade of 75 [C+] or better) • By the end of the spring term, the professional development office will increase their offerings for faculty and staff by 10% over what was offered last year (workshops, college courses, conferences, onsite trainings, etc.)

26. What Happened • Deans used the results to make a case for resources • The administration became interested in what was learned through the review process • Instruction created a position to deal mainly with program review and IE issues within instruction • Faculty knew their programs were working

27. Unit Review – Lessons Learned • To do this well you need several critical pieces: 1. Support from the top (President or Chancellor, Vice Presidents or Vice Chancellors) 2. Buy in from the grassroots level (participation in the development of the process) 3. Across-the-college participation (no one is exempt) 4. Technology to make it easy (web page and review templates) 5. Technical support from institutional research

28. IE Committee Decision • The instructional program review process was successful • The review of instructional units was important – but administrative/ESS units helped created an environment conducive to learning at the college and supported the learning process • Administrative units should go through a similar process • Create a committee to draft a similar process for the administrative and students services areas of the college • Administrative units were spread across three VP areas • Two representative from each VP area participated in drafting the new review process • We spent approximately six months creating a workable process

29. Timeline for Administrative Units • During the: • 1999-2000 year – six units went through the review • 2000-2001 year – eight units went through the review • 2001-2002 year – nine units went through the review • Vice Presidents set an annual timeline for their units with due dates for each section

30. The Administrative Unit Review Design I. The Unit/Program Profile A. The Mission/Purpose 1. Role unit plays in the college mission 2. Unit/program goals as they relate to the college’s mission B. The Staff 1. Professional and administrative staff (since the last review) a. Position description/duties b. Credentials (full and part-time, if any) c. Accomplishments (if applicable) d. Service to college, community and nation

31. The Administrative Unit Review (continued) B. The Staff (continued) e. Professional development activities 2. Classified Staff a. List of names and positions b. List of required credentials (if any) C. The Customer/Client Served 1. Breakdown of students/faculty or staff by type or demographic information (thorough explanation of who is served)

32. The Administrative Unit Review (continued) II. Definition of Services or Program A. Definition of day-to-day duties of the unit B. Innovations, new projects, new initiatives, local, state-wide or national efforts C. Required functions of unit (description and status of compliance) 1. SACS "must" statements 2. State mandates 3. Federal mandates 4. Other

33. The Administrative Unit Review (continued) III. Administrative Objectives and Student Outcomes (where appropriate) A. Administrative Objectives (2-3 objectives) B. Outcomes (or status if incomplete) of innovations, new projects, new initiatives, local, state or national efforts C. Assessment explanation (what was assessed, who, when, how many) D. Results of Administrative Objectives Based on Assessment

34. Assessment of Administrative Units • During the year of program review, the “Annual Faculty/Staff Survey” contained questions from those units being reviewed. • Results were given to each unit and broken out by campus and job type http://inside.cpcc.edu/planning • There’s a link to survey results.

35. The Administrative Unit Review (continued) IV. Need for Change A. Strengths identified by external sources, faculty, staff and students B. Weaknesses identified by external sources, faculty, staff and students C. Recommendations by faculty, staff, external sources and students to improve the unit's services and programs D. Strategies for change (based on input from Sections A, B & C) - closing the loop E. A one-year follow-up brief report to the Unit VP reporting on the progress of D above (due at the end of the year following review)

36. The Administrative Unit Review (continued) V. Future Issues - Resources needed for future efforts A. Market trends within the broad service unit or program area (based on best- practices, the literature or training received) B. Anticipated future changes and needs (based on market trends) C. Resources, equipment, space, staffing and work load changes needs for future growth or continuation D. Future plans of unit

37. Assistance from the Website • The IE website: • http://inside.cpcc.edu/IE • Explanation of IE process • Templates and forms for review • “Perfect” examples for clarification • Schedule for review

42. Overall Benefits • #1No recommendations in the area of Institutional Effectiveness from SACS during the October 2002 re-accreditation visit • The college became change-oriented • Units had to define strategies for change • Didn’t have to be perfect but rather making continuous progress • Strategies for change helped identify needed resources for units • Units had to “close the loop” with the one-year follow-up (couldn’t promise and not deliver)

43. Overall Benefits • Units became empowered to perform their functions in an optimal manner and to ask for what they needed (no one noticed them before, now they do) • Created data to support needs • Accomplishments were reported to major administrative groups across the college (President’s Cabinet, Planning Council, etc.)

44. Overall Benefits • The college community understood the purpose and function of every unit • Senior administration realized the benefits and became strong supporters • Data are reported annually from program/unit review • VPs became stronger advocates for their units making changes to improve services

45. Overall Lessons Learned Through Program/Unit Review • The response rate to on-line surveys was double that of pencil-paper surveys • Faculty and staff did not know what many units did • Faculty and staff were unsure of many policies (how to obtain funds from the Foundation, etc.) • Email was the preferred method of communication for faculty and staff • Overall, faculty and staff were pleased with services • Wednesday afternoons were the best time to hold trainings

46. Overall Lessons Learned Through Program/Unit Review (cont.) • Faculty and staff had good ideas on how to improve services – if we just asked them • Use of P-Cards was saving the college time and money and faculty/staff preferred using them • Faculty and staff wanted to continue to receive paycheck receipts in the mail (all employees are direct deposit) • More than 80% of faculty and staff were using remote access to email • Faculty and staff wanted changes in the budgeting process – too complicated

47. Administrative Response • Units spend a lot of time working on the program review. • If we want them to take it seriously, we have to take it seriously. • Once reviews are reported at the end of the year, their Dean or Vice President/Chancellor need to: • 1. Read them • 2. Respond to them

48. From Personal Experience • Sit down with the unit (group meeting) • Give them my attention • Discuss what was learned • Discuss their major issues • Discuss what they need to make improvements • Actually attempt to direct resources to them to make those improvements • Then they do not see it as an academic exercise

49. How It Works for Us Administrative Units Reviewed in 2003 • Administrative Technology Services • Compliance & Audit, Inventory Control, Basic Skills Compliance & Reporting • Facilities Services • Information Technology Services • Institutional Advancement/Foundation • Planning and Research • Resource Development

50. Faculty/Staff Survey 2003 • During the Spring 2003, an on-line faculty-staff survey was developed, administered and analyzed. • A total of 222 faculty and staff completed the survey with the following numbers by type: