Incorporating Assessment into Program Review

1. Incorporating Assessment into Program Review Dr. Maura Stevenson Ms. Mary Kate Quinlan Community College of Allegheny County Pittsburgh, PA

2. Session Objectives Participants will be able to… • Identify the steps needed to incorporate assessment into a community college program review process • Create a plan to implement program assessment • Determine the effectiveness of our process based on evidence presented

3. CCAC – Facts and Figures • 30,773 credit students • Multi-campus: 4 campuses & 7 centers • About 86 programs • Unionized faculty • Governance process

4. Assessment at CCAC • Assessment of Student Learning Committee (ASL) • Faculty led • Support and participation by Administrators • 5 - Faculty liaisons • One for each campus • One for student services • Assessment focus • Year 1 – classroom • Year 2 – program & general education

6. Embed Assessment into Existing Processes

7. Why We Chose Program Review • It’s required (annual and 5-year reviews) • New Academic Affairs leadership valued the process • Assessment was actually a requirement (but widely ignored) • Credibility – approved through governance (supported by union) • Process in existence for 12 years (4th iteration)

8. Key Steps • ASL recommended changes to the program review process to strengthen the assessment components: • Annual - program assessment • 5-year - General education goal alignment

9. Key Steps • ASL forwarded proposal to Academic Planning subcommittee • The initiator distributes the proposal to the entire college community • The subcommittee conducts a hearing and may recommend changes before deciding to forward the document to the full council

10. Key Steps • College Council reviewed proposal and made the decision to forward to the president for approval • The president approved the proposal

11. CCAC’s Experience • The process went very quickly (about 7 weeks) • College was under pressure by accrediting agency to conduct assessment at the program level • Full ASL committee support and presence at the hearing • Few people questioned the changes

12. Create a Process That’s Beneficial to Participants

13. Our Approach • ASL is a resource to faculty conducting program assessment • Develop tools • Train the trainers – deans and department heads • Answer questions • The academic deans maintained authority

14. 3 Step Process • Pick one program objective • Pick two measures (one direct and one indirect) • Discuss the results • Based on Barbara Walvoord’s “no-frills assessment plans”

15. Timeline • Fall semester (week prior to start of class) – faculty meet to develop program objectives and develop assessment plan • Submit plans by September 15th • Fall semester – faculty conduct assessment • Spring semester (week prior to start of class) – faculty discuss assessment results • End of spring semester – faculty submit a program assessment report on the use of results

16. Benefits of This Method • Authentic – utilizes what’s happening in the classroom (not an add on) • Flexible – faculty choose what, who, and when • Yields useful results • Faculty worked together across campuses • Focused on common values

17. The Process (in detail)

18. 1. Start with Goals • Our Hierarchy • General education goals • Program objectives • Course outcomes • Handout • Worksheet – writing program objectives

19. 2. Create a Plan • Handouts • A Guide to Program Assessment for Faculty • Definitions • Examples • Resources • Sample Plans • Good and bad fictional examples • Tips for writing a good plan

20. 3. Defining Expectations • Handout • Rubric for program assessment plans Benefits: • Clarifies expectations for participants • Reduces subjectivity when evaluating plans • Issue: inter-rater reliability

21. 4. Keep People on Track • Handout • Checklist Benefits • A gentle reminder of required tasks • Helps the deans manage the process • Another opportunity for ASL faculty liaisons to make contact with instructors on their campus • Sends a message “assessment isn’t going away”

22. 5. Collecting Reports • Handouts • Program Assessment Report Template • Sample Report

23. Our Results (so far): Rubric Scores • Received 59 plans • No plans from13 programs without FT faculty (15%) • 14 unaccounted for (16%) • Average score: 14 out of 15 • Areas of strength – 1st and 2nd measures (3.0) • Area of weakness – statement of method (2.62)

24. Lessons Learned • Rubric – practice scoring to clarify understanding • Clarify liaison role • Discovered programs without FT faculty • Forgot to require assessment criteria – did not ask them to establish an acceptable level of performance • Current debate – should we have required writing 5 program objectives at once rather than one at a time?

25. Contact Maura Stevenson mstevenson@ccac.edu Mary Kate Quinlan mquinlan@ccac.edu Our website www.ccac.edu/assessment