Plotting a New Course for Evaluation of Service Departments

1. Plotting a New Course for Evaluation of Service Departments Alan Vladicka Jodene Baccus Strategic Planning, Grant MacEwan College MacEwan Strategic Planning

2. Plotting our Course MacEwan Strategic Planning

3. The MacEwan Context • Largest college in Alberta (10,500 FTE) • Well-established process for program evaluation, but until last year, no systematic approach to evaluating service areas • First college to receive approval to grant undergraduate degrees under Alberta’s new accreditation model (Campus Alberta Quality Council) MacEwan Strategic Planning

4. Service Evaluation – Why Now? • Perceived imbalance between rigorous program evaluations & infrequent, ad hoc service evaluations – political issue with program chairs & faculty • Strategic focus on enhancing quality student & college services required a more systematic approach to service evaluation MacEwan Strategic Planning

5. Service Evaluation – Why Now? • Institutional “readiness” assessment by Quality Council included recommendation for regular service evaluations • Accreditation review in 5 years (based on degree program approvals) will include assessment of progress on this recommendation MacEwan Strategic Planning

6. Guiding Principles • Parallel program evaluation process & policy as appropriate • Base evaluation on clearly articulated and approved goals and performance measures for each service area (not just a “happiness survey”) • Involve clients in planning, oversight & follow-up • Ensure follow-up, action planning and accountability MacEwan Strategic Planning

7. Rigging the Vessel MacEwan Strategic Planning

8. The Evaluation Framework • Balanced Scorecard methodology was chosen as the basis for articulation of departmental goals and measures • Places evaluation in the broader context of strategic planning for the area • First cycle is a two-year process: • Articulation of department’s strategic goals and scorecard measures • Evaluation based on scorecard MacEwan Strategic Planning

9. Evaluation Components • Evaluation Committee (inc. client reps) guides the process • Input from clients & staff based on key measures • Self-study, including benchmarking with appropriate comparators • Follow-up & action planning • Accountability loop to Services Review Committee MacEwan Strategic Planning

10. Our Maiden Voyage MacEwan Strategic Planning

11. Student Resource Centre • Evaluation of the Student Resource Centre (SRC) at MacEwan • SRC Strategic Plan • Mission and Vision • Six Strategic Goals • Success Indicators MacEwan Strategic Planning

12. Evaluation Methodology • Sources of data: • Current students • Current users of the SRC • SRC staff • Other faculty/staff who refer students to the SRC MacEwan Strategic Planning

13. Evaluation Methodology • Data collection method: • Current students – online questionnaire • Current users of the SRC – on-site questionnaire • SRC staff – online questionnaire • Other faculty/staff who refer students to the SRC – online questionnaire

14. Evaluation Methodology • Information collected • Frequency of use • Information sources • Satisfaction with services and staff • Efficiency of service delivery • Strengths and areas for improvement • Evaluation of SRC resources, responsiveness, and the working environment

15. Timeline: September-October, 2006 • Meet with SRC co-chairs • Finalize methodology and instruments

16. Timeline: November – December, 2006 • Orientation meeting with SRC staff • Online survey for SRC staff administered • First wave of on-site survey for users of the SRC (3-week period)

17. Timeline: January – February, 2007 • Online survey for general student body administered • Second wave of on-site survey for users of the SRC (3-week period) • Online survey of non-SRC faculty/staff (“referrers”) administered.

18. Timeline: March, 2007 - present • Data analysis and report preparation • Report distribution (July, 2007) • September, 2007 – Follow-up meeting with Service Evaluation Committee to develop action plan. • November, 2007 – scheduled presentation of findings to SRC staff.

19. Balanced Scorecard Collaborative process Breadth of data Self-study Service Evaluation Committee Benchmarking Timeline Lessons Learned Strengths Areas for Improvement

20. Sailing Onward

21. Goals for the Future • Intent is to build a culture and habit of self-assessment & quality improvement based on evidence • Scorecard framework & measures are designed to provide the basis for ongoing self-study & external benchmarking • Departments will be challenged to compile longitudinal performance data based on scorecard

22. Goals for the Future • 2nd and subsequent rounds of evaluation (5-7 year cycle) will be more focused on area self-studies, supported by “arms-length” collection of client feedback by Strategic Planning • Development of customized service department “report cards”, based on key scorecard measures • Annual “mini-evaluation” & action planning to entrench a quality improvement culture across MacEwan

23. Questions?

24. Alan Vladicka Executive Director Strategic Planning Grant MacEwan College 780-497-5391 vladickaa@macewan.ca Jodene Baccus, PhD Research Analyst Strategic Planning Grant MacEwan College 780-633-3638 baccusj@macewan.ca For further information MacEwan’s Service Evaluation Policy: (D9000): http://www.macewan.ca/web/services/board_gov/about/DetailsPage.cfm?id=1036