The FUS Faculty Evaluation System

1. The FUS Faculty Evaluation System IDEA Workshop Cleveland Ohio October, 2008

2. The Purposes of the FUS Faculty Evaluation System Provide an opportunity for self-reflection, self-direction, and practical reasoning in departments for both faculty and program improvement Provide feedback from Chair and VPAA for faculty development Provide evidence of distinction in performance areas – for merit award decisions Provide comparative data for enquiry seeking standards of internal practices of FUS faculty

3. FUS a Catholic and Franciscan University is: Faithfully Catholic and purposefully seeking to operate from the Heart of the Church Comprehensive; Associates, Bachelors, and Masters liberal arts University located in Steubenville At about 2400 students and 40+ programs Has a structure with a strong VPAA and rotating Department Chairs as well as Graduate Program Directors who are teaching and researching while managing 2 Decade User of IDEA

4. Since 2002 at FUS: Fr Terrence Henry was named as President Dr. Max Bonilla took the position as acting VPAA and then in 2004 was named VPAA Completed a NCA/HLC PEAQ Team visit resulting a progress report on Assessment due in Fall 2009 Saw Dr. Bonilia commit to the faculty in 2004 that he would seek appropriate ways to widen the evidence used in Faculty Evaluation at FUS New CORE Program Development starts in 2007 A revised set of forms for faculty evaluation was used, starting in 2007, which asks for more quantitative evidence and allows participation in The University of Delaware Study (UDELII)

5. Practical Reasoning • Franciscan University, in its very Mission, as a part of its self understanding as a community, is committed to a wide conception of rationality with a rich understanding of practical reason. • Department Chairs and Faculty have been individually and collectively encouraged to consider the connections between Department Planning (Program Review), Assessment of Student Learning data and evidence, and individual Faculty Performance Evaluation - especially in setting future goals • It is through the qualitative narrative in these documents that Chairs and faculty communicate the situational contexts in which their performance efforts are intentionally made. The arguments made for funding, curriculum changes, faculty development or other items needing approval are made in these reports. The argument is part of our praxis, practical reasoning • This practical reasoning of the faculty and chairs closest to the Internal Goods (in MacIntyre’s Sense) of the practices of Teaching/Learning, Scholarship and Service

6. IDEA is Great - but not for Measures and Evidence of : Instructional Design & Planning Instructional Delivery & Innovation Feedback Given to students on work Advising, thesis mentoring and other instructional departmental activity valued at FUS and your Universities Scholarship, Internal Service or External service

7. Evaluating Teaching and Learning is hard, but Scholarship and Service? The University of Delaware Outside the Classroom Productivity Study (UDELII) Measures and definitions Operationally defining, and therefore understanding better many of the tasks, projects and processes which make up the practices of Teaching, Scholarship and Service at Universities The study aims at the widest generality and most exhaustive comprehension of measures prudent Comparative data from other institutions can inform the practical reasoning demonstrated in evaluations and reports

8. Review, Consider and Comment on FUS Reports • Participants at tables review sample Analysis Reports

9. The following IDEA analysis is like one given to each faculty member each evaluation at FUS Note: Average among courses for evaluation Comparative Z scores Discipline vs. Overall T score The following UDEL analysis gives data by CIP CODE from participating institutions These are not intended and cannot be used through simple comparisons as norms These can inform prudential argumentation about contribution These can inform the dialogue about standards of practice IDEA and UDEL Reports at FUS

10. Quantity is not Quality But quantity is one dimension or aspect of quality - just ask the working parent who tries intensely to have “Quality Time” Faculty at FUS are encouraged to put the quantitative aspects (UDEL) of faculty efforts in dialogue and in context through their narrative SoTL like projects and funded projects are asked for in quantitative sheets These are Funded through mini-grants from Assessment Office They are accompanied by AQIP like action planning (NCA/HLC alternative accreditation path) These then become exemplary projects for assessment / accreditation and faculty development They are saved on the ASPIRE website at FUS and become artifacts in our Narrative of Practice at FUS Triangulation and Practice

11. On Going Faculty Formation and Input Faculty may (and have) added items to the quantitative worksheet Faculty are encouraged and paid to engage in SoTL activities and advance their own development in Scholarly Teaching as well as Scholarship Proper (including SoTL) The assessment office helps with project preparation and seeks approval from VPAA as well as aids in getting SoTL work published or presented. Assessment office helps with the creation of the “Flight Plan” which is a larger Narrative or plan for Teaching Scholarship and Service

12. Appraisal and Action The appraisal for Merit (or probation) level award is made by the VPAA in consultation with the Chair after reviewing evidence and commentary from the faculty narrative, the chair evaluation, and an analysis from a central office reader and preparer of the materials. The consistency of merit recommendations between the preparer and the VPAA is calculated via inter-rater reliability. Last year’s Kappa Value was at at 0.76 or “substantially similar” The Faculty member and chair are encouraged to create individual action plans and longer range “Flight Plans” The VPAA sends suggestions for goals and actions with his feedback to each faculty annually The assessment officer aids faculty in achieving plans and goals

13. IDEA and UDEL help in achieving some level of objectivity in measures but… …by definition the personal judgment used in making a performance appraisal is subjective

14. SOFIA: A Continuous Improvement Process Seeking-Searching for Standards of our Practices at FUS IDEA, UDEL, Faculty Standards Committee at FUS, Best Practices from Peer Universities etc. Ongoing- Formation Feedback & Input - from Faculty through the assessment office to aid in creating excellent practices at FUS and when possible to create public SoTL projects Appraisal (summative) Annual or other (Promotion Tenure) and Action Planning SOFIA is also evolving and ongoing as we assess our appraisal system and its value to us at FUS

15. Like others, we at FUS have found stages of faculty resistance* Stage 1: Disdainful Denial. Stage 2: Hostile Resistance. Stage 3: Apparent Acquiescence. Stage 4: Attempts to Scuttle. Stage 5: Grudging Acceptance. After two (or more- we will see) years of operation, the faculty find that the system can actually be of some value. This is as good as it gets! There is no subsequent stage where faculty are happy with the system. *Taken and adapted from: http://ctl.stanford.edu/Tomprof/postings/610.html 7/7/08

16. Conclusion • Thank you for your attention. • Any Questions?