C e l t
Download
1 / 27

C.E.L.T - PowerPoint PPT Presentation


  • 80 Views
  • Uploaded on

C.E.L.T. Developing a Comprehensive Faculty Evaluation System. Jeff Schlicht, Ph.D. Associate Professor Department of Health Promotion and Exercise Sciences schlichtj@wcsu.edu. Measurement vs. Evaluation.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'C.E.L.T' - quanda


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
C e l t

C.E.L.T

Developing a Comprehensive Faculty Evaluation System

Jeff Schlicht, Ph.D.

Associate Professor

Department of Health Promotion and Exercise Sciences

schlichtj@wcsu.edu


Measurement vs evaluation
Measurement vs. Evaluation

Measurement… is the process of systematically assigning numbers to the individual members of a set of objects or persons for the purpose of indicating differences among them in the degree to which they possess the characteristics being measured.

Robert L. Ebel, “Measuring Educational Achievement”, 1965


Measurement vs evaluation1
Measurement vs. Evaluation

Evaluation is the process of interpreting a measurement by comparing it to a value (or set of values) that represent an agreed upon standard. This process involves judgment.


A comprehensive system measurement and evaluation
A Comprehensive System = Measurement and Evaluation

In order to develop a comprehensive faculty evaluation system, we must:

1) Develop technically good (valid & reliable) measurement instruments

2) Agree upon a university standard for desirable faculty performance


How do we currently measure faculty teaching performance
How Do We Currently Measure Faculty Teaching Performance?

In the HPX department we use:

1) Student rating forms

2) Peer observation (in classroom)

Are either of these forms of measurement valid and/or reliable?

The answer is 1) we don’t know and 2) no


Student ratings of faculty performance
Student Ratings of Faculty Performance

The development of a valid and reliable questionnaire, i.e. a student rating form, is a technical process that has a body of science behind it.

Our department does not know who created the current rating form, and whether it has ever been tested for validity and reliability.


Student ratings of faculty performance dos and don ts
Student Ratings of Faculty PerformanceDos and Don’ts

DO include a balanced response scale

  • equal number of positive and negative responses

  • mirror-image opposites

    • Strongly agree agree disagree strongly disagree

      DO develop items (questions) important to faculty AND students

      DO conduct field tests of the items ( = collect data) to do reliability and validity analyses

      DO develop norms (a year’s worth of data) to allow fair and appropriate interpretation (i.e. comparison)

      DO consider using a professionally designed student rating form


Student ratings of faculty performance dos and don ts1
Student Ratings of Faculty PerformanceDos and Don’ts

DON’T

  • Have two items in one question (double-barreled response item)

  • Simply hand back a summary sheet of student ratings to instructor – include personal consultation to foster instructional improvement

  • Consider individual student responses in faculty evaluation


Peer observation
Peer Observation

The short answer: DON’T do it for personnel decisions

(can be useful for faculty enrichment)

The addition of another faculty member or administrator into the classroom ALWAYS alters the classroom dynamic in such a way as to provide an abnormal view of what usually happens there

THOSE DATA ARE NEVER VALID


Peer observation1
Peer Observation

If you must do it for faculty evaluation consider:

  • multiple visits by multiple (4) peers

    • 8 visits = adequate sampling

    • develop valid, reliable checklist

      • train observer team in use

    • prepare students, instructor

    • schedule post-observation conference

  • videography


8 steps to a comprehensive faculty evaluation system
8 Steps to a Comprehensive Faculty Evaluation System

Step 1: Define the faculty role model

  • What should be evaluated?

    • Start at the department level

      • Have faculty write down everything they do at WCSU to create a master list of responsibilities

      • Determine faculty roles

        • What are the things we are accountable for?

      • Assign the items from the master list of faculty responsibilities to the various faculty roles


8 steps to a comprehensive faculty evaluation system1
8 Steps to a Comprehensive Faculty Evaluation System

Step 2: Determine faculty role model parameters on a University (system?) level

  • How important should each role be?

    • Static vs. dynamic role model

      • Static = fixed weights

      • Dynamic = range of weights

        This is missing from our current system, and therefore we do not control subjectivity as well as we could


8 steps to a comprehensive faculty evaluation system2
8 Steps to a Comprehensive Faculty Evaluation System

Step 2: Determine faculty role model parameters on a University (system?) level

  • Static Role Model

    • Load Credit = 50%

    • Creative Activity = 25%

    • Productive Service = 15%

    • Professional Service = 10%


8 steps to a comprehensive faculty evaluation system3
8 Steps to a Comprehensive Faculty Evaluation System

Step 2: Determine faculty role model parameters on a school, then University (system?), level

  • Dynamic Role Model

    Minimum Maximum

    • Load Credit 35% 75%

    • Creative Activity 20% 50%

    • Productive Service 5% 45%

    • Professional Service 0% 40%


8 steps to a comprehensive faculty evaluation system4
8 Steps to a Comprehensive Faculty Evaluation System

Step 2: Determine faculty role model parameters on University (system?) level

  • Dynamic Role Model

    • The advantage of a dynamic role model is that it allows various levels of the University to play to their strengths

      • Allow each department to set dynamic range values, then select extreme max. and min. as University standards


8 steps to a comprehensive faculty evaluation system5
8 Steps to a Comprehensive Faculty Evaluation System

Step 3: Use observable or otherwise documentable achievements/products/performances to define all the roles in the faculty role model

  • Teaching (student interactions that promote learning)

    • Content appropriateness/currency

    • Instructional design skills

    • Instructional delivery skills

    • Instructional assessment skills

    • Course management skills


8 steps to a comprehensive faculty evaluation system6
8 Steps to a Comprehensive Faculty Evaluation System

Step 4: Define weights for each component of a single role – departments should be allowed to have their own weights, as long as they fit into the University structure. This embeds department values into the evaluation design:

  • Teachingmin max

    • Content appropriateness/currency 10% 40%

    • Instructional design skills 10% 30%

    • Instructional delivery skills 20% 60%

    • Instructional assessment skills 20% 50%

    • Course management skills 0% 30%


8 steps to a comprehensive faculty evaluation system7
8 Steps to a Comprehensive Faculty Evaluation System

Step 5: Identify where you will collect the data to evaluate the roles

  • Use sources that have first hand experience with the performance in question – preferably multiple sources of input

  • Teaching

    • Content appropriateness/currency

      • syllabus review by peer

      • Self-review

    • Instructional delivery skills

      • Student evaluation

      • Peer evaluation???

      • Self-review

    • Course management skills

      • Chair or administrative assistant evaluation

      • Self-review


8 steps to a comprehensive faculty evaluation system8
8 Steps to a Comprehensive Faculty Evaluation System

Step 6: Define weighting for information provided by different sources

  • Teaching min max

    • Students 15% 50%

    • Self 15% 40%

    • Peer 15% 30%

    • Chair 15% 30%


8 steps to a comprehensive faculty evaluation system9
8 Steps to a Comprehensive Faculty Evaluation System

Step 7: Determine how to gather information

  • Questionnaire

  • Interview

  • Checklist


8 steps to a comprehensive faculty evaluation system10
8 Steps to a Comprehensive Faculty Evaluation System

Step 8: Design data collection instruments


Evaluation for promotion and tenure
Evaluation for Promotion and Tenure

Evaluation is the process of interpreting a measurement by comparing it to a value (or set of values) that represent an agreed upon standard. This process involves judgment.

In a system like the one described today, evaluation is an on-going process involving multiple sources and evaluators. We are constantly comparing ourselves against the WCSU Faculty Role Model, so we have immediate feedback about our performance.


Evaluation for promotion and tenure1
Evaluation for Promotion and Tenure

Promotion and tenure decisions can be based on comparing our performance against a pre-determined, minimum required composite Faculty Role Model score. In other words, prior to becoming eligible for advancement, we would already know the outcome.


Comprehensive faculty evaluation
Comprehensive Faculty Evaluation

Practical steps we can take to improve the system:

  • Initiate a discussion about what it means to be a faculty member at WCSU in the 21st century

  • Discover new ways to measure faculty performance, tests those ways to make sure they are valid and reliable, and create a feedback loop that uses that information to enhance faculty performance

  • Lobby the union and the University to more actively support faculty enrichment


Final thoughts about faculty evaluation
Final Thoughts About Faculty Evaluation

Why should we do it?

  • Promotion and tenure is important, but…

    the true value of assessment lies in its ability to direct faculty enrichment. If evaluation is not tied to enhancing faculty performance, it will be perceived as punitive, a hurdle, rather than as a path toward self-improvement.


Acknowledgements
Acknowledgements

These materials are summarized from the ideas of Raoul A. Arreola, University of Tennessee Health Science Center, presented at the Center for Educational Development and Assessment professional enrichment seminar in Orlando, FL. March 10-11, 2008, and available in the text:

“Developing a Comprehensive Faculty Evaluation System,”

3rd edition, Arreola RA, Anker Publishing Co., Bolton, MA 2007


C e l t1

C.E.L.T

Developing a Comprehensive Faculty Evaluation System

Jeff Schlicht, Ph.D.

Associate Professor

Department of Health Promotion and Exercise Sciences

schlichtj@wcsu.edu


ad