EVAL 6000: Foundations of Evaluation

1. EVAL 6000: Foundations of Evaluation September 9, 2010

2. Agenda • Course overview • Course description • Course Website • Required textbooks and readings • Learning objectives • Course components and assessment methods • Succeeding in this course • Schedule • Introductions • Who are we? • Who are you? • Questions or concerns • Activity

3. Course Description • This course is designed to provide you an overview of the field and discipline of evaluation • Comparative study of theory, research, and practice perspectives • In-depth analysis of core concepts and definitions, rationale and uses, the field’s history and standards, alternative models and approaches, and emerging areas and enduring issues • Additional topics include personnel evaluation, performance evaluation, policy analysis, and product evaluation This is the only comparative evaluation theory course at WMU (and, virtually anywhere), so for those of you studying evaluation (please) take it seriously!

4. Course Website • The Website for this course is located athttp://www.wmich.edu/evalctr/idpe/courses/eval-6000-foundations-of-evaluation/ • From this site you can access • The course syllabus • List of required textbooks • List of required readings • Weekly lecture notes • Other materials related to the course Lecture notes will be posted on the course Website approximately 2-3 days prior to course meetings

5. Required Textbooks Alkin, M. C. (Ed.). (2004). Evaluation roots: Tracing theorists’ views and influences. Thousand Oaks, CA: Sage. Mathison, S. (2005). Encyclopedia of evaluation. Thousand Oaks, CA: Sage. Scriven, M. (1993). Hard-won lessons in program evaluation. New Directions in Program Evaluation, 58. San Francisco, CA: Jossey-Bass. Shadish, W. R., Cook, T. D., & Leviton, L. C. (1991). Foundations of program evaluation: Theories of practice. Thousand Oaks, CA: Sage. Start reading these books now!

6. Required Readings Too many to list! You should read all of the assigned articles carefully as they will sometimes form the basis for class discussions and activities You also may see questions about the articles in quizzes and exams Additional readings may be assigned (and, often are) Start reading articles now!

7. Primary Learning Objectives • A deep understanding of a wide array of evaluation theory and practice perspectives • An in-depth understanding of the origins and history of evaluation as well as its evolution toward an independent discipline and where it is headed—its past, present, and future • A clear understanding of key evaluation concepts, vocabulary, and terminology

8. Primary Learning Objectives • A clear understanding of the nature and purpose of evaluation, and the distinctions between evaluation and basic and applied research • An ability to describe, distinguish among, and critically evaluate the usefulness and validity of various evaluation approaches • A firm grasp of the fundamental logic and methodology of evaluation

9. Primary Learning Objectives An evaluative and critical thinking mindset, in general A basic understanding of how to integrate traditional research methodologies with evaluation-specific methodologies

10. Secondary Objectives • Throughout the course you also will work on improving your communication and interpersonal skills (even if you believe they are adequate or advanced) • Conveying constructive criticism in a professional, balanced, and tactful manner • Facilitating discussion to engage others in dialogue about evaluation theory, method, and practice • Writing clearly and concisely for both academic and non-academic audiences • Giving high quality, professional oral presentations for both academic and non-academic audiences

11. Course Components and Methods of Assessment Attendance & class participation = 10% Mini papers (5) = 15% Major papers & presentations (3) = 45% Midterm Examination = 15% Final Examination = 15% A = 100% - 95% BA = 94% - 90% B = 89% - 85% CB = 84% - 80% C = 79% - 75% F =< 75% Standards-based assessment is used in this course (coupled with traditional methods)

12. Succeeding in this Course This course is part of the core for the Interdisciplinary Ph.D. in Evaluation (and 1/3 of the Evaluation, Measurement, and Evaluation [EMR] program) and these students are expected to master the course’s content

13. Schedule • The general schedule of topics includes, but is not limited to: • Basic concepts and principles • Stage one theories • Stage two theories • Stage three theories • Methods-oriented approaches • Valuing-oriented approaches • Use-oriented approaches • Contemporary approaches

14. The hats shown in this photo are optional and not required for this course, but are available for purchase upon request My evaluation experiences have primarily focused on the evaluation of healthcare and educational programs Currently, my interests include the Program Evaluation Standards, metaevaluation, the evaluation of mentoring programs, and the application of statistical methods to evaluation practice I have conducted over 20 teaching and speaking engagements and have produced 8 scholarly works, several of which are pending publication in peer-reviewed journals I have conducted approximately 75 evaluations and have given presentations, speeches, workshops, or provided instruction in more than 10 countries My (current) interests are research and evaluation design, psychometrics and measurement, structural equation modeling, and meta-analysis I have published more than 75 peer-reviewed articles, book chapters, and other scholarly works

15. A few final notes regarding this course: Carl and I will not regurgitate the material from the assigned readings (they will, however, be discussed and will be covered on exams and other assignments) You are expected to THINK! You are expected to THINK! You are expected to THINK! You can disagree with us (or what you read in this course) at any time if (and, only if) you have completed #2, #3, and #4 Frightening alien-like images are not part of this course, though they will appear from time to time for my own (and your) amusement

16. Who are you? Why are you here (e.g., required)? What do you expect to learn? What prior experiences do you have with evaluation? Research? Design? Measurement? Statistics (or analysis in general)?

18. Individually, answer the following by writing your responses on the notecard provided (you have 10 minutes and should keep your responses clear and concise): What is evaluation? What is the general function or purpose of evaluation? How, or does, evaluation differ from basic or applied research? If so, or not, in what way(s)? Exercise #1