Program Evaluation for Accredited Programs at MCC: Overview and Strategy

1. Program Evaluation for Accredited Programs at MCC: Overview and Strategy Presented at Training Session of Program Evaluation Leaders September 2014 Michael A. Heel Assistant Director of Curriculum and Assessment Monroe Community College

2. Background • The College Assessment and Program Evaluation (CAPE) Committee developed a protocol for the assessment of accredited programs at the College in 2014, to ease the burden on these program faculties for assessment • These guidelines can be fulfilled once, and then “re-checked” every cycle, further easing the workload • Important that the work is done well, since all other aspects of MCC’s program evaluation guidelines are excluded from project.

3. Our Task Together • Conduct a full and complete program evaluation that makes sense for your respective program • Plan accordingly, so you can finish on time • Invite the Coordinator of Academic Assessment into the process • I can help make your process more efficient • You will need assistance in troubleshooting obstacles in the process • I can help you better understand institutional standards and expections

4. GETTING STARTED

5. What Makes Good PLOs? • Program Learning Outcomes are broad statements of what knowledge, skills, and competencies a student will take with him/her after successfully completing the program • Statements should be broad but measurable • Bloom’s taxonomy is a good resource • Outcomes should, as a set, should fully express the program’s “deliverables.”

6. What Makes Good PLOs (cont.) ? • Content reflects specific aggregate learning targets for a program • Clarity – parents and other external stakeholders can easily discern the value of the lessons learned in the program • Complete – full “educational value” of a program is expressed by set of PLOs • Consistent – language/wording is similar or the same as that used college-wide

7. What Makes for a Good Set of CLOs? • A “complete set” of CLOs should express the “minimum common learning content” of a course • We say “minimum” because the CLOs shouldn’t be a laundry list of all the important lessons students take away • “Common” refers to the premise that multiple instructors may teach the same class – there should be agreement among those instructors about the course’s “core learning” • Effectively communicate to substitute or adjunct instructors the learning that is intended

8. What Makes for a Good Set of CLOs (cont.)? • The design of the set of CLOs should be course-focused, even if a course qualifies for SUNY Gen. Ed., or is part of a program • The CLOs should be an authentic representation of some/much of the learning a student should experience by taking the course • There is no correct number of CLOs • Clearly, assuming “3 or 4” is not the way to go • The number of CLOs will vary depending on the design of the course • For most courses, however, 6-9 will be “typical”

9. Summary of General Guidelines for “Good Sets of CLOs” • Faculty members teaching the course cover the listed CLOs • Faculty assign student work that corresponds to the listed CLOs • CLO set is not comprehensive, but covers “core learning” common across all sections • CLO verbs reflect varied levels of learning in a course • Number of CLOs reflects the curriculum design and type of learning students experience

10. How Should CLOs be designed? • Assume the introductory phrase: “A student who successfully completes this course should be able to…” • Start each CLO with a measureable action verb • Bloom’s taxonomy is a good guide, particularly for providing varying levels of learning targets • Practically speaking, CLOs should be designed with student assignments in mind

11. How Should CLOs be designed (cont.)? • There are important “potholes” to avoid: • Each stated CLO should be simply stated so as to express single learning activities • One action verb assures that to be the case • If several action verbs are used in tandem, make sure that they logically express a single learning activity (such as “identify and describe” or “compare and contrast”) • “demonstrate” should be for observable, physical activities • “understand” cannot be measured • “demonstrate an understanding of…” is imprecise

12. CLOs as Part of Larger Curriculum • If Course Learning Outcomes (CLOs) represent the core learning in a course, then these CLOs should relate to course objectives and to program learning outcomes. • If a course is listed as qualifying for SUNY General Education credit, then the CLOs should likewise support the SUNY SLOs (Student Learning Outcomes) • The linkages between CLOs and course objectives, and the linkages between CLOs and SLOs and PLOs constitute “curriculum maps”

13. Linking Program Learning Outcomes to Courses • Distribute a numbered/lettered list of Program Learning Outcomes (PLOs) among the relevant faculty members • Challenge each faculty member • For each course he/she teaches in the program, determine which of the PLOs each course covers as a significant part of the course content • This exercise benefits from input from multiple faculty members who teach different sections of the same course, but who respond independently • Collect and display the information

14. Example: Program Learning Outcomes A graduate of the Beekeeping program will be able to: • Construct and maintain beehives of appropriate design relative to the beekeeper’s purpose (breeding, honey production, supporting agriculture, etc.) and the breed of bee; • Create appropriate breeding environments and apply professionally-recommended techniques in promoting healthy fertility and growth of bee populations; • Apply and interpret basic genetic tests of bee samples; • Conduct appropriate hybridization techniques; • Identify, recognize, and respond to insect behaviors, including those of bee, competing insect, and predator insect species; • Promote health and wellness among bee populations; • Cultivate plant species supportive of the healthy maintenance of bee populations; • Recognize and manage responses to threats to bee populations, such as changes in the seasons, extreme weather conditions, and the presence of pesticides and other toxins in the environment; • Apply basic business management principles to the management of the costs and expenses of beekeeping; • Demonstrate an understanding of the ethical principles underpinning beekeeping as both a hobby and a profession; • Demonstrate an understanding of the role of bee species in the maintenance of a healthy and sustainable agricultural business; • Teach and train apprentice beekeepers in the rudimentary aspects of the practice of beekeeping; • Capably communicate, orally and in writing, the basic tenets of good beekeeping practices to individuals unfamiliar with bees and the practice of beekeeping.

16. Preliminary Review Indicated: • Program Learning Outcome “m” is not covered • Faculty discussion revealed that virtually every faculty member thought others were covering the outcome in their respective courses • Most faculty members did not feel qualified to teach or grade students on their communication skills • ENV 112 seemed of little value to the curriculum, BUT it does fulfill science requirement • PLOs “c,” “d” and “i” are under-covered

17. Follow-up on the initial mapping • The program faculty is now both empowered and challenged to “fix” these curriculum issues • Benefit of this simple procedure is that the pictorial representation gives participants a quick and easy understanding of an otherwise complex system • Establishing linkage and coverage is not enough – now the faculty must be in a position to document learning

19. Ascertaining an Assessment Strategy of Courses • Some courses are stronger in some areas of interest than others • Although multiple courses may cover the same outcome, not every course provides good assessment opportunities • Some courses offer breadth over depth • Some PLOs can only be covered (and measured) in specific courses

20. Final “Linkage” documented • The faculty has to determine which Course Learning Outcomes best suit the matching PLO • The assumption is that individual courses are taught according to the respective CLOs • If CLOs are supported by graded (and thus, embedded) student assignments, then that documentation will directly support the PLO • Common sense rules for assessment still apply (as do the broad MSCHE assessment principles)

22. Bringing everything together • Note that all courses are participating in the assessment of at least one Course Learning Outcome • All PLOs are covered by at least one assessed course • MSCHE cares only that we demonstrate that PLOs have been assessed; they aren’t concerned about the minutia • This process allows us to show that by fulfilling linked CLOs, we are likewise showing that PLOs have been measured and analyzed

23. Questions/Comments/Criticisms?