Proposal for Direct Assessment of Program Outcomes (For Faculty Meeting, Nov. 28, ’05)
Terminology • Program Outcomes: • Knowledge and skills that students possess by the time of graduation. • Criterion 3 Outcomes: • Set of 11 required outcomes: • (3.a) an ability to apply knowledge of math, science and engineering; • (3.g) an ability to communicate effectively; • (3.j) knowledge of contemporary issues; • Three groups: Technical; “soft skills”; gen. ed.-related.
Terminology (contd.) • Additional outcomes (if any): As specified by program: • (II.i) an understanding of differential and integral calculus, and of statistics. • (IV.ii) an ability to communicate effectively.
Criterion Requirement • Program must assess degree of student achievement of outcomes. • Recent “interpretation”: • Must include “direct” assessments. • Direct assessment: Based on actual student work, not opinion surveys and the like.
Problem • Accreditation team reported a weakness with respect to this requirement. • Main criticism: Lack of direct assessment. • What other programs have done: Collect samples of student work from several courses; have a committee “assess” them against their program’s outcomes.
Problem (contd.) • ABET does not precisely specify what it expects. • Seems to vary with evaluator/team/... • Conjecture: • Direct assessment must be done by a committee; • Must be properly documented (over several years?);
Possible Approaches • 1. Based on samples of student work from courses: • Identify specific courses (560, 601, 625, ...?); • Require instructors to collect samples of student work from each; • Have a committee(s) assess these against the outcomes (and propose needed changes); • Maintain records of this activity.
Possible Approaches (contd.) • 2. An easily-administered exam that all BS-CSE majors must take near graduation: • Multiple-choice (GRE-CS-like) exam based on higher-level required courses (560, 601, 625, 655, 660, 670, 675, 680); • Tailor questions so they map to the “technical” Criterion 3 outcomes (including social/ethical issues in computing-outcome); • Offered once a quarter;
Possible Approaches (contd.) • 2. (Exam option – contd.): • UGSC will evaluate results (once a year). • Noteworthy results concerning each area will be brought to attention of appropriate faculty; • Any resulting changes will be made via standard curriculum processes;
Soft Skills-Outcomes • Four skills: Oral comm.; written comm.; team-working; lifelong learning. • Rely on capstone courses for these: • Identify rubrics for oral comm., team-work; • All capstone courses will use these and report results to UGSC;
Soft Skills-Outcomes (contd.) • Written communication, Lifelong learning: • Introduce, in all capstone courses, a requirement of a 3-4 page paper dealing with a new tool or technology or process, etc. • Students will be required to research the item on their own and report findings in their paper. • A common rubric will be used in all the capstone courses to evaluate this paper, with respect to written communication and lifelong learning skills. • Results will be reported to UGSC.
Proposal (part 1) • Introduce a multiple-choice exam that all BS-CSE majors will be required to take near graduation. • Topics: CSE 560, 601, 625, 655, 660, 670, 675, 680. • Questions: Tailored to correspond to Outcomes 3.a, 3.b, 3.e, 3.f, 3.k. • Results: Discussed once a year by UGSC.
Proposal (part 2) • Revise capstone courses to use a common rubric to evaluate oral communication; and a common rubric to evaluate team-work. • Revise capstone courses to require a paper on a new tool. technology, etc; use a common rubric to evaluate it with respect to lifelong learning and written communication skills. • Report results to UGSC (to be discussed once a year).