Overcoming the Teacher’s Dilemma (Breadth vs. Depth)

2. Overcoming the Teacher’s Dilemma(Breadth vs. Depth) The American River College Model of College-Wide Course-Level SLO Assessment*********** Mr. John AubertSLO Assessment Coordinator, Geography Dr. Yuj ShimizuFaculty Researcher, Psychology

3. Our Goals Today • Provide background • Our college • Our transition to new standards • Share our course-level assessment program • Development and implementation • Integration with institutional planning • Breadth vs Depth approach • Discuss preliminary results

4. American River College Background and Context 5 days… Located in Sacramento, CA (one of four colleges in the Los Rios CCD) 36,646 unduplicated students 65 academic departments 2,197 individual courses 3,473 sections taught (Fall 2009) 13,318 course level SLOs defined Accreditation site team arrives in

5. 31 The SLO/Assessment Index… 24 The number of times that the word “outcome” appears in the accreditation standards*… The number of times that the words “assess” and “assessment” appear in the accreditation standards*… *2006, Accrediting Commission for Community and Junior Colleges

6. Accreditation Workshop Today! AUBERT Accrediting Commission for Community and Junior Colleges

7. American River College Transition to New Standards OUTCOMES OBJECTIVES broad, overarching specific, discrete Late 1990s: Blooms taxonomy utilized in writing course objectives 2004: SLO creation/revision embedded in curriculum process The Outcomes vs Objectives distinction treated as a continuum…

8. American River College Transition to New Standardscontinued… 2004-2006: SLO creation/revision continues 2006-2007: Focus shifts toward assessment Spring 2007: Reassigned time allocated to develop assessment program Fall 2007: SLO Assessment Coordinator appointed (20-40%)

9. Our Task… • To develop an SLO Assessment program which would: • Improve teaching and learning • Address the needs of a large college • Be consistent with our campus culture • Meet accreditation standards • To survive the process…

10. Our Guiding Principles… Respect faculty workload Rely on faculty expertise Be flexible Integrate with existing processes Practice shared governance Meet or exceed accreditation standards

11. What do they want from us? Pervasive, widespread dialogue Assessment which is: institutionalized and integrated systematic and continuous (not episodic) Data analysis which leads to change Change which improves student learning Documentation!

12. Our solution… Comprehensive assessment program adopted September 2007 New standing committee formed Program review streamlined 65 departments divided into 3 cohorts Cohort 1 entered 1st year (three year cycle) Substantial outreach and training !

13. The American River CollegeSix YearInstitutional Planning Cycle

14. SLO Assessment Cohorts

15. American River CollegeTwo-Part SLO Assessment for Courses<<< Three-Year Cycle for Departments >>> YEAR 1 YEAR 2 YEAR 3 Part I Student Self Assessment(every course assessed) Action Plandevelopandimplement Action Plancontinue implementation reassess Part II Faculty Designed Assessment(one course assessed)

16. Assessment Cycle Version 1.0 DENIAL ANGER ACCEPTANCE BARGAINING DEPRESSION

17. SLOs and Assessment • The Problem: Are students learning what we expect them to? • The Challenge: How we can measure this in a sensible way?

18. Our solution: Proposed Model of Assessment • To develop a process that would: • minimize workload • gather data broadly • gather data systematically • be specific to each course’s SLOs • be transparent • improve teaching and learning

19. Course Outline of Record Course SLO’s

20. Student Self Assessment-Survey Pilot Template

21. Course SLO’s Student Self Assessment-Survey Pilot

22. Positive Ratings!!!

23. Student Self Assessment Survey • Survey piloted Spring 2007. • Fast and Simple • Results matched teacher predictions • 2007-2008 (1st Cohort): Full term, face to face courses (43% Return Rate) • 2008-2009(2nd Cohort): All courses (63% Return Rate) • 2009-2010 (3rd Cohort): Now in progress (Return Rate ???)

24. Student Self Assessment Survey • Adopted Class Climate Software by Scantron to administer surveys, store survey data, and to report results (centralized approach) • Our approach can be easily modified for a decentralized / departmental approach

25. Sample Student Self Assessment Survey

26. Survey Results Self assessments/Student ratings can be both reliable (Ross, Rolheiser, & Hogaboam-Gray, 2002) and valid (Cohen, 1981; Fox & Dinur, 2006; Ross, 2006)

27. Criticisms of Student Self Report Data Known biases in self, course and instructor ratings (Dunning, Heath, & Suls, in press; Frye, 2005) • Self presentation concerns, overconfidence • Class size • Class format (Lecture vs. Seminar) • Class purpose (elective, major, GE requirement) • Class level (intro vs. 2nd year course) • Subject matter (Behavioral Science, Humanities, Sciences) • Difficulty of instructor grading • Liking for instructor • “Hotness” and “Ease” (Coladarci & Kornfield, 2007)

28. Interpreting Results Correctly SLO 1 → SLO 2 → SLO 3 → SLO 4 → SLO 5 → SLO 6 → SLO 7 → Look at Relative Ratings, Not Absolute Ratings

29. Typical Instructor Reaction to Student Self Assessment Results • Instructors typically agree and can make quick sense of the data: • Aligns with time spent on each SLO, • Realize that they do not teach that SLO anymore, • The SLO is at the end and they never get to it, • Realize that the SLO is the most difficult part of the course, • Written in a way that students would not understand

30. Thus, survey provided Yeah but… • Quick • Broad and systematic • Sensitive to outliers (produced actionable data) • Indirect • Lacks flexibility • Does not utilize faculty expertise • Lacks depth

31. Airport Security Screening Device

33. Part 2: Faculty Designed Assessment (Direct Assessment) Customized assessment documented on a common template • What course and why? • What SLO(s)? • When? • How broadly? • How will you assess? (describe tool or rubric) • What is the criteria for successful SLO achievement? • Who will be administering / scoring? • Submitted to the SLO Assessment Committee

34. American River CollegeTwo-Part SLO Assessment for CoursesThree-Year Cycle for Departments YEAR 1 YEAR 2 YEAR 3 Part I Student Self Assessment(every course assessed) Action Plandevelopandimplement Action Plancontinue implementation reassess Part II Faculty Designed Assessment(one course assessed)

35. Year 2: SLO Action Plan Departments respond to both parts of assessment • Part 1: Respond to the Student Self-Assessment • Part 2: Respond to the Faculty Designed Assessment

36. Year 2: SLO Action Plan Detailed Instructions Action Plan and Instructions approved through an 8 month process

37. Where we are now Of the 65 Academic departments… • 2/3’s have participated in the college wide SLO Assessment process • 1/ 3 have completed Action Plans • 63 Separate Actions taken/planned as a result of SLO assessment

38. Conclusion “The college may be said to have completed the awareness and development levels and one-third of the proficiency level.”

39. Thank you! John Aubert, SLO Coordinator, Geography aubertj@arc.losrios.edu or 916-484-8637 Yuj Shimizu, Faculty Researcher, Psychology shimizy@arc.losrios.edu or 916-484-8149 http://inside.arc.losrios.edu/~slo/