2007 Assessment Symposium WELCOME! May 22, 2007

1. 2007 Assessment SymposiumWELCOME!May 22, 2007

2. Study Abroad Centerhelping students gain global perspective Lisa Loberg Director, Study Abroad

3. Continuous Improvement ProjectProcess • Purpose: • To review coordination between campus offices as student participation increases • Examine systems to create new procedures • Team Representatives: • Business Office Registrar • Institutional Research Study Abroad • International Admissions • Outcomes: • Created Datatel codes for all programs and screen • Entirely new system of budgeting and accounting for department • Building relationships and improving communication • Review: • Team to review implemented changes and make new recommendations as needed

4. Welcome Back Survey Purpose Evaluation of: • Specific Study Abroad Program • Academics • Location • Housing • Activities • Overall experience • Study Abroad Center • Effectiveness • Suggestions for improvement • Impact of student experience • Evidence of student ability to • Demonstrate Cultural Competency • Growth in Identity and Values • Appreciate Diversity

5. 2006-07 Survey Design Goals • Review previous data and questions • Incorporate questions that measure Student Learning Outcomes • Convert paper-based survey to electronic Flashlight survey

6. Design Chronology • Prior to 2004: No Study Abroad Experiences Survey. • Spring 2005 1st version: • Paper survey with all open-ended questions. • Useful but difficult to tabulate results. • Spring 2006 2nd version: • Paper survey with both open-ended questions and rank-order scale. • Rank-order helpful; needs development. • Fall 2006 3rd version: • Paper survey with open-ended questions and rank-order scale with scale levels defined. • Provided more accurate information; needs collection improvement. • Spring 2007 Current version: • Electronic Flashlight survey with open-ended questions and rank-order scale. Incorporated specific questions to measure student learning outcome goals. • Results tabulated immediately. Facilitates reporting.

7. Analysis of Data Study Abroad Emerging Themes • Experiences are life-changing. • Cultivates global perspective and respect for people of diverse cultures. • Increases students’ ability to Demonstrate Cultural Competency, Grow in Identity and Values, and Appreciate Diversity. Lessons Learned • Creating surveys in Flashlight is simple and is effective. • Flashlight results are tabulated immediately and easily reported. • Attention to survey design enriches quality of responses and ability to use data. • Survey design can capture the achievement of Student Learning Outcomes.

8. 2007-08 Assessment Goal • Every returning student completes a survey • Incorporate additional questions relating to Student Learning Outcomes • Add a demographics section to make national reporting of statistics even easier

9. Criminal Justice-Program Review Robert J. MeadowsHelen Ahn Lim

10. COAS - Department of Criminal Justice • Two full-time faculty • Robert J. Meadows, Ph.D., Ed.D. ; Professor and Program Chair • Helen Ahn Lim, Ph.D.; Assistant Professor • Three to four adjunct instructors • Approximately 100 majors • About 10-12 classes each semester with an average class size of 20 students CJ Program Features • Approximately 100 majors • Offers B.S. in Criminal Justice (42 units) • Offers Minor in Legal Studies (20 units) • Comprehensive with interdisciplinary focus and cross listing of courses (sociology, psychology, political science) • One of fastest growing UG majors • Internship requirement • Alpha Phi Sigma honor society

11. 2006-07 Program Review Goal To assess how the Program prepares students for employment or post graduate study, to identify graduate’s perceptions regarding the value of their degree and courses, and to learn of their education/employment progress through alumni surveys

12. Alumni Survey Data Collection and Analysis • Collected data from 2000-2006 CJ graduates • 98 graduates identified (46 usable questionnaires) • Questionnaire responses • Employment of alums (33/46 or 71% in jobs related to CJ) including law enforcement, corrections, law, security and Government jobs related to justice. • Alums attending or completing post graduate studies (14/46 or 38%) including degrees in Business (MBA), Psychology, Education, and Criminal Justice.

13. Alumni Survey Data Collection and Analysis • Questionnaire responses • Degree preparation for career (36/46 or 78% believed the degree somewhat prepared or prepared them very much for their present vocation). • Degree preparation for graduate or professional studies. (All respondents attending post grad studies believed the degree prepared them for post graduate study). • Overall reflection of learning experience (39/46 or 85% indicated that their educational experiences and courses taken in criminal justice were either excellent or good).

14. Analysis of Data What did the data say? • The degree provides a relevant and valuable education • Overall, curriculum is consistent with Program goals, CJ and CLU Learning Outcomes • Higher scores than most other programs for Challenge and Engagement in Arts and Sciences • Most diverse student profile in Arts and Sciences What themes emerged? • Problem with understaffing given the number of majors What was learned? • Alumni Survey: Most graduates are employed in relevant field; several attend graduate or law school. Demonstrates the value of senior seminar course and internship requirement. • Courses: Consistent with major criminal justice programs in other schools. However, Alumni Survey suggests that more courses would further enrich the program. • CLU is the only private residential school offering a four-year degree in Ventura, Los Angeles and Santa Barbara Co.

15. Goals and Recommendations Outcomes of Program Review and Measurable Goals • Comprehensive and unique program (meets CJ goals and aligns with CLU Learning Outcomes) • Experienced tremendous growth and is a highly marketable degree • Need to hire more full-time faculty in the future (in order to expand curriculum, and further enhance the quality of the program) • Provide more leadership opportunities for students (group projects, CJ honor society) Recommendations for growth based on analysis • Hire more full-time faculty • Improve facilities and offices

16. Connecting the Dots:School of Business Assessment SystemAn Overview Charles MaxeyBonnie Johnson

17. Key Problem Points Continuous Improvement Model Institutional  School or Department  Program  Course  Assignment  Assessment  Data Collection  Reporting, Discussion, Analysis  Educational Effectiveness and Continuous Improvement

19. The UniversitySchool of Business Professional Preparation Field Specific Knowledge and Experience Ethical and Professional Judgment Interpersonal and Teamwork Skills Liberal Learning Critical Thinking Information Literacy Written Communication Oral Communication Ability to comprehend issues from disciplinary and interdisciplinary perspectives Understanding of cultural and global diversity Integration of knowledge with ethical reflection Character and Leadership Development Growth in Identity and Values Appreciation of Diversity Service to the Community Principled Leadership Professional Preparation Field Specific Knowledge and Experience Ethical and Professional Judgment Interpersonal and Teamwork Skills, Principled Leadership Planning and Organization Statistical Data Analysis Liberal Learning Creative and Critical Thinking Information Literacy and Computer Skills Written Communication Oral Communication and Listening Skills Evaluation Understanding of cultural and global diversity Creation of Value Character and Leadership Development Student Learning Outcomes California Lutheran University and CLU School of Business (Alignment and Expansion) University: School of Business:

20. Standard Pages

21. Following the Process Sample Syllabus: Bus367 – Behavior in Organizations (taught by Dr. Susan Murphy) Sample Assignment Description from Bus367 – Personal Application Assignment Sample Rubric for Assessing Personal Application Assignment Sample Report for Further Departmental Discussion and Continuous Improvement

22. CLU’s Assessment System

23. Library Services Review Cindy Campbell Director of Library Operations and Systems Information Systems & Services

24. 2006-07 Assessment Goal To evaluate Pearson Library services for the purposes of improvement: • Library Services in general • Library collection • Library space • Information literacy instruction

25. Assessment Design Measurements • LibQual Survey • MyCLU Portal Survey Participants • Stratified random sampling • University population • Confidence interval of 90%

26. Data Collection LibQual • Survey Levels and Scale • Minimum, Desired, & Perceived • 1 to 9 with 9 being the most favorable • 22 core items in 3 areas of service provided by the ISS Library Service group: • Affect of Service • Library as Place • Information Control MyCLU Portal Survey

27. Analysis of Data with Resulting Goals • Superiority = Perceived – Desired • Employees who are consistently courteous • Giving users individual attention • Adequacy = Perceived – Minimum

28. Analysis of Data with Resulting Goals

29. Department of PhilosophyProgram Review and Evaluation William Bersley, Chair and Associate Professor Nathan Tierney, Professor Xiang Chen, Professor The Philosophy Department conceives of philosophy as an enterprise of both mind and spirit. The faculty are committed to providing the student with the knowledge and skills of critical and reflective thinking, moral judgment, and understanding of major philosophical traditions and perennial philosophical questions. Such knowledge and skills not only enable the student to become a philosophically literate person, but facilitate the integration of this knowledge with each student’s process of moral, spiritual and intellectual growth.

30. 2006-07 Program Review Goal To compare departmental grade scores to mean grade distributions for Arts and Sciences and Humanities Division.

31. Assessment Design

32. Data Grade Distributions (4 point scale)

33. Analysis of Data • Departmental grade distributions were comparable to both Arts and Sciences (School) and the Humanities Division, with the exception of Spring ‘04 and Fall ‘03 which were somewhat below the averages of the School and Division. • External review of this (and other) data suggests creation of departmental/internal assessment strategy to improve quality, e.g., longitudinal evaluation of capstone writings and supplementary survey/questionnaires for core required courses to determine value to undergraduate education and skills transferable to other courses.

34. Goals and Recommendations • Provide faculty with current data concerning grade distribution in order to prevent grade inflation (see previous data) • Sustain highly evaluated “vital teaching” and articulate the centrality of Philosophy within the university • Increase the role of Philosophy in the Honors Program • Implement the use of Philosophy 400 to initiate a Philosophy Honors Program/Capstone Seminar (requires staffing and budgetary support) • Develop an assessment strategy that authentically improves Philosophy on its own terms • Increase mentoring of sustained student projects • Implement the establishment of a Center of Global Ethics under the leadership of Dr. Nathan Tierney

35. The Academic Assistance Program(The AAP)An essential piece of our retention strategies Damien A. Peña, MSW Senior Director of Academic Programs

36. What is the AAP? • Academic Assistance Program (AAP) • Retention strategy for Probation students and Conditionally Admitted students • Modeled after student success plans in Student Support Services • Evaluates each student’s academic progress through administered assessments • Self-funded (advising fee assessed to students)

37. 2006 – 2007 Program Goals To provide Academic Assistance (strategies and skills) to: • Students on Probation • To be removed from probationary status • Conditionally Admitted Students • To achieve academic success at CLU • Self-selected Students • To stay “ahead”

38. Design and Data Collection Triangulated Strategy • Exit Interview • Gather student perceived performance as a result of being in the program • 1 on 1 with the Academic Counselor • 5 questions • 93 responses out of 99 • Exit Flashlight Survey • Gather student perceived performance using anonymous survey during the last session • Walked student to computer • 9 Open ended questions • 17 scale questions concerning services and the CAAR Office • 93 responses out of 99 • Retention Data • Determine number of students removed from Probation • Data pending until grades submitted

39. Analysis of Data • Students’ perceived outcomes/goals • Wanted to improve grades • Wanted to improve study habits • Positive impact on students • Accountability • Connection with someone on campus • Challenges of program • Communication with Faculty • Capacity issues • Library hours

40. 2007-2008 Program Goals Addressing Challenges • Communication with Faculty: • Capacity: Outside funding to address capacity issues • Library Hours: In Addition • Assess AAP performance using benchmarks identified as indicators of success • Conduct longitudinal study using quantitative and qualitative data (retention data & exit surveys) • Expand services for self-selected students • Achieve 100% return rate for student evaluations

41. Athletic Facility Scheduling Dan Kuntz, Director of Athletics Coaches, Assistant Coaches, Athletics Administrative Assistant

42. 2006-2007 Assessment Goal Goal/Problem To develop and implement an Athletic Facility Master Schedule incorporating internal and external requests for: Gilbert Arena Samuelson Aquatics Center Soiland Recreation Center Tennis courts Game and practice fields To evaluate Effectiveness of Athletics Facility Scheduling Plan Factors and limitations • 19 sports programs / nearly 400 student athletes / 6 practice sessions per week per program / between August 15 and May 15 / overlapping seasons. • 18 sports / 4 to more than 25 home contests (No track facility for Track and Cross Country home contests). • Single venues used by multiple sports teams limits schedule options. • Status of coaches and class schedules. • PT v FT - afternoon hours for practices. • Lack of lighted facilities for outdoor sports. • SCIAC schedules set days and times.

43. Qualitative and Quantitative Research Design Methodology • Data collected annually • Collaborate with Athletic facility users to discover needs for facility use • Data Collection Measures: Electronic facility request forms, e-mails, phone calls, meetings, observations, conference schedules, practice times, internal and external special event requests • Population: Potential users, internal, community Athletics Academics Student Activities Admissions ESSM Classes Alumni Relations Conferences and Events Renters Church Relations Residence Life Club sports Marketing and Communications Intramurals Recreation Multi Cultural / International Programs Open facility requests for students, faculty, staff, alumni and CLA community members

44. Qualitative and Quantitative Research Design

45. Analysis of Data Data Reveals Emerging Themes • Requests growing from internal and external community users for access to CLU Athletic facilities. • New emerging ideas for Athletic facility usage. Additional factors • Maintenance costs are climbing. Lessons Learned • There is confusion about scheduling needs and processes. • Diplomacy is necessary. • Education and communication is needed in the scheduling process- tolerance. • Centralized scheduling is important. • Open access to Athletics facilities can be limited by rental of facilities. Future Challenges • Class scheduling. Lack of lights on fields. Lack of sufficient quality and quantity of fields. Weather variables. League requirements. Special access. • Demand for Athletic facility access and usage will likely grow internally and externally.

46. 2007-08 Assessment Goal 2006 – 2007 Outcomes • Internal and external Athletic facility schedules are coordinated. • Users are satisfied with access. Growth Recommendations • Centralize Athletic facility scheduling for efficiencies to potentially reduce conflicts, miscommunication, and misunderstanding. • Prioritize scheduling of Athletic facilities for CLU students, faculty, staff and alumni despite growing pressures for revenue production from facility rentals. • Increase personnel resources. • Purchase scheduling software. • Implement On-line scheduling. Measurable goals for 2007 - 2008 • Access and flexibility for CLU program offerings sustained. • Provide more student, faculty, alumni and staff access to Athletic facilities. • Attract sports events and activities that bring prospective students and families to CLU. • Develop innovative community outreach programs that attract visitors to see the value of CLU programs.

47. Sharing Responsibility for Multiple Learning OutcomesNew partnerships across Academic Departments and Student Affairs Dr. Silva Karayan Director, Center for Academic Service-Learning Stine Odegard Coordinator, Community Service Center

48. Sharing Responsibility for Multiple Learning Outcomes Purpose: To enhance CLU’s educational effectiveness in a global era, focus on the integration of multiple learning outcomes. Service-learning pedagogy facilitates mission-focused integrative learning through intentional collaboration: Academic Affairs Office Academic Departments Center for Academic Service-Learning (CASL) Community Service Center (CSC)

49. Key Partnerships on Campus • Community Service and Co-curricular Service-learning (CSC) • One coordinator, 4 student workers (10 hours/week) • 2 week-long trips, 4 weekend experiences, 35 day experiences • Co-curricular service-learning: action-reflection model • Center for Academic Service-learning and SLO’s (CASL) • One director, many faculty partners from various departments • Ongoing faculty development opportunities & individualized mentoring • High quality service-learning project components: • Paradigm Shift: From “Deficit Model” to “Empowerment Model” • Orientation Shift: From “Charity and benevolence” to “Learning and making a difference”

50. Assessment Goal and Design Survey Goal To determine student interest and discover the meaning of their experience in the March 10, 2007 Community Service Day (extra-curricular activity) Survey Design and Data Collection • Pre administered on-site, paper format (67 respondents) • Post survey (Flashlight) administered via email (42 respondents) • Review of BCSS/NSSE data • Qualitative and quantitative data Demographics of 117 participants • M/F ratios: 23 males (17%), 94 females (80%) • Fields of study: (Bus.-12%, Comm.-12%, ESSM-8%, Lib. Studies-5%) • Previous service involvement: 65% participated in service at CLU before 85% participated in service during high school