Maximizing Your NSSE & CCSSE Results

1. Maximizing Your NSSE & CCSSE Results Nathan Marti Senior Research Associate CCSSE Jillian Kinzie Associate Director NSSE

2. Agenda • Why is assessment of student engagement important? • What is student engagement? • NSSE & CCSSE fundamentals • Interpreting NSSE and CCSSE data • Using benchmarks • Q & A SDSU Assessment Conference NSSE/CCSSE Workshop

3. Introductory Activity • What questions do you have coming into this session? • Assessment at your institution: • What do you want to know about your students? • Why do you want to know this? • What is the purpose of your assessment initiative(s)? • To what extent have you used NSSE and CCSSE data? SDSU Assessment Conference NSSE/CCSSE Workshop

4. Why assessment of student engagement? “Colleges and universities, for all the benefits they bring, accomplish far less for their students than they should.” -- Derek Bok “We can tell people almost anything about education except how well students are learning.” -- Patrick M. Callan, President, National Center for Public Policy and Higher Education SDSU Assessment Conference NSSE/CCSSE Workshop

5. What Do We Need to Know? What constitutes quality in undergraduate education? How might we measure quality? What types of evidence of quality would be helpful in guiding improvement efforts? SDSU Assessment Conference NSSE/CCSSE Workshop

6. Foundations of Student Engagement • Quality of effort(Pace) • Student involvement(Astin) • Social and academic integration (Tinto) • Good practices in undergraduate education(Chickering & Gamson) • Learning and development model(Pascarella) SDSU Assessment Conference NSSE/CCSSE Workshop

7. Agreement within the Research • What matters most is what students do, not who they are. • Educationally effective institutions channel student energy toward the right activities. SDSU Assessment Conference NSSE/CCSSE Workshop

8. Principles of Good Practice in Undergraduate Education • Student-faculty contact • Active learning • Prompt feedback • Time on task • High expectations • Experiences with diversity • Cooperation among students (Chickering & Gamson, 1987) SDSU Assessment Conference NSSE/CCSSE Workshop

9. Path Model for Assessing Change in Student Learning(Pascarella) SDSU Assessment Conference NSSE/CCSSE Workshop

10. What Really Matters in College: Student Engagement • Because individual effort and involvement are the critical determinants of college impact, institutions should focus on the ways they can shape their academic, interpersonal, and extracurricular offerings to encourage student engagement. Pascarella & Terenzini, How College Affects Students, 2005, p. 602 SDSU Assessment Conference NSSE/CCSSE Workshop

11. Two Assertions • The time and energy students devote to educationally purposeful activities is the single best predictor of their learning and personal development. • These things can be influenced and measured by institutions. SDSU Assessment Conference NSSE/CCSSE Workshop

12. Two Components of Student Engagement • What studentsdo – time and energy devoted to educationally purposeful activities • 2. What institutionsdo – using effective educational practices to induce students to do the right things SDSU Assessment Conference NSSE/CCSSE Workshop

13. Educational Process Indicators National Survey of Student Engagement(pronounced “nessie”)Community College Survey of Student Engagement(pronounced “sessie”) College student surveys that assess the extent to which students engage in educational practices associated with high levels of learning and development SDSU Assessment Conference NSSE/CCSSE Workshop

14. NSSE – The College Student Report Based on effective educational practices Items have face and content validity Designed & tested for validity and reliability Relatively stable over time Credibility of self-reported data Students will participate Actionable data SDSU Assessment Conference NSSE/CCSSE Workshop

15. NSSE Administration Centrally administered by third party Spring term Random samples of first-year & senior students Administration modes: • Paper • Web-only • Web + 40% response rate Over 275,000 students at 600 institutions annually 40+ consortium, state systems SDSU Assessment Conference NSSE/CCSSE Workshop

16. NSSE’s Institutional Report NSSE Overview Institutional data Respondent characteristics Means and frequency comparisons by selected peers, Carnegie, national Using NSSE results Accreditation Toolkits, Facilitators Guide PowerPoint presentation SDSU Assessment Conference NSSE/CCSSE Workshop

17. Benchmark Reports • Effective Educational Practices Level of Academic Challenge Active and Collaborative Learning Student-Faculty Interaction Enriching Educational Experiences Supportive Campus Environment SDSU Assessment Conference NSSE/CCSSE Workshop

18. What have we learned from NSSE so far? • The single best predictor of student satisfaction with college is the degree to which they perceive the college environment to be supportive of their academic and social needs. • Effective educational practices measured by NSSE are independent of institutional selectivity. • Schools that have a lower student-faculty ratio, more full-time faculty, and more classes with fewer than 20 students generally score higher on all five NSSE benchmarks. • Grades, persistence, student satisfaction, and engagement go hand in hand. SDSU Assessment Conference NSSE/CCSSE Workshop

19. Grades, persistence, and engagement go hand in hand SDSU Assessment Conference NSSE/CCSSE Workshop

20. Benchmark Scores for All Students by Undergraduate Enrollment SDSU Assessment Conference NSSE/CCSSE Workshop

21. Academic Challenge at Two Public Universities Student engagement varies more within than between institutions. SDSU Assessment Conference NSSE/CCSSE Workshop

22. Relationship Between NSSE & Graduation Rates *All correlations are significant at p<.01 SDSU Assessment Conference NSSE/CCSSE Workshop

23. NSSE Promising Findings • More than 75% of “A” students say they are highly motivated to succeed compared with only half of the “C” students. • At institutions where faculty members use effective educational practices more frequently in their classes, students are more engaged over all and gain more from college. • A majority of students (54% of first-year students and 63% of seniors) says they often discuss ideas from readings or classes with others outside of class, and well over 90% do this at least sometimes. SDSU Assessment Conference NSSE/CCSSE Workshop

24. NSSE Results • Are diagnostic; to help institutions look holistically at undergraduate experience • Help pinpoint aspects not in line with mission, or what institution expects • Identify weaknesses and strengths in educational program • Help institutions know what to focus on to improve student learning and success SDSU Assessment Conference NSSE/CCSSE Workshop

25. Questions to answer with NSSE results • How many hours per week do first-year students spend studying? Do women study more than men? • What % of seniors work with faculty members on activities other than coursework (activities, committees)? Does this differ by major? • What % of FY and SR spend 0 hours in co-curricular involvements? Is this more than at peer institutions? • Do FY students work more frequently with classmates on assignments outside of class than their counterparts at peer institutions? SDSU Assessment Conference NSSE/CCSSE Workshop

26. Questions to answer with NSSE results cont’d • Do NSSE results match our mission and what we say about a [INSTITUTION] experience? • Are we meeting our own expectations for having a supportive campus environment? • Since implementing a new multicultural education initiative and expanding diversity programming, has our score on the diversity scale changed? • Are FY who withdraw from the institution different in terms of engagement than students who are retained? • How are we performing compared to our select peers (normative benchmarking) or to our institutionally identified standards (criterion benchmarking)? SDSU Assessment Conference NSSE/CCSSE Workshop

27. NSSE suite of instruments Beginning College Survey of Student Engagement Faculty Survey of Student Engagement National Survey of Student Engagement • Annual survey of first-year students at baccalaureate degree-granting institutions • 2005-2006 pilot administration at 70 institutions • Administered prior to start of classes, usually at orientation or welcome week • Parallel survey to measure faculty expectations for student engagement • 2006 administration at 131institutions • Administered during the spring semester • Annual survey of first-year and senior students • 2006 administration at 571institutions • Administered during the spring semester SDSU Assessment Conference NSSE/CCSSE Workshop

28. Faculty Survey of Student Engagement (FSSE) • Parallels NSSE content • Fosters campus-wide discussions of teaching and learning • To date more than 53,000 faculty members at over 300 four-year institutions SDSU Assessment Conference NSSE/CCSSE Workshop

29. Beginning College Survey of Student Engagement – BCSSE Designed for entering first-year students as a companion to NSSE Measures: pre-college academic and co-curricular experiences expectations for educationally purposeful activities during college New! SDSU Assessment Conference NSSE/CCSSE Workshop

30. BCSSE Instrument Three pilots: ‘04, ‘05, and ‘06 2004 pilot with 28 schools, 15,890 students 2005-2006 pilots with 80 institutions, 39,986 students Study effect of students’ background on NSSE scores Use to examine gap between expectations and engagement Registration now open SDSU Assessment Conference NSSE/CCSSE Workshop

31. NSSE Accreditation Toolkit SDSU Assessment Conference NSSE/CCSSE Workshop

32. NSSE Accreditation Toolkit SDSU Assessment Conference NSSE/CCSSE Workshop

33. Connecting NSSE Data to Accreditation Standards - Example • Accreditation standard: “Demonstrate effectiveness of student academic and social support services” • Evidence for institutional self study: • Information about availability and student use of tutoring, writing support, peer study groups, counseling services • NSSE indicates FY & SR believe institution emphasizes spending time studying and support for student success; 79% seniors tutored or taught peers; positive correlation between peer collaboration outside of class, satisfaction and first-year retention • Positive student satisfaction data about support services • Area for improvement - seniors indicate low gains in writing and completing drafts of papers; institution responds with examination of writing requirement in senior capstone and targets seniors for increased use of writing center SDSU Assessment Conference NSSE/CCSSE Workshop

34. Institutional Example: NSSE and Enrollment Management • The enrollment management area at Meredith has used NSSE results to help guide the enrollment marketing strategies. They look closely at trends and make adjustments to programs and campus visitation days to ensure that students are more cognizant of student involvement and engagement opportunities. • An academic dean reports using NSSE information when speaking to parents at an admissions event. "Parents seemed impressed that there was data to support the points that I was making about what we say about the student/faculty relationships and educational opportunities at Meredith." SDSU Assessment Conference NSSE/CCSSE Workshop

35. SDSU Assessment Conference NSSE/CCSSE Workshop

36. Institutional Example: Hanover College • A detailed summary of NSSE is sent to the faculty as well as the Admission and Student Life staffs to ensure the results, both good and bad, are understood by key folks on campus. Last year, Admission requested an additional presentation and discussion of findings to help them better understand the strengths of the Hanover experience and how that impacts student fit. SDSU Assessment Conference NSSE/CCSSE Workshop

37. Institutional Example: NSSE & Assessing General Education goals • Used NSSE items in 11a-p to assess institutional impact on college-level competencies (a.k.a., indirect measures of student learning outcomes) • Undergraduate seniors 2005 NSSE results confirmed findings from 2004 • Most seniors (75%+) reported that KSU experience had “substantial impact” (VM+QAB) in 9 or 16 college-level competencies • KSU rank ordered competencies, showing connection to mission, and compared to other master’s instit where KSU was sig. higher, comparable, sig. lower on competencies SDSU Assessment Conference NSSE/CCSSE Workshop

38. Institutional Example: Program Development and Strategic Planning • NSSE results framed a “Sophomore Experience” • 2005 = Pace’s 5th year of participation • Concern regarding SP- JR persistence; FY results offers context for understanding exp. as students enter SP year • Established “SP Experience Working Group” to investigate if FY exp. carried over in SP year. Focused on low NSSE score items, conducted focus groups, created sophomore survey. Led to pilot of “Pace Plan” (mentoring), includes Career Exploration Course, Sophomore Kick-Off Day • NSSE also used in strategic indicators, Accred, NCATE, AACSB, Faculty Development/Colloquia, items used by offices (Technology, Multicultural Affairs), studies performed by Enrollment Mngmt. SDSU Assessment Conference NSSE/CCSSE Workshop

39. CCSSE History • In 2001, received 3-year grant funding from Lumina Foundation and Pew Charitable Trusts to design, pilot and administer a community college survey of student engagement • Pilot administration in the fall of 2001 - 12 institutions participated • National Field Test in the spring of 2002 - 48 institutions participated • Annual, national, administrations in the spring of each year: • 2003 - 93 institutions • 2004 - 152 institutions • 2005 - 257 institutions • 2006-193 institutions • 2007-279 institutions • Self-funding project beginning September 2004 SDSU Assessment Conference NSSE/CCSSE Workshop

40. CCSSE’s Survey Administration • Stratified random sample of credit courses • Stratification – morning, afternoon, evening • In-class, paper and pencil administration • Average administration time = 35 minutes • Completed surveys returned to CCSSE for scanning and analyses SDSU Assessment Conference NSSE/CCSSE Workshop

41. Community College Faculty Survey of Student Engagement (CCFSSE) • Pilot test in fall of 2004 • First national administration in spring of 2005 • Over 20,000 participants SDSU Assessment Conference NSSE/CCSSE Workshop

42. CCSSE Institutional Report • CCSSE Overview • College Results: A Synopsis • Means Summary • Frequency Distributions • Benchmark Reports • Using CCSSE Results • CCFSSE • Codebook • Appendix SDSU Assessment Conference NSSE/CCSSE Workshop

43. CCSSE Annual Report • Review of CCSSE Benchmarks • Active and Collaborative Learning • Student Effort • Academic Challenge • Student-Faculty Interaction • Support for Learners • Focus on trends and issues for community and technical colleges SDSU Assessment Conference NSSE/CCSSE Workshop

44. Overview of 2005 National Survey Results • More than 133,000 community college students from 257 community and technical colleges in 38 states responded to the 2005 CCSSE survey. • Since 2002, more than 320,000 students in 404 colleges across 43 states have responded to CCSSE. • Consortia for 2005 include statewide participation in Indiana, Louisiana, North Dakota, Virginia, and West Virginia. Other state-based consortia include groups of colleges in Georgia, Illinois, Minnesota, North Carolina, and Northeast Minnesota. SDSU Assessment Conference NSSE/CCSSE Workshop

45. Institutional Use: Example • Estrella Mountain (AZ) • Received CCSSE results in the summer of 2004 • Implemented an intensive improvement program that included student engagement in the college-wide assessment process • Student engagement is now an Estrella Mountain Core Indicator of Effectiveness for Student Success • Conducted three improvement sessions with students, faculty and staff; discussed the implications of CCSSE at the annual strategic planning retreat, and identified five improvement strategies for increasing student engagement. • Developing a freshmen institute (with learning communities), • Improving publicity of workshops and services available to students (through the use of electronic kiosks), • Developing student life activities to increase contact with faculty in non-academic settings, • Creating facilities to improve adjunct faculty interaction with students, and; • Integrating student support services into the course curriculum • René G. Willekens, Dean of Planning, Research and Effectiveness SDSU Assessment Conference NSSE/CCSSE Workshop

46. Institutional Use: Example • Northwest Vista College (TX) • CCSSE results are used in the Quality Enhancement Plan (QEP) at Northwest Vista College (NVC). • NVC’s QEP concentrates on assessing student learning outcomes for the core curriculum by serving as a focused area of NVC’s Attitude, Skills, and Knowledge (ASK) model. • CCSSE supports both NVC’s QEP and ASK by providing information about NVC’s student learning outcomes, including critical thinking, cooperative learning, and student collaboration; • CCSSE survey items directly and indirectly relate to many of the 12 student learning outcomes identified as important by NVC faculty. • Christa Emig, Math Instructor SDSU Assessment Conference NSSE/CCSSE Workshop

47. Institutional Use: Example • North Hennepin Community College (MN) • We designed an exercise on the web site in which faculty are asked to guess how students answered CCSSE questions. After guessing, they see the real answer. Instead of doing this on paper, we used personal response system "clickers" and got immediate feedback after guessing at each question. The data could be displayed in several ways for discussion. We had staff, faculty, and administration very engaged! • We used questions from each of the benchmark groupings, and after getting everyone interested in the data, we divided them randomly into benchmark groups and asked them to identify 2 priorities for change at the college in that area. These are now worked into our assessment plan initiatives. • Mary Diedrich, Active Learning and Assessment Coordinator SDSU Assessment Conference NSSE/CCSSE Workshop

48. Questions?