Before & After: What Undergraduates and Alumni Say About Their College Experience and Outcomes

Before & After: What Undergraduates and Alumni Say About Their College Experience and Outcomes • Angie L. Miller, NSSE & SNAAP Research Analyst • Amber D. Lambert, NSSE & SNAAP Research Analyst • Becca Houghton, SNAAP Project Coordinator • Assessment Institute • October 30th, 2012

Introduction • Surveys are a common means of assessment in higher education • Student surveys are conducted on a variety of topics, from student engagement to use of campus resources to faculty evaluations • Alumni surveys are used to gather information about satisfaction, acquired skills, and career attainment

Research Questions • Question 1: Are there differences in how students and alumni perceive aspects of their institutional experiences and the skills and competencies that they acquire at their institutions? • Question 2: What discipline-specific strengths and weaknesses exist when exploring students’ reported development of skills and abilities?

What is SNAAP? Strategic National Arts Alumni Project • On-line annual survey of arts graduates • Investigates educational experiences and career paths • Provides findings to educators and policymakers to improve arts training, inform cultural policy, and support artists

Who is Surveyed? • Graduates of: • Arts schools, departments, or programs in colleges and universities • Independent arts colleges • Arts high schools • Both graduate and undergraduate degree recipients • All arts disciplines

SNAAP Questionnaire Topics • Formal education and degrees • Institutional experience and satisfaction • Postgraduate resources for artists • Career • Arts engagement • Income and debt • Demographics

SNAAP 2011 Administration Information • Administered in Fall 2011 • 66 participating institutions • 58 postsecondary and 8 high schools • Over 36,000 total respondents

What is NSSE? National Survey of Student Engagement • NSSE gives a snapshot of college student experiences in and outside of the classroom by surveying first-year and senior students • NSSE items represent good practices related to desirable college outcomes • Indirect, process measures of student learning and development

NSSE Purpose • NSSE annually gathers valid, reliable information on the extent to which students engage in and are exposed to proven educational practices that correspond to desirable learning outcomes. • Results indicate how students spend their time and what they gain from college.

NSSE 2012 Administration Information • Administered in Spring 2012 • 546 participating U.S. institutions • Over 285,000 total respondents • Each year, experimental item sets appended at end of core survey

Question 1: Methodology Participants from 6 institutions that participated in both SNAAP11 and NSSE12 Senior NSSE respondents from arts majors in corresponding SNAAP participating programs (n = 222) Alumni of undergraduate SNAAP programs from graduating cohorts of 2001-2010 (n = 593)

Question 1: Respondents *SNAAP asks race/ethnicity in a check-all format; NSSE asks race/ethnicity in a forced-choice format

Question 1: SNAAP items

Question 1: NSSE items

Question 1: Results Means comparison for overall rating of institutional experience (4-point scale from “Poor” to “Excellent”) suggests that alumni give higher general appraisals *p<.05; **p<.01; ***p<.001

Question 1: Results (cont.) Means comparisons for satisfaction with aspects of time at institution (4-point scale after removing “Not Relevant” option) suggests that alumni give lower specific appraisals for certain aspects *p<.05; **p<.01; ***p<.001

Question 1: Results (cont.) Means comparisons for amount of institutional contribution to acquired skills and competencies (4-point scale from “Not at all” to “Very much”) show a similar pattern, with alumni giving lower specific appraisals for certain skills

*p<.05; **p<.01; ***p<.001

Question 2: Methodology Participants from 37 institutions that participated in NSSE12 and received the experimental items derived from the SNAAP survey Senior NSSE respondents from all majors* (n = 23,726) *Excluding “other” and “undecided” categories

Question 2: Respondents

Question 2: Respondents’ Majors

Question 2: Results Looking at frequencies for students in the different majors reporting that their institution contributed “Very Much” to acquiring certain skills and competencies reveals interesting patterns Some patterns are expected; others highlight less apparent similarities among major fields

% Reporting Institution Contributed “Very Much” to Acquiring…

Discussion Question 1 Alumni may be viewing their institutional experience as a whole through rose-colored glasses when they think about “the good old days” Post-graduation experiences in the workplace may better enable alumni to reflect on certain aspects of their time Alumni may also learn that they needed to develop some skills more once they have gained work experience

Discussion (cont.) Question 2 The skills that students acquire while at their institutions can vary greatly among different majors Many of these differences are expected (and increase our confidence in the survey instrument) Other patterns link seemingly dissimilar majors, such as arts and engineering, or business and education

Limitations May not represent ALL students and alumni, data only available for those participating in both SNAAP and NSSE (and those receiving experimental NSSE items) Difficulty of surveying alumni: response rates, tracking accurate contact information Relies on self-reported data

Conclusions Important institutional information can be gained through surveying both students and alumni Students may be better able to provide information about affective components of their experience, while alumni may be better judges of specific things needed in the workplace A variety of perspectives can also be gained through exploring differences by majors

http://3millionstories.com/March 7-9, 2013 Nashville, TN

Indiana University Center for Postsecondary Research Strategic National Arts Alumni Project snaap@indiana.edusnaap.indiana.edu National Survey of Student Engagement nsse@indiana.edu nsse.indiana.edu Angie L. Miller anglmill@indiana.edu Amber D. Lambert adlamber@indiana.edu Becca Houghton rfagen@indiana.edu

Before & After: What Undergraduates and Alumni Say About Their College Experience and Outcomes