using nsse to answer assessment questions
Download
Skip this Video
Download Presentation
Using NSSE to Answer Assessment Questions

Loading in 2 Seconds...

play fullscreen
1 / 35

Using NSSE to Answer Assessment Questions - PowerPoint PPT Presentation


  • 256 Views
  • Uploaded on

Using NSSE to Answer Assessment Questions. Regional User’s Workshop October 2005. Shimon Sarraf Center for Postsecondary Research, Indiana University Bloomington. “NESSIE”. Overview. Why should “engagement” be assessed Assessment Techniques with NSSE data Group Exercise and Discussion.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Using NSSE to Answer Assessment Questions' - arleen


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
using nsse to answer assessment questions

Using NSSE to Answer Assessment Questions

Regional User’s Workshop

October 2005

Shimon Sarraf

Center for Postsecondary Research, Indiana University Bloomington

overview
“NESSIE”Overview
  • Why should “engagement” be assessed
  • Assessment Techniques with NSSE data
  • Group Exercise and Discussion
why should engagement be assessed
Why should engagement be assessed?

Because individual effort and involvement are the critical determinants of college impact, institutions should focus on the ways they can shape their academic, interpersonal, and extracurricular offerings to encourage student engagement.

Pascarella & Terenzini, How College Affects Students, 2005, p. 602

who says engagement is important
Who says engagement is important?

Quality of Effort (Pace)

Student Involvement (Astin)

Social and Academic Integration (Tinto)

Good Practices in Undergraduate Education (Chickering & Gamson)

Student Engagement (Kuh)

assessment approaches
Assessment Approaches
  • Normative - compares your students’ responses to those of students at other colleges and universities.
  • Criterion - compares against a predetermined value or level appropriate for your students, given your institutional mission, size, curricular offerings, funding, etc.
  • Longitudinal – compare your average scores over time
assessment with nsse data
Assessment with NSSE Data
  • Descriptive displays of engagement patterns by any number of student characteristics
    • Use individual items and/or scales
  • Year-to-year tracking of student engagement
  • Multivariate models for retention, degree attainment, grades, other outcomes
  • Special peer comparisons with aspirational, regional, and mission-related institutions
descriptive analysis
Descriptive Analysis
  • Comparisons by Student Background
    • Minority Students
    • First Generation College Student
  • Comparisons by Enrollment Characteristics
    • Greek
    • Athletes
    • College and/or Department
approaches to descriptive analysis
Approaches to Descriptive Analysis
  • Most valued activities

What is most valued at your institution, in departments, what does the data show?

  • Investigate “Nevers”

Work on reducing or eliminating reports by students of never doing specific engagement activities.

  • How much variation?

Box & Whiskers

slide9
Descriptive Analysis

Responses of Seniors by Major

slide10
Descriptive Analysis

Responses of Seniors by Major

slide13
T-test: p<.000; Effect Size: -.29

Descriptive Analysis

Seniors Scale Scores by Transfer Status

data consideration disaggregating results
Data Consideration: Disaggregating Results
  • Experience indicates that survey results are most likely to be used when the results are disaggregated by specific program or unit (e.g., college or department).
  • Targeted oversamples of specific units may be warranted.
    • Sampling error statistics may not be a good indicator of data quality with smaller units.
slide16
Comparisons Across Years

FY Student Responses to Stu-Fac Items by Year

slide17
Comparisons Across Years

FY and Senior Stu-Fac Scale Scores by Year

slide18
Comparisons Across Years

FY Scores on Four Scales by Year

slide21
Multivariate Modeling

Regression model predicting grades at the end of the first year.

slide22
Multi-equation Modeling

A structural equation model explaining longitudinal relationships that lead to FY grades.

Pre-college

Engagement

Outcome

special peer comparisons
Special Peer Comparisons

Selecting a peer group

By mission

By size

By department

By Race

By Locale

Current or Aspirant Peers

slide24
Special Peer Comparisons

Standard Frequency Report with Selected Peer Group

slide25
Living on-campus

Commuters

Special Peer Comparisons

Carnegie Group

slide26
Special Peer Comparisons

Student Level Benchmark Report

special peer comparisons student distributions
Special Peer Comparisons: Student Distributions
  • First-year academic challenge scores
  • Are these two schools the same?
  • Same median benchmark score
  • Different range of scores
data considerations
Standard error of mean (precision of estimate)

Non-response bias

Weighting your sample to look like the population

Comparability of survey items year-to-year

Use other assessment techniques (i.e., focus groups, other surveys) to validate your findings—NSSE is but one source of assessment information

Data Considerations
nsse consortium
NSSE Consortium
  • 6 or more institutions sharing comparative data
  • Great way to add value to participation
  • Often times mission specific
  • Ability to ask additional questions
assessment exercise department level analysis
Assessment Exercise :Department-Level Analysis
  • Scenario
    • Nesseville State University is preparing for an upcoming accreditation related to its engineering program
    • The college was encouraged to incorporate more “student voice” into their educational outcomes assessment
    • The University Provost and College Dean have worked to increase buy-in for using NSSE to collect information
assessment exercise department level analysis32
Assessment Exercise :Department-Level Analysis
  • Concerns to Address
    • Faculty are concerned that the Engineering College places too little emphasis on challenging and engaging pedagogical practice
    • The Dean is concerned that some of the departments are not preparing their students for life after graduation as well as others
    • The Provost would like to know how NSU engineering students compare to Engineering students nationwide
    • In previous Campus Surveys Engineering students have voiced dissatisfaction with their undergraduate experience
assessment exercise department level analysis33
Assessment Exercise :Department-Level Analysis
  • Building the Analysis
    • In submitting their population file, Nesseville State University included an extra variable to identify Engineering students and their departments within the College
    • Nesseville State indicated that they wished to oversample all Engineering seniors not identified for the random institutional sample
    • NSU constructed several NSSE student-level scales to use as a basis for their analysis, as well as requested a special analysis from NSSE to get normative data
assessment exercise department level analysis34
Assessment Exercise :Department-Level Analysis
  • What are some patterns that are evident in these results?
  • Were the expressed stakeholder concerns confirmed?
  • What differences are notable among departments?
  • What are some other sources of data that would be ideal to shed light on these results?
  • What additional analyses would you want to conduct?
using nsse to answer assessment questions35
Using NSSE to Answer Assessment Questions

Shimon Sarraf

Research Analyst

Indiana University Center for Postsecondary Research

1900 East 10th Street

Eigenmann Hall, Suite 419

Bloomington, IN 47406

Ph: 812-856-2169

[email protected]

www.nsse.iub.edu

ad