slide1 n.
Skip this Video
Loading SlideShow in 5 Seconds..
John Hayek Jillian Kinzie George Kuh James Moran SHEEO Higher Education Policy Conference August 13, 2009 Denver Colorad PowerPoint Presentation
Download Presentation
John Hayek Jillian Kinzie George Kuh James Moran SHEEO Higher Education Policy Conference August 13, 2009 Denver Colorad

Loading in 2 Seconds...

play fullscreen
1 / 41

John Hayek Jillian Kinzie George Kuh James Moran SHEEO Higher Education Policy Conference August 13, 2009 Denver Colorad - PowerPoint PPT Presentation

  • Uploaded on

Assessment for Improvement and Accountability: Examples from the States. John Hayek Jillian Kinzie George Kuh James Moran SHEEO Higher Education Policy Conference August 13, 2009 Denver Colorado. Context. Global Competitiveness in Degree Attainment The New Majority and Demographic Gaps

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation

PowerPoint Slideshow about 'John Hayek Jillian Kinzie George Kuh James Moran SHEEO Higher Education Policy Conference August 13, 2009 Denver Colorad' - judah

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript

Assessment for Improvement and Accountability:

Examples from the States

  • John Hayek
  • Jillian Kinzie
  • George Kuh
  • James Moran
  • SHEEO Higher Education Policy Conference
  • August 13, 2009
  • Denver Colorado

Global Competitiveness in Degree Attainment

The New Majority and Demographic Gaps

Questionable Levels of Student Performance

In an Environment of Increasing Fiscal Strain…

 We Need Higher Levels of Student Achievement at an Affordable Price

  • Purposes of assessment
  • First lessons from NILOA
  • Examples of NSSE use
advance organizers
Advance Organizers

What are the most important SLOs to measure in your state?

What is being done to assess these outcomes?

What are the major obstacles or challenges to SLO assessment?

What do you need to further SLO assessment in your state?


Two Paradigms of Assessment

Ewell, Peter T. (2007). Assessment and Accountability in America Today: Background and Context. In Assessing and Accounting

for Student Learning: Beyond the Spellings Commission. Victor M. H. Borden and Gary R. Pike, Eds. Jossey-Bass: San Francisco.

  • Far too little is known about assessment practices on campuses around the country
  • NILOA’s mission is to document SLO assessment work, identify and disseminate best practices, and support institutions in their assessment efforts



  • Lumina Foundation for Education
  • Carnegie Corporation of New York
  • The Teagle Foundation


niloa activities
NILOA Activities
  • Web scans: 700 institutions
  • Interviews & focus groups with key actors:

-- AAC&U

-- ACE

-- AIR

  • “Go to” Web site being developed (
  • Case studies
  • White papers
niloa white papers
NILOA White Papers
  • Banta et al.: Authentic assessment approaches
  • Borden: “Measuring Quality II”
  • Ewell: The improvement and accountability purposes of assessment
  • Lenth & Hill: Assessment and state policy
  • Wellman: Assessment results and resource allocation
niloa activities1
NILOA Activities
  • Web scans: 700 institutions
  • Interviews & focus groups with key actors
  • “Go to” Web site being developed
  • Case studies
  • White papers
  • Survey of institutional assessment practices
niloa 2009 provost survey
NILOA 2009 Provost Survey
  • All accredited, undergraduate degree-granting 2- and 4-year public, private, and for-profit institutions in the US (n=2809)
  • 53% response rate (n=1518)
tentative conclusions
Tentative Conclusions
  • Perhaps more assessment underway than some acknowledge or wish to believe
  • More activity at the department/unit level than institution level
  • Accreditation is a major force shaping assessment
  • More attention needed to using and reporting assessment results
  • Involving faculty is a major challenge
  • More investment likely needed to move from data to improvement
examples from the states
Examples from the States
  • South Dakota
  • Texas
  • Tennessee
  • Kentucky
  • Pennsylvania
nsse state and university system participation 2000 2009
NSSE State and University System Participation 2000-2009
  • California State Universities
  • City University of New York
  • Concordia Universities
  • Connecticut State Universities
  • Kentucky Council on Postsecondary Education
  • New Jersey Public Univ.
  • North Dakota University System
  • Ontario Universities
  • Penn State System
  • Pennsylvania System of Higher Education
  • South Dakota Public Universities
  • State University of New York
  • Tennessee Publics
  • Texas A&M System
  • Texas Six
  • University System of Georgia
  • West Virginia State System
  • University Systems of Hawaii, MaineMaryland, Massachusetts, Missouri, New Hampshire, North Carolina, Texas, Wisconsin
state system use of nsse
State System Use of NSSE
  • Institutional improvement
  • Research
  • Accountability & public reporting
  • Performance funding
  • Decision-making models  
  • More specifically….
    • track student engagement overtime,
    • set engagement performance goals,
    • explore engagement patterns by student characteristics
    • merge NSSE data with other system data to examine factors associated with student success
examples student engagement data in state systems

Examples: Student Engagement Data in State Systems

Assessment and Research

  • Use data to examine student experience and in predictive models for retention, degree attainment

Institutional Improvement

  • Data for institutional improvement initiatives, and to encourage collaboration among system & campuses to address common challenges


  • Monitor and demonstrate quality in undergraduate education
south dakota board of regents
South Dakota Board of Regents
  • Data from 5 NSSE administrations since 2002 for all public institutions
  • Longitudinal analysis shows upward trend in student engagement
south dakota board of regents1

South Dakota Board of Regents

Research project findings:

  • Strong links between student performance and student-faculty interaction
  • Student effort in-and-outside of class correlated with other Regents Assessments of general education goals, Collegiate Assessment of Academic Proficiency (CAAP)
    • Academic performance
    • Critical thinking
    • Persistence

So. Dakota School of Mines & Technology

south dakota board of regents2

South Dakota Board of Regents

Findings Influenced Improvements and Policies:

  • Expansion of universities’ research capacity to foster collaborative projects between faculty and students
  • Salary competitiveness to retain high quality faculty
university of texas system

University of Texas System

NSSE Use for Accountability detailed in U.T. 2008 System Report:

  • Inform Texas legislature, Board of Regents, U.T. System Faculty Advisory Council and Student Advisory Council, and general public
    • VSA
    • U.T. System Accountability Report
    • Legislatively mandated Consumer Satisfaction requirements
university of texas system1

University of Texas System

NSSE used for benchmarking:

  • NSSE results considered useful for benchmarking because items are strongly associated with student success
  • NSSE Benchmarks provides continuing source of comparisons to gauge U.T. System progress
  • Where significant differences occur, institutions are encouraged to identify areas for improvement
tennessee higher education system

Tennessee Higher Education System

All TN Public 4yr institutions participated in NSSE as required by TN performance funding program.

  • Student engagement assessment & research
  • Feedback for internal use
  • Appropriate peer comparisons
  • Focus for improvement
  • Performance funding
tennessee higher education system1

Tennessee Higher Education System

Interest in NSSE as a process measure… What happens to students while attending higher education?

  • Research about the relationship between engagement and satisfaction at TN public institutions showed the importance of:
    • Quality of academic advising
    • Relationship with peers, faculty & staff
    • Institutional support for success
tennessee higher education system2

Tennessee Higher Education System

  • Identified State & Institutional Policy Implications:
  • More research to examine relationship between engagement, satisfaction and degree production (develop longitudinal database, track individual students surveyed)
  • Institutions encouraged to evaluate conditions of student services and improve advising; etc.
using student engagement data in kentucky to promote student success

Using student engagement data in Kentucky to promote student success

John Hayek

Vice President

August 13, 2009

SHEEO conference


Kentucky Postsecondary Education Enterprise

  • 8 public universities, 16 community colleges and technical, 20 regionally accredited, nonprofit, independent colleges and universities, and numerous other postsecondary options.
  • 212,000 undergraduate students and 27,000 graduate and professional students in 2007.
  • 49,700 total degrees and certificates awarded; 18,600 bachelor’s degrees in 2007.
  • 600,000 college and university alumni.
  • $183 million in federal Pell grants, $486 million in federal loans, and over $180 million in state financial aid in 2006.
  • $4.2 billion in total revenue generated by Kentucky’s public universities and colleges in 2007.
  • 1,869 public postsecondary education buildings with more than 47 million gross square feet.
  • 30,000 full-time faculty and staff employed at Kentucky’s public universities and colleges in 2008.
  • $327 million in extramural R&D expenditures combined at the University of Kentucky and University of Louisville in 2006.

Budget Update

Net General Funds have been cut $78 million or 7.2% over the past 18 months, while K-12, Medicaid, and Corrections have been exempted.

Source: CPE Comprehensive Database

  • Tuition increases (3% for community college, 4% for comprehensive universities, and 5% for research universities), lowest in a decade

The 50 States

Relationship between per capita income and college degrees

Why it Matters

The single factor

with the greatest power

to explain differences in

per capita income between states

is the percentage of college graduates.

Milken Institute, 2002



  • Postsecondary Reform Legislation in 1997 - Long-term goals
  • Public Agenda – Short-term strategic plan updated every four years
  • Accountability System – Key performance indicators and goal setting process

NSSE (2001, 2003, 2005, 2007, & 2009)

  • College-level learning pilot (2002)
  • CCSSE (2006, 2007, and 2008)
  • Regular reports to Council on progress
  • Faculty development meetings
  • Accreditation and quality enhancement plans
  • NSSE Workshop

KY Action

Kentucky was one of the first states to participate as a system in NSSE.


Increased attention to quantity and quality

  • Improvement on 25% of benchmark scores between 2001 and 2007
  • FTFT retention rates improved from 71% (1998) to 74% (2007)
  • Graduation rates improved from 37% (1998) to 47% (2006)
  • Bachelor’s degrees increased from 14,600 (1998) to 18,600 (2007)


Key is to keep student engagement and student outcomes data on the radar screen.


Performance Matters

Pennsylvania State System of Higher Education

Jim Moran

Vice Chancellor for Academic and Student Affairs

August 2009

  • What is the Pennsylvania State System of Higher Education (PASSHE)?
  • How did we develop a culture of accountability and performance?
  • What results have we achieved?
passhe in brief
  • Fourteen (14) regional public universities
  • Focus on baccalaureate and masters programs (doctoral mission at one university)
  • Rural settings
  • Over 112,000 students
  • 90% undergraduate / 90% Pennsylvania residents
  • System created in 1983
measuring what matters
Measuring What Matters
  • Defining values and goals
  • Developing meaningful, reliable measures
  • Data driven decision-making
  • Quantitative Measures
  • Qualitative Measures
  • What have we learned?
  • Where are we going?
  • Evolving Use of the NSSE
  • Performance driven by values and goals
  • Solid data upon which to make decisions is everyone’s business
  • Accountability for results is shared
for discussion

For Discussion…

  • What other ways can NSSE and other assessment tools be used to stimulate improvement in state system institutions?
  • How can NSSE be used to advance state system concerns, or inform policy?
  • NSSE advises against using results in performance funding. What is an appropriate use?
  • What other uses and tools are you considering?