1 / 41

John Hayek Jillian Kinzie George Kuh James Moran SHEEO Higher Education Policy Conference August 13, 2009 Denver Colorad

Assessment for Improvement and Accountability: Examples from the States. John Hayek Jillian Kinzie George Kuh James Moran SHEEO Higher Education Policy Conference August 13, 2009 Denver Colorado. Context. Global Competitiveness in Degree Attainment The New Majority and Demographic Gaps

judah
Download Presentation

John Hayek Jillian Kinzie George Kuh James Moran SHEEO Higher Education Policy Conference August 13, 2009 Denver Colorad

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Assessment for Improvement and Accountability: Examples from the States • John Hayek • Jillian Kinzie • George Kuh • James Moran • SHEEO Higher Education Policy Conference • August 13, 2009 • Denver Colorado

  2. Context Global Competitiveness in Degree Attainment The New Majority and Demographic Gaps Questionable Levels of Student Performance In an Environment of Increasing Fiscal Strain…  We Need Higher Levels of Student Achievement at an Affordable Price

  3. Overview • Purposes of assessment • First lessons from NILOA • Examples of NSSE use

  4. Advance Organizers What are the most important SLOs to measure in your state? What is being done to assess these outcomes? What are the major obstacles or challenges to SLO assessment? What do you need to further SLO assessment in your state?

  5. Two Paradigms of Assessment Ewell, Peter T. (2007). Assessment and Accountability in America Today: Background and Context. In Assessing and Accounting for Student Learning: Beyond the Spellings Commission. Victor M. H. Borden and Gary R. Pike, Eds. Jossey-Bass: San Francisco.

  6. NOLOA • Far too little is known about assessment practices on campuses around the country • NILOA’s mission is to document SLO assessment work, identify and disseminate best practices, and support institutions in their assessment efforts www.learningoutcomesassessment.org

  7. NOLOA FUNDERS • Lumina Foundation for Education • Carnegie Corporation of New York • The Teagle Foundation NATIONAL ADVISORY PANEL

  8. NILOA Activities • Web scans: 700 institutions • Interviews & focus groups with key actors: -- AAC&U -- ACE -- AIR • “Go to” Web site being developed (www.learningoutcomesassessment.org) • Case studies • White papers

  9. NILOA White Papers • Banta et al.: Authentic assessment approaches • Borden: “Measuring Quality II” • Ewell: The improvement and accountability purposes of assessment • Lenth & Hill: Assessment and state policy • Wellman: Assessment results and resource allocation

  10. NILOA Activities • Web scans: 700 institutions • Interviews & focus groups with key actors • “Go to” Web site being developed • Case studies • White papers • Survey of institutional assessment practices

  11. NILOA 2009 Provost Survey • All accredited, undergraduate degree-granting 2- and 4-year public, private, and for-profit institutions in the US (n=2809) • 53% response rate (n=1518)

  12. Tentative Conclusions • Perhaps more assessment underway than some acknowledge or wish to believe • More activity at the department/unit level than institution level • Accreditation is a major force shaping assessment • More attention needed to using and reporting assessment results • Involving faculty is a major challenge • More investment likely needed to move from data to improvement

  13. Examples from the States • South Dakota • Texas • Tennessee • Kentucky • Pennsylvania

  14. NSSE State and University System Participation 2000-2009 • California State Universities • City University of New York • Concordia Universities • Connecticut State Universities • Kentucky Council on Postsecondary Education • New Jersey Public Univ. • North Dakota University System • Ontario Universities • Penn State System • Pennsylvania System of Higher Education • South Dakota Public Universities • State University of New York • Tennessee Publics • Texas A&M System • Texas Six • University System of Georgia • West Virginia State System • University Systems of Hawaii, MaineMaryland, Massachusetts, Missouri, New Hampshire, North Carolina, Texas, Wisconsin

  15. State System Use of NSSE • Institutional improvement • Research • Accountability & public reporting • Performance funding • Decision-making models   • More specifically…. • track student engagement overtime, • set engagement performance goals, • explore engagement patterns by student characteristics • merge NSSE data with other system data to examine factors associated with student success

  16. Examples: Student Engagement Data in State Systems Assessment and Research • Use data to examine student experience and in predictive models for retention, degree attainment Institutional Improvement • Data for institutional improvement initiatives, and to encourage collaboration among system & campuses to address common challenges Accountability • Monitor and demonstrate quality in undergraduate education

  17. South Dakota Board of Regents • Data from 5 NSSE administrations since 2002 for all public institutions • Longitudinal analysis shows upward trend in student engagement

  18. South Dakota Board of Regents Research project findings: • Strong links between student performance and student-faculty interaction • Student effort in-and-outside of class correlated with other Regents Assessments of general education goals, Collegiate Assessment of Academic Proficiency (CAAP) • Academic performance • Critical thinking • Persistence So. Dakota School of Mines & Technology

  19. South Dakota Board of Regents Findings Influenced Improvements and Policies: • Expansion of universities’ research capacity to foster collaborative projects between faculty and students • Salary competitiveness to retain high quality faculty

  20. University of Texas System NSSE Use for Accountability detailed in U.T. 2008 System Report: • Inform Texas legislature, Board of Regents, U.T. System Faculty Advisory Council and Student Advisory Council, and general public • VSA • U.T. System Accountability Report • Legislatively mandated Consumer Satisfaction requirements

  21. University of Texas System NSSE used for benchmarking: • NSSE results considered useful for benchmarking because items are strongly associated with student success • NSSE Benchmarks provides continuing source of comparisons to gauge U.T. System progress • Where significant differences occur, institutions are encouraged to identify areas for improvement

  22. Tennessee Higher Education System All TN Public 4yr institutions participated in NSSE as required by TN performance funding program. • Student engagement assessment & research • Feedback for internal use • Appropriate peer comparisons • Focus for improvement • Performance funding

  23. Tennessee Higher Education System Interest in NSSE as a process measure… What happens to students while attending higher education? • Research about the relationship between engagement and satisfaction at TN public institutions showed the importance of: • Quality of academic advising • Relationship with peers, faculty & staff • Institutional support for success

  24. Tennessee Higher Education System • Identified State & Institutional Policy Implications: • More research to examine relationship between engagement, satisfaction and degree production (develop longitudinal database, track individual students surveyed) • Institutions encouraged to evaluate conditions of student services and improve advising; etc.

  25. Using student engagement data in Kentucky to promote student success John Hayek Vice President August 13, 2009 SHEEO conference

  26. Kentucky Postsecondary Education Enterprise • 8 public universities, 16 community colleges and technical, 20 regionally accredited, nonprofit, independent colleges and universities, and numerous other postsecondary options. • 212,000 undergraduate students and 27,000 graduate and professional students in 2007. • 49,700 total degrees and certificates awarded; 18,600 bachelor’s degrees in 2007. • 600,000 college and university alumni. • $183 million in federal Pell grants, $486 million in federal loans, and over $180 million in state financial aid in 2006. • $4.2 billion in total revenue generated by Kentucky’s public universities and colleges in 2007. • 1,869 public postsecondary education buildings with more than 47 million gross square feet. • 30,000 full-time faculty and staff employed at Kentucky’s public universities and colleges in 2008. • $327 million in extramural R&D expenditures combined at the University of Kentucky and University of Louisville in 2006.

  27. Budget Update Net General Funds have been cut $78 million or 7.2% over the past 18 months, while K-12, Medicaid, and Corrections have been exempted. Source: CPE Comprehensive Database • Tuition increases (3% for community college, 4% for comprehensive universities, and 5% for research universities), lowest in a decade

  28. The 50 States Relationship between per capita income and college degrees Why it Matters The single factor with the greatest power to explain differences in per capita income between states is the percentage of college graduates. Milken Institute, 2002

  29. Context • Postsecondary Reform Legislation in 1997 - Long-term goals • Public Agenda – Short-term strategic plan updated every four years • Accountability System – Key performance indicators and goal setting process

  30. NSSE (2001, 2003, 2005, 2007, & 2009) • College-level learning pilot (2002) • CCSSE (2006, 2007, and 2008) • Regular reports to Council on progress • Faculty development meetings • Accreditation and quality enhancement plans • NSSE Workshop KY Action Kentucky was one of the first states to participate as a system in NSSE.

  31. Increased attention to quantity and quality • Improvement on 25% of benchmark scores between 2001 and 2007 • FTFT retention rates improved from 71% (1998) to 74% (2007) • Graduation rates improved from 37% (1998) to 47% (2006) • Bachelor’s degrees increased from 14,600 (1998) to 18,600 (2007) Outcomes Key is to keep student engagement and student outcomes data on the radar screen.

  32. Performance Matters Pennsylvania State System of Higher Education Jim Moran Vice Chancellor for Academic and Student Affairs August 2009

  33. Overview • What is the Pennsylvania State System of Higher Education (PASSHE)? • How did we develop a culture of accountability and performance? • What results have we achieved?

  34. PASSHE In Brief • Fourteen (14) regional public universities • Focus on baccalaureate and masters programs (doctoral mission at one university) • Rural settings • Over 112,000 students • 90% undergraduate / 90% Pennsylvania residents • System created in 1983

  35. Measuring What Matters • Defining values and goals • Developing meaningful, reliable measures • Data driven decision-making

  36. Discussion • Quantitative Measures • Qualitative Measures • What have we learned? • Where are we going? • Evolving Use of the NSSE

  37. Conclusions • Performance driven by values and goals • Solid data upon which to make decisions is everyone’s business • Accountability for results is shared

  38. For Discussion… • What other ways can NSSE and other assessment tools be used to stimulate improvement in state system institutions? • How can NSSE be used to advance state system concerns, or inform policy? • NSSE advises against using results in performance funding. What is an appropriate use? • What other uses and tools are you considering?

More Related