Life After VSA:
1 / 34

Life After VSA: An Update - PowerPoint PPT Presentation

  • Uploaded on

Life After VSA: An Update. Oklahoma Association of Institutional Research and Planning Fall 2009 Conference October 23, 2009. Voluntary System of Accountability (VSA).

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation

PowerPoint Slideshow about ' Life After VSA: An Update' - rafael-duke

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Life after vsa an update

Life After VSA:

An Update

Oklahoma Association of

Institutional Research and Planning

Fall 2009 Conference

October 23, 2009

Voluntary system of accountability vsa
Voluntary System of Accountability (VSA)

Initiative for public universities to supply basic, comparable information on the undergraduate student experience to important constituencies through a common web report – the College Portrait.

Sponsored by APLU & AASCU. Funded by the Lumina Foundation for Education.

Life after vsa an update

Provide a useful tool for students during the college search process

Assemble information that is transparent, comparable, and understandable

Demonstrate accountability and stewardship to public

Measure educational outcomes to identify and enhance effective educational practices

VSA Goals

Life after vsa an update

Multiple Audiences process

Prospective students and families

High school, community college counselors

Policymakers and elected officials

Campus faculty and staff

Accrediting associations, institution and state boards

Taxpayers and public-at-large

Vsa participation
VSA Participation process

  • 329 institutions from 49 states and Puerto Rico, including 7 from Oklahoma(10/05/09)

  • 63% of 520 APLU & AASCU institutions

  • 250+ College Portraits posted on

Vsa context
VSA Context process

Perceived lack of useful and transparent data preventing institutions from demonstrating accountability and contribution to public good (Spellings Commission)

Policymakers and employers demand evidence of educational outcomes – particularly broad transferable skills

A community effort
A Community Effort process

  • Concept outlined in APLU white papers, refined through feedback from members

  • Strengthened by AASCU endorsement

  • Structure developed by task forces - presidents, provosts, student affairs, IR

    • 82 members from 70 institutions

    • Input from associations and larger HE community

Result college portrait
Result: College Portrait process

Free web report that communicates user-friendly, consistent information about college campuses.

  • Student & Family Information

  • Student Experiences & Perceptions

  • Student Learning Outcomes

Www collegeportraits org process

No rankings, no spin ... just the facts!

Student family information
Student & Family Information process

Institution description, highlights

Student characteristics

Cost of attendance, net cost calculator

Success & progress rate

Degree programs, class sizes

Post graduation plans

Life after vsa an update

Introduction to Campus Community process

Menu links take users to each section and data element

Campus Defined Buttons

Life after vsa an update

Success & Progress Rate process

  • Tracks student progress across 2-year and 4-year institutions using NSC data

  • New cohort query available to all Student Tracker users

  • Developed as alternative to IPEDS graduation rate

  • Focuses on student success in higher education system

Student experiences perceptions
Student Experiences & Perceptions process

group learning

interaction with faculty

institutional commitment

active learning

diversity experiences

student satisfaction

  • Selected senior responses from one of 4 surveys: NSSE, CSS, CSEQ, UCUES

  • Snapshot of student experiences on campus within 6 common areas

Life after vsa an update

6 Common Constructs process

Selected responses from Seniors

Life after vsa an update

Student Learning Outcomes process

Learning gains in critical thinking, analytic reasoning, written communication measured at institution level

Reported in a common format using CAAP, CLA or MAPP

Student learning outcomes test validity study

Student Learning Outcomes processTest Validity Study

Funded by the Fund for the Improvement of Post-Secondary Education (FIPSE).

Background process

  • VSA measures value-added for student learning outcomes of critical thinking and written communications.

  • VSA offers a choice of three tests of student learning outcomes: CAAP, CLA and MAPP.

  • Testing companies agreed to adopt the same value-added methodology and reporting format, controlling for entering student quality.

  • Validity study needed to ensure that the three tests produced consistent scores.

Methodology process

  • Students take each of the three tests.

  • 13 tests administered to 1,100+ students at 13 universities

  • 13 tests components of CLA, CAAP, MAPP

    • 4 tests of critical thinking

    • 2 tests of reading

    • 2 tests of mathematics

    • 4 tests of writing

    • 1 test of science

Participating universities

University of Michigan, Ann Arbor process

University of Minnesota, Twin Cities

University of Texas, El Paso

University of Vermont

University of Wisconsin, Stout

Participating Universities

  • Alabama A&M

  • Arizona State

  • Boise State

  • California State-Northridge

  • Florida State

  • MIT

  • Trinity College

  • University of Colorado-Denver

Conclusions process

  • Validity and reliability are high across tests and should not affect choice of measure

  • Factors affecting adoption:

    • Acceptance by students, faculty, administrators or other policy makers

    • Trade-offs in cost, easy of administration, etc

    • Utility of the test for other purposes such as supporting campus activities and services or providing guidance on improving learning

  • Thus VSA will continue to permit universities to choose to use either CLA, CAAP or MAPP.

Final reports
Final Reports process

Two reports are available on the VSA website:

  • The complete (and technically complex) Test Validity Study full report

  • An interpretative report for VSA participating schools can be found at

Life after vsa an update

P e t e r D process













How Should Colleges Assess & Improve College Learning?

Employers’ Views on the Accountability Challenge

Key findings from survey among 301 employersConducted November 8 – December 12, 2007for

Employers find college transcripts of limited use in evaluating potential
Employers Find College Transcripts Of Limited Use In Evaluating Potential

How useful do you find the college transcript in helping you evaluate job applicants’ potential to succeed at your company?

Not sure

Very useful

Fairly useful

Not useful

Just somewhat useful


Assessments effectiveness in ensuring college graduates have skills knowledge
Assessments’ Effectiveness In Ensuring College Graduates Have Skills/Knowledge

Very effective

Fairly effective

Supervised/evaluated internship/community-based project where students apply college learning in real-world setting


Advanced comprehensive senior project, such as thesis, demonstrating student’s depth of knowledge in major & problem-solving, writing, and analytic reasoning skills


Essay tests to evaluate level of problem-solving, writing, and analytical-thinking skills


Electronic portfolio of student’s college work, including accomplishments in key skill areas and faculty assessments


Multiple-choice tests of general content knowledge



Assessments usefulness in helping employers evaluate college graduates potential
Assessments’ Usefulness In Helping Employers Evaluate College Graduates’ Potential

Very useful

Fairly useful

Faculty supervisor’s assessment of applicant’s student internship/ community-based project applying college learning in real-world setting


Sample of applicant’s student senior project and overview of faculty assessment of the project


Electronic portfolio of applicant’s college work, including accomplishments in key skill areas and faculty assessments


Applicant’s score on essay test to evaluate level of problem-solving, writing, and analytical-thinking skills


Applicant college’s score showing how the college compares to others in advancing students’ critical-thinking skills


Applicant’s score on multiple-choice test of general content knowledge



An obvious place to begin

An obvious place to begin: College Graduates’ Potential

Tell The Students What They Are Expected to Accomplish

The power of rubrics for assessing learning
The Power of Rubrics for Assessing Learning College Graduates’ Potential

  • Help guide students around learning expectations

  • Help guide faculty

  • Reinforces assessment for learning

  • Places individual faculty judgment within national shared experience

  • Can build up from course level to institutional reporting needs AND down from general to specific program/course context

Open edpractices org
Open College Graduates’

A website open to everyone in higher education

Free to join, easy to sign up

A library of sorts…a site where a collection of rubrics (and other best practices) can be:




Built with open source software to share knowledge and best practices openly

Value process
VALUE Process College Graduates’ Potential

Develop criteria from rubric collection

Each rubric development team member wrote a row of performance descriptors for one criteria

Draft rubric then discussed and debated, edited and transformed

Testing and feedback – e-Portfolios

Value project
VALUE Project College Graduates’ Potential

Life after vsa an update

  • Voluntary Framework of Accountability College Graduates’ Potential

  • Initiative to report comparable information on community college

  • Use measures better suited to community colleges

  • Funded by the Bill & Melinda Gates Foundation and the Lumina Foundation for Education

  • Sponsored by American Association of Community Colleges (AACC), College Board and Association of community College Trustees

Life after vsa an update

  • Oklahoma College Graduates’ Potential

  • Encourage participation in VSA

  • Participate in Achieving the Dream

  • Revise Assessment Policy

  • Use measures in Brain Gain funding

Life after vsa an update

Dr. Debra L. Stuart College Graduates’ Potential

Vice Chancellor for Educational Partnerships