VSA & Assessment of Learning Outcomes

1. VSA & Assessment of Learning Outcomes Sally Frazee, Temple University Criss Gilbert, University of Minnesota-Twin Cities Susan Morgan – Appalachian State University

2. V S A ? Voluntary System of Accountability 2007 APLU Association of Public & Land-grant Universities AASCU Association of American State Colleges & Universities “College Portrait” of Undergraduate Education

4. College Portrait

6. Student Learning Outcomes • Measure improvement in student abilities in three broad areas • General cognitive skills that cut across disciplines: critical thinking, analytical reasoning, and written communication • VSA sponsored review of sixteen Outcomes Assessment instruments…3 were selected - CAAP - CLA - Proficiency Profile • Pilot project – expectation that best practices will emerge as data are collected

7. How Does This Work? Institutional learning outcomes as students enter and exit Random sample of freshmen & seniors University High Stakes _____ Student Low Stakes Assess within 4 years, & then administer every 3 years Cross-sectional methodology

8. CAAP • CLA • Proficiency Profile Outcomes Measures Approved by VSA Sampling Costs Administration Student recruitment Student motivation Use of data

9. Proficiency Profile (formerly MAPP) • ETS • Integrated test of general education skills • Paper-and-pencil or online delivery • Proctored and unproctored administrations

10. Standard Form 108 items 2 hours Abbreviated Form 36 items 40 minutes No minimum of test takers to produce aggregate reports Score Reports include • a total score and subscores and proficiency classifications for individual students • total scores, norm-referenced subscores and criterion-referenced scores at the group level • an alphabetical roster with 17 individual student scores Minimum of 50 test takers to produce aggregate reports Score Reports include • a total score, but no subscores, for individual students • total scores, norm-referenced (scaled) subscores and criterion-referenced scores (proficiency classifications) at the group level.

12. Outcomes Measure Journey at Temple • Signed on for VSA in 2007 • 2007 CAAP Pilot & 2008 MAPP Pilot • Opportunity to try out administration • Opportunity to try out recruitment • Honoraria to fund faculty workshop • 2009: Faculty Workshop: Looking at Learning • 2010: Pilot Administration of the Proficiency Profile, with no VSA obligation to publish results

13. Faculty Workshop Multiple Ways of Looking at Assessment (Periodic Program Review) VSA – How and Why (Vice Provost, MARC) Using Published Tests (Chair, Educational Psychology) A Look at Professional Exams (MARC, Institutional Research)

14. Fall 2010 Administration • GOAL: Abbreviated version to 200 freshmen within 6 weeks of the start of the semester • PLAN: • Email to all 4,300 incoming freshmen on 9/16 • Lots of convenient test sessions • $20.14 in Diamond Dollars • Spread the word – • in-class & dorm presentations • posters and plasma screens • newspaper articles • information desk & giveaways • Budget will be $15,000-$20,000 per year

15. VSA & Assessment of Learning Outcomes Questions?