The “Self Regarding Institution” Using Assessment for Accountability and Improvement

1. The “Self Regarding Institution” Using Assessment for Accountability and Improvement Douglas R. Davenport Dean, College of Arts and Sciences Truman State University Kirksville, Missouri 63501 douglas@truman.edu

2. Indiana University South BendAssessment WorkshopMarch 28, 2008

3. What are your goals? • List three things you hope to gain by attending this workshop. • Rank them in terms of importance.

4. Questions to Consider… • Why does higher education engage in assessment? • What motivates YOU to do assessment?

5. Assessment at IUSB… • What are the best examples of assessment you know of on campus? • What might you learn from them? • What are the key challenges facing the implementation of assessment on campus? • Who is responsible for assessment at IUSB? • (Everybody!)

6. “Begin with the end in mind.” (Covey) Where Do We Begin?

7. What are our Learning Goals?” • Write down the student learning goals for… • Your favorite course to teach • Your academic program • The university • Can they be assessed?

8. Writing Student Learning Outcomes Learning outcome statements should: • reflect essential knowledge, skills or attitudes; • focus on results of the learning experiences; • reflect the desired end of the learning experience, not the means or the process; • Learning outcomes statements may be considered to be exit behaviors. (Adapted from Lesch, 2008)

9. Writing Student Learning Outcomes Key Components: • Action word (performance) • Learning statement (the learning) • Criterion (the conditions of the performance) (Adapted from Lesch, 2008)

10. Action words to avoid: Appreciate Enjoy Know Realize Be aware of Perceive Appropriate action words: Use Develop Analyze Express Evaluate Organize Create Write Plan Apply (Adapted from Lesch, 2008) Writing Student Learning Outcomes

11. Exercise… Write one or more learning outcome statements for the following goals… • Graduates will be able to communicate effectively in writing. • Graduates will have an understanding of cultures beyond the United States. • Graduates will appreciate the fine and performing arts.

12. Time for a Temperature Check… “What is the most important thing you have learned thus far?” “What important questions remain unanswered?”

13. How Do We Assess Outcomes? General design principles… • Know your purposes (accountability and improvement) • Use multiple measures and mixed-methods • Make it your own • Don’t reinvent the wheel • Start slowly • Don’t assess every outcome every year • Involve faculty • Use authentic assessment wherever possible (e.g., embed assessment in courses, etc)

14. The Truman Experience • Began assessment program in 1972 • Started with senior tests • Added a writing assessment (SWE) and portfolio (1980s) • Emphasized value-added assessment (1990s) • Added self-reports (CSEQ, GSQ, CIRP) (2000s)

15. Truman Assessment Today • Nationally-normed senior tests in every major • CLA • MAPP and CAPP • CSEQ • CIRP • Portfolio Project • Student Interview Project • Graduating Student Questionnaire • http://assessment.truman.edu

16. Using Portfolios for Multiple Assessments at Truman • Writing • Critical Thinking • “Modes of Inquiry” • Interdisciplinary Thinking • http://assessment.truman.edu/components/Portfolio/index.asp

17. Writing Assessment & Portfolios:Rationale • “Leverage” current assessment activities • A natural location • Retains the benefit of embedded assessment • Minimizes additional student work

18. Writing Assessment & Portfolios:Putting it into Practice • Tested in 2004, using Critical Thinking Submissions • Incorporated in 2005, using a new “Critical Thinking and Writing” prompt

19. Portfolio Findings • The vast majority of students demonstrated competence in all three areas • Organization – 81% • Style – 81% • Mechanics – 84%

20. Portfolio Findings

21. Portfolio Findings

22. Portfolio Findings

23. Questions? • e-mail: douglas@truman.edu

24. Why Assessment? - Accountability • External Forces… • Spellings Report, A Test of Leadership (NCLB, Part 2) • Public Funding Priorities • Rising Cost of Higher Education • Call for Objective, Comparative Measures • “It's proper for the government to define ‘legally drunk,’ but not ‘legally intelligent.’”(Eubanks, 2006)

25. College Portrait: Voluntary System of Accountability (VSA) • Sponsored by NASULGC and AASCU • Provides consumer information regarding institutional measures, student experiences and perceptions, and student learning outcomes. • Learning outcomes– CLA, MAPP, and CAPP • Experiences and perceptions – CSEQ, CIRP, NSSE, CSS

26. Why Assessment? - Improvement • Assessment for Learning • The Learning Organization (Senge) • Turbulent Times • “It is knowledge creation, not knowledge reproduction, that creates competitive advantage.” (Banta, 2007)

27. Are These Motivations Mutually Exclusive? • Not at all… • Accountability is a hallmark of democratic institutions. • Higher education requires transparency. • However, assessment to merely pacify is problematic – it reinforces shallow learning. • So…

28. Characteristics of Assessment by Purpose

29. Turbulent Times • Did you know?… • Today’s learner will have 10-14 jobs by age 38. • The top 10 jobs that will be demand in 2010 did not exist in 2004. • We are preparing students for jobs that do not yet exist, using technologies that have not been invented in order to solve problems that we have not yet recognized.

30. Components of Truman's Assessment Program.  Click on an instrument in the left column to learn more. Nationally Normed/Criterion Referenced Assessment Instruments – General • Collegiate Learning Assessment (CLA - CAE) • College Basic Academic Subjects Examination (C BASE – Univ. of MO) • Collegiate Assessment of Academic Proficiency (CAAP - ACT) • Measure of Academic Proficiency and Progress (MAPP - ETS) • iSkills (formerly ICT Literacy Assessment - ETS) • Standardized Assessment of Information Literacy Skills (SAILS – Kent State) • WorkKeys (ACT)

31. Components of Truman's Assessment Program.  Click on an instrument in the left column to learn more. Nationally Normed/Criterion Referenced Assessment Instruments – Subject Specific • Area Concentration Achievement Test (ACAT) • Major Field Test (MFT – ETS) • Praxis II (ETS) • A vast array of subject specific exams are administered through accrediting bodies.

32. Components of Truman's Assessment Program.  Click on an instrument in the left column to learn more. Nationally Normed Assessment Instruments – Self Report • National Survey of Student Engagement (NSSE) • College Student Experiences Questionnaire (CSEQ) • Cooperative Institutional Research Program (CIRP) Freshman Survey (HERI) • College Senior Survey (CSS – HERI)

33. Components of Truman's Assessment Program.  Click on an instrument in the left column to learn more. Local Instruments – Some Examples • Portfolios • Exit Surveys • Capstone Projects • Juried Theses, Exhibits, Performances

34. What Do Our Constituents Think? Employers deem both multiple-choice tests of general content knowledge and institutional assessment that show how colleges compare in advancing critical-thinking skills of limited value for evaluating applicants’ potential for success in the workplace …faculty-assessed internships, community-based projects, and senior projects would be the most useful in gauging graduates’ readiness for the workplace. P. Hart Research Associates, AAC&U (January, 2008)