Fall Faculty Workshop August 21, 2008 Assessment [ CLA ]

1. Fall Faculty Workshop August 21, 2008 Assessment [CLA]

2. Why Assessment • Four Approaches to Assessment • The Collegiate Learning Assessment [CLA] • Using the CLA at Morningside

3. Why Measure Educational Outcomes? • We measure what we value, we value what we measure • Improve quality of student learning by creating a culture of assessment • Informing pedagogy and curricular design • Making assessment integral part of teaching and learning • Create formative evaluation for the institution, programs, major • Enable conversations about academic expectations and standards CAE –[CLA] slide

4. Why Measure Educational Outcomes? • Given accountability pressures, if we don’t initiate, it will be done for us CAE –[CLA] slide

5. Four Approaches • Actuarial indicators (graduation rates, access) • Quality rankings (US News & World Report) • Student surveys (NSSE, CIRP) • Direct measures of student learning [CLA] CAE –[CLA] slide

6. Four Approaches: Actuarial Indicators • What are the admissions test scores of entering students? • What percent of students graduated? • How diverse is the student body? • For example, see U-Can at: http://www.ucan-network.org/searchInstitution.asp CAE –[CLA] slide content

7. Four Approaches: Surveys To what extent has your experience at this institution contributed to your ability to think critically and analytically? Very Much  Quite a Bit  Some  Very Little  For example: The National Survey of Student Engagement (NSSE) http://www.usatoday.com/news/education/2007-11-04-nsse-how-to_N.htm?loc=interstitialskip CAE –[CLA] slide content

8. Four Approaches: Rankings • Question for presidents, provosts and deans of admission: rate the academic program of other universities, where 1 = marginal and 5 = distinguished. • Measuring value-added: what percentage of your students did you expect to graduate, and what percentage actually did? • For example: US News http://colleges.usnews.rankingsandreviews.com/usnews/edu/college/rankings/brief/t1ccbach_mw_brief.php CAE –[CLA] slide content

9. Current Approaches: Direct Measures • Measures that assess what students are actually able to do For example, The Collegiate Learning Assessment [CLA] CAE –[CLA] slide

10. The CLA measures four sets of higher order skills The CLA measures in a holistic manner: Critical thinking Analytic reasoning Problem solving Written communication CAE –[CLA] slide

11. CLA Scoring CriteriaCritical Thinking, Analytic Reasoning, and Problem Solving Skills • Evaluation of Evidence • Analysis and Synthesis of Evidence • Drawing Conclusion • Acknowledging Alternative Explanations/Viewpoints CAE –[CLA] slide

12. CLA Scoring CriteriaWriting Skills • Presentation • Development • Persuasiveness • Mechanics • Interest CAE –[CLA] slide

13. The CLA gets at some of our educational outcomes: • Demonstrate analytic, synthetic, creative, evaluative, and quantitative thinking. + • Communicate effectively + • Behave ethically and responsibly - • Use knowledge of cultures to enhance understanding of themselves and others - • Apply knowledge and skills from multiple, diverse disciplines and practical experiences to understand complex issues and solve problems ? • Exhibit a passion for life-long learning - • Effect positive change through leadership or active participation in communities - • Articulate their own spirituality and values, while understanding those of others -

14. There are three types of CLA measures Analytic Writing Task: Make-an-Argument Analytic Writing Task: Critique-an-Argument Performance Task CAE –[CLA] slide

15. Analytic Writing Task: “Make-an-Argument” Writing a persuasive, analytic essay to support a position on an issue CAE –[CLA] slide

16. Make-an-Argument “Government funding would be better spent on preventing crime than in dealing with criminals after the fact.” Directions: 45 minutes, present your perspective on the issue, using relevant reasons and/or examples to support your views CAE –[CLA] slide

17. Government Funding Establish a thesis Maintain the thesis Support the thesis with examples Anticipate and counter opposing arguments CAE –[CLA] slide

18. Analytic Writing Task: “Critique-an-Argument” Critiquing Written Arguments CAE –[CLA] slide

19. Analytic Writing Task: “Critique-an Argument” “Butter has now been replaced by margarine in Happy Pancake House restaurants throughout the southwestern United States. Only about 2 percent of customers have complained, indicating that 98 people out of 100 are happy with the change. Furthermore, many servers have reported that a number of customers who still ask for butter do not complain when they are given margarine instead. Clearly, either these customers cannot distinguish margarine from butter, or they use the term "butter" to refer to either butter or margarine. Thus, to avoid the expense of purchasing butter, the Happy Pancake House should extend this cost-saving change to its restaurants in the southeast and northeast as well.” Directions: 30 minutes, discuss what is wrong with the argument CAE –[CLA] slide

20. The Happy Pancake House Identify logical flaws or fallacies How do these logical flaws influence the argument’s conclusion CAE –[CLA] slide

21. Performance Task Analyzing complex, realistic scenarios CAE –[CLA] slide

22. Performance Task • Features: • Open-Ended • Document-Based • “Real Life” Scenarios • Components: • Document Library CAE –[CLA] slide

23. Crime Reduction “Pat Stone is running for reelection as the mayor of Jefferson, a city in the state of Columbia. Mayor Stone’s opponent in this contest is Dr. Jamie Eager. Dr. Eager is a member of the Jefferson City Council. You are a consultant to Mayor Stone. Mayor Stone wants to make sure the best policy for reducing crime in Jefferson is identified, so has asked you to analyze the strengths and/or limitations of Dr. Eager’s proposal.” Directions: 90 minutes, use the evidence from the Document Library to answer the following questions. CAE –[CLA] slide

24. Performance Task Synthesize information from multiple sources Recognize conflicting evidence Interpret data, tables, figures correctly Identify logical fallacies Develop conclusions based on available evidence CAE –[CLA] slide

25. Crime Reduction • Student is advising the mayor, who is running for re-election. • There is an upcoming mayoral debate, for which the student must help the incumbent mayor prepare. CAE –[CLA] slide

26. Crime Reduction • There are two policy approaches for reducing crime: • Drug education program • Increasing number of police officers on the streets CAE –[CLA] slide

27. Crime Reduction • Documents available to the student include: • Newspaper article about crime in the community • Research abstracts about drug education program • Report about success of a drug education program in another community • Police report (with table of data) about crime and drug use in the community • Plots of the relationship between police offers and crime • Private investigator report about possible connection between opponent and drug education program CAE –[CLA] slide

28. Crime Reduction • Using these data, what does the student advise that the mayor should do? CAE –[CLA] slide

29. Performance Tasks • Not all quantitative • Artwork and Technology CAE –[CLA] slide

30. At Morningside College • Invited to join the CIC-CLA consortium • approx 40 colleges • four year commitment • Will administer the CLA to at least 100 first year students and 100 seniors each year • This year all 300 first year students • Share data and work with the consortium CAE –[CLA] slide

31. CIC-CLA Consortium Alaska Pacific University Allegheny College Aurora University Averett University Barton College Bethel University (MN) Cabrini College Carlow University Charleston Southern Univ. College of Notre Dame (MD) College of St. Benedict/Saint John’s University Dominican University Drake University Franklin Pierce University Hilbert College Illinois College Indiana Wesleyan Univ. Jamestown College John Carrol Univ. Juniata College LaGrange College Lynchburg College Marion University (WI) Morningside College Nebraska Wesleyan Univ. Pace University Pacific University Seton Hill University Southwestern University (TX) Springfield College Stephens College Stonehill College Texas Lutheran University The College of St. Scholastica University of Findlay Trinity Christian College University of Charleston University of Evansville University of Great Falls Upper Iowa University Ursinus College Ursiline College Wagner College Wartburg College Westminster College (MO) Westminster College (UT) Williamette University William Woods University

32. Questions? Other Options

34. CLA in the Classroom helps address the “What Now?” question The key to accountability is not just transparency, but also improvement CLA in the Classroom connects the institution-wide CLA results to the classroom-level and faculty work on student learning A means to link institution-wide summative assessment with local, formative work

35. One component of the program supports diagnostic work Use of a retired (“[cla]ssic”) performance task in a classroom, with administration, scoring and advising resources In addition, classroom activities are suggested Designed to provide diagnostic feedback to students to understand why they achieved the scores they did, and what to do next to improve their skills

36. Another key component enables curricular efforts In addition, faculty walk through a process to develop their own performance tasks Faculty can embed course content in the tasks they create These faculty-developed tasks could be used as class assignments or group projects

37. CLA in the Classroom Academy Gain a deeper understanding of the CLA Learn how to administer and score a disclosed CLA Performance Task Develop performance tasks where course content can be embedded Learn more about rubric-based assessment as it applies to the CLA Brainstorm strategies for using these approaches in courses or across the institutions