Developing and Assessing Students’ Complex Reasoning Abilities

1. Developing and Assessing Students’ Complex Reasoning Abilities Applications of the Reflective Judgment Model 2008 International Assessment & Retention Conference

2. Director of Student Learning Assessment Metropolitan State College of Denver sthomp83@mscd.edu 303.352.4495 Sheila S. Thompson, PhD

3. What are your objectives for this session?

4. My objectives • Facilitate your comprehension of the Reflective Judgment Model • Engage you in interactive exercises that illustrate practical applications of the model • Assist you in understanding how the RCI can provide useful data for developing strategies to enhance students’ complex reasoning abilities

5. Reflective Judgment Model • Describes a developmental progression that occurs between childhood and adulthood in the ways that individuals view knowledge and justify their beliefs about ill-structured problems (epistemic cognition) • Developed by Patricia King (University of Michigan) and Karen Kitchener (formerly University of Denver) • More information at http://www.umich.edu/~refjudg/index.html • Epistemology defined as the study of the origin, nature, limits, methods, and justification of human knowledge (Hofer and Pintrich, 2002)

6. Support for this model • Pascarella and Terenzini (How College Affects Students, 1991, p. 23) “the best known and most extensively studied” model of postformal reasoning/critical thinking • Nuhfer and Pavelich (The National Teaching and Learning Forum, 2001, p. 9) “the most solidly backed by extensive data derived from students at many schools, from many disciplines, and levels through doctoral students” • Hofer and Pintrich (Personal Epistemology, 2002, p. 6) “one of the most comprehensive bodies of work in the study of intellectual development”

7. Reflective Judgment LevelsLynch, Kitchener, King (1995) Pre-Reflective • Level 1: I know what I have seen. • Level 2: If it is on the news, it has to be true. • Level 3: When there is evidence that people can give to convince everybody one way or another, then it will be knowledge; until then it’s just a guess.

8. Reflective Judgment Levels (cont.) Quasi-Reflective • Level 4: I’d be more inclined to believe evolution if they had proof. It’s just like the pyramids: I don’t think we’ll ever know. Who are you going to ask? No one was there. • Level 5: People think differently and so they attack the problem differently. Other theories could be as true as my own, but based on different evidence.

9. Reflective Judgment Levels (cont.) Reflective • Level 6: It’s very difficult in life to be sure. There are degrees of sureness. You come to a point at which you are sure enough for a personal stance on an issue. • Level 7: One can judge arguments by how well thought out the positions are, what kinds of reasoning and evidence are used to support it, and how consistent the way one argues on this topic is as compared to other topics.

10. Exercise One • Review examples of student thinking on handouts • In small groups, discuss where these might fit in terms of the three broad reflective judgment levels using level summary as a guide • Look for key words or phrases that may assist you in identifying a level

11. Some economic experts claim that a less restrictive immigration policy adds to the overall economic prosperity of the United States.  Admission of new immigrants, they argue, expands the tax base and economic competitiveness of American products and services.  Other economic experts suggest that such policies result in a drain on the medical, financial and educational resources of the United States.  These experts argue that a less restrictive immigration policy harms the economic well-being of the country. Experts disagree about this issue because, like everyone else, they are confused about the role that immigration policy plays in economic prosperity.  So what they conclude is just their opinion. Experts disagree because of the different ways they were brought up and/or the different schools they attended. Experts arrive at different conclusions because the evidence itself is complex and they examine it from several perspectives.  They arrive at a decision based on synthesizing their knowledge, experience and other expert opinions. Assessing Reflective Judgment: Sample RCI item

12. Some RCI Data (1999-2005) • These scores are about one level higher than scores obtained from the Reflective Judgment Interview that was originally used to measure reflective judgment.

13. Exercise Two • Review characteristics and examples of ill-structured problems • Identify two or three ill-structured problems in your student service area or academic discipline • Talk about these with your colleagues

14. Exercise Three • Review the educational strategies described for assisting students in moving through the reflective judgment levels typical for undergraduates • Using an ill-structured problem you previously identified, create an activity or assignment for your students

15. Challenge and Support(Lynch, Kitchener and King, 1995) • Assign tasks that actively engage students in thinking about ill-structured problems • Pay careful attention to the sequence in which you introduce challenges • Provide feedback to students • Provide adequate opportunities for students to practice relevant tasks