ASSESSMENT OF MORAL REASONING AT TWO FAITH BASED INSTITUTIONS

1. ASSESSMENT OF MORAL REASONING AT TWO FAITH BASED INSTITUTIONS David W. Kale, Ph.D. Director of Assessment, Mount Vernon Nazarene University dkale@mvnu.edu Joel Frederickson, Ph.D. Chair, Psychology Dept., Bethel University frejoe@bethel.edu

2. ASSESSMENT OF MORAL REASONING BETHEL UNIVERSITY

3. Moral Thinking: Using the DIT-2 as a Measure of Cognitive Development

4. What does the DIT-2 measure? • A revised measure of moral reasoning based on Kohlberg’s theory of moral development. • Focus is on schemas NOT stages. • Five scenarios (original DIT had six). • Strong correlations between DIT-1 and DIT-2 (r=.79). • Example Dilemma from original DIT: Heinz & the drug

5. Schema Scores • Personal Interest Schema Score: The proportion of items selected that appeal to stage 2 & 3 thinking. Stage 2 focuses on the direct advantages to the actor and on the fairness of simple exchanges of favor for favor. Stage 3 focuses on the good or evil intentions of the parties; concern for maintaining friendships and approval.

6. Schema Scores • Maintaining Norms Schema Score. The proportion of items that appeal to stage 4 thinking. Focus on maintaining existing legal system, existing roles and organizational structure.

7. Schema Scores • Postconventional Schema Score (P score) Focus on organizing society by appealing to consensus-producing procedures (majority vote), insisting on due process, and safeguarding basic rights. Organizing social arrangements & relationships in terms of intuitively appealing ideals.

8. Example • Famine Scenario: Read through this scenario and the accompanying “issues” related to the dilemma. • Place the “issues” into the categories they are measuring • Personal Interests • Maintaining Norms • Postconventional Thinking • Meaningless/pretentious items

9. New Index (N2) • N2 score is a new index and is considered superior to the P score (post conventional reasoning score). • There are two parts to this score: the degree to which respondents support post conventional responses (P score), plus the degree to which personal interest items receive lower ratings. • Essentially, the N2 is a score that reflects the degree to which respondents will reject bad arguments (personal interest arguments).

10. Validity & Reliability of DIT • Differentiation of various age/education groups: 30% to 50% of the variance in DIT scores is attributable to level of education.

11. Education and Moral Thinking • As education level goes up, degree of postconventional thinking increases.

12. Validity & Reliability of DIT • Differentiation of various age/education groups: 30% to 50% of the variance in DIT scores is attributable to level of education. • Longitudinal gains: Reviews of a dozen studies of Freshman to Senior college students show effect sizes of .80, one of the most dramatic effects of college.

13. Validity & Reliability of DIT • Sensitive to moral education interventions • Reliability is adequate (Test-retest and Cronbach alphas in upper .70s, lower .80s).

14. Use of the DIT at Bethel for Assessment • Look at cross-sectional and longitudinal differences from Freshman to Senior year. • We have also analyzed these Freshman to Senior changes by department. • Example: Business department typically had low growth. They made some changes to the curriculum (and faculty) and saw better growth in moral reasoning.

15. Use of the DIT at Bethel for Assessment • Usually we see typical growth from Freshman to Senior year (.70-.80 effect sizes). • A small cohort in our degree completion Business Management program saw similar growth. • Just beginning to get longitudinal data for our MBA program.

16. Use of the DIT at Bethel for Assessment • Colleague uses pre/post DIT in his course “Being Just in an Unjust World” (essentially a course in Moral Thinking). Typically sees a big change for a semester (about .70 effect size). • This is the amount of change you would typically see in 4 years!

17. Pre/post DIT N2 Scores by Type of Course

18. Use of the DIT at Bethel for Assessment • A small subset of students who had taken this course (N=23) took the DIT again four months later. • Students not only maintained growth, but showed a marginally significant increase.

19. ASSESSMENT OF MORAL REASONING AT MOUNT VERNON NAZARENE UNIVERSITY COMBINING THE QUANTITATIVE WITH THE QUALITATIVE

20. QUALITATIVE ASSESSMENT • The Potter Box Definition of situation Loyalties Principles Values Ralph B. Potter, “The Logic of Moral Argument” in Towards a Discipline of Social Ethics, ed. Paul Deats (Boston Univ. Press, 1972).

21. DEFINITION OF THE SITUATION • The letter is the engineer’s personal and professional correspondence. • The letter is privileged communication between the engineer and his attorney. • What other aspects of the situation would you consider important to consider?

22. VALUES • Privacy • Truth • What other values do you see? • What is your highest value?

23. PRINCIPLES Always tell the truth

24. PRINCIPLES • Tell the truth • Protect the privacy of others

25. LOYALTIES • To whom is moral duty owed? - The 17 year old - The engineer - Who else has a stake in this situation that needs to be protected? • To whom do you have the highest loyalty?

26. MORAL AND ETHICAL THEORIES • Aristotle’s Golden Mean • Biblical concept of love • Utilitarianism • Rawls’ Theory of Justice • Kant’s Categorical Imperative

27. BUILDING A RUBRIC FOR STUDENT SELF ASSESSMENT • After dealing with several case studies, I ask students to answer two questions anonymously. • 1. In what ways have you improved in your moral and ethical reasoning ability? • 2. In what ways would you like to continue to improve?

28. BUILDING A RUBRIC FOR STUDENT SELF ASSESSMENT • I identified eight dimensions of moral reasoning in their answers. • 1. Using clearly worded arguments when I state my position; • 2. Making sure I have good support for my position (facts, reasoning, statistics) • 3. Taking the views of others into consideration when I make my decision. • 4. Supporting my arguments with biblical truth.

29. BUILDING A RUBRIC FOR STUDENT SELF ASSESSMENT • 5. Having a clear process for moving all the way to a decision; • 6. Making my decisions based on reason rather than letting my emotions get the upper hand; • 7. Taking my time rather than rushing into a decision. • 8. Staying well informed on the issues of the day.

30. BUILDING A RUBRIC FOR STUDENT SELF ASSESSMENT • I then have students vote on the top five. • 1. Using clearly worded arguments when I state my position. • 2. Making sure I have good support for my position (facts, reasoning, statistics). • 3. Taking the views of others into consideration. • 4. Supporting my position with biblical truth. • 5. Taking my time rather than rushing into a decision.

31. BUILDING A RUBRIC FOR STUDENT SELF ASSESSMENT • Building a rubric in this fashion is an assessment strategy in itself. • If there is an aspect of moral reasoning that does not turn up in their list that I think is important, that tells me either that they have not learned it or do not think it is nearly as important as I do.

32. QUANTITATIVE ASSESSMENT USING THE DIT-2 AT MVNU • On the traditional side, we administer the DIT-2 to our incoming first year students, to our Juniors on Testing Day and to our seniors at the end of the Senior Colloquium course. • We also administer the DIT-2 to our nontraditional students at the beginning and at the end of their degree completion programs.

33. N2 scores

34. Trad. And Non-Trad. StudentsPost conventional Reason. Scores

35. SUMMARY • There appears to be strong evidence from the use of the Defining Issues Test at two faith based institutions that a senior level Christian ethics course significantly improves students’ moral reasoning ability. • Using both quantitative and qualitiative measures provides richer data as to exactly what aspects of students’ reasoning has improved.