Automated Scoring of Open-ended Ethics Questions. Kelly Laas Alan D. Mead Illinois Institute of Technology. Agenda. Importance of ethics and ethics assessment Use of scenario-based assessment and open-ended responses Research Goal Method Results Future Directions.
Alan D. Mead
Illinois Institute of Technology
Davis, Michael. “Instructional Assessment in the Classroom: Objectives, Methods and Outcomes.” in Practical Guidance on Science and Engineering Ethics Education for Instructors and Administrators. Washington D.C.: National Academies Press, 2013. p. 30.
“At the beginning of the semester, IPRO 111 went on a tour of Gelco Inc., a company who is one of the largest widget producers in North America. The goal of IPRO 111 was to assist Gelco in developing a more efficient way of shipping widgets …”
“Ethical problems are “ill-structured: because there is often no clearly specific goal, only incomplete information, several possible solutions, and several parts to reach each. Since a single, simple response is not an option (or at least not a good one), students must investigate the problem, seek relevant information, consider alternative possible solutions, and evaluation short and long term consequences.”
Keefer, M.W. and Davis, M. (2012).“Curricular Design and Assessment in Professional Ethics.” Teaching Ethics, 13(1): 89.
= argmax P(item|domain)P(domain)/P(item)
= argmax P(item|domain)P(domain)
= argmax P(w1,w2,…,wn|domain)
≈ argmaxP(w1|d.)P(w2|d.) …P(wn|d.)