Designing Valid Assessments of Student Learning

Designing Valid Assessments of Student Learning Rodolfo Rincones CETaL Workshop # 1 Enhancing Student Learning UTEP Library - Blumberg Auditorium September 3, 2009

Outline Context of assessment in higher education Purpose of assessment Linking learning and assessment Types of assessment

Activity What we know ( 5 minutes) What we want to know ( 5 minutes) What we learned (at the end 5 minutes)

Context of Higher Education Decreasing public funding sources Tight budgets Expanding pressures to do more with less Escalating tuition Public trust Increase productivity Accountability

Context of Higher Education • Factory-production model based on semester credit hours, certification, transfer, articulation, and “student success” • Limited records that reflect actual student learning outcomes • Paradigm shift from teaching to learning Donohue-Lynch, B. Assessment with 21st Century Tools http://www.carnegiefoundation.org/perspectives/sub.asp?key=244&subkey=2140

Are we assessing student learning to appease a growing tide of legislators and government executives who increasingly question the full impact of colleges and universities on student learning? or Are we assessing student learning to improve our own educational practices, curriculum choices, and instruction? McKitrick, Sean , Assistant Provost for Curriculum, Instruction, & Assessment at Binghamton University (SUNY)

Undergraduate education in research universities requires renewed emphasis on a point strongly made by John Dewey almost a century ago: Learning is based on discovery guided by mentoring rather than on the transmission of information. Inherent in inquiry-based learning is an element of reciprocity: Faculty can learn from students as students are learning from faculty The Boyer Commission on Educating Undergraduates in the Research University, 1998

Today’s Learning Today Learning in the 21st Century

Curriculum • “Curriculum is the formal and informal content and process by which learners gain knowledge and understanding, develop skills, and alter attitudes, appreciations, and values under the auspices of that school.” (Keller,2004)

Creating the learning environment • Although shared knowledge is an important component of a university education, no simple formula of courses can serve all students in our time. • Integrated educational experience • Collaborative learning experiences • Promote analysis, synthesis, and evaluation The Boyer Commission on Educating Undergraduates in the Research University, 1998

Teaching • Involves developing the knowledge, skill, mind, character, or ability of others. It “means not only transmitting knowledge, but transforming and extending it as well.” • “Dynamic endeavor involving all the analogies, metaphors, and images that build bridges between the teacher’s understanding and the student’s learning. Pedagogical procedures must be carefully planned, continuously examined, and relate directly to the subject taught.” (Boyer 1990)

Learner-Centered Teaching How to tie teaching, curriculum, and objectives of learning rather than to the content delivery alone. Focuses attention on what the student is learning, how the student is learning, the conditions under which the student is learning, whether the student is retaining and applying the learning, and how current learning positions the student for future learning. (Weimer, 2002)

Purposes of educational assessment • Assessment is a comparison of achieved results to intended goals (Ewell, 1985) • Traditional • Diagnose students’ strengths and weaknesses • Monitor students’ progress • Assign a grade • Determine program and institutional effectiveness

Purposes of educational assessment Formative - to aid learning Summative - for review, transfer and certification Summative - for accountability to the public

Reasons for Assessing Shupe, D. (2006). Toward a higher standard: the changing organizational context of accountability for educational results, ON THE HORIZON, Vol. 16, No. 2 2008, pp. 72-96.

Student Learning Outcome Student learning outcomes should refer normally to competencies or attainment levels reached by students on completion of an academic program

Courses must define two specific sets of learning objectives • Traditional knowledge and skills associated with the course • Abilities addressed in the course. (Keller,2004)

Abilities Integrated Developmental Transferable • Social Interaction • Global Perspectives • Effective Citizenship • Aesthetic Responsiveness • Communication • Analysis • Problem Solving • Valuing in Decision-Making

Learning Assessment What is meant by “assessment” often varies greatly—embracing everything from job placement, through student satisfaction, to self-reported gains in skill and knowledge on the part of students and former students. (Ewell, 2001)

Types of Measures • Direct Measures- provide clear and compelling evidence of what students are learning • Course-embedded assessments, including written work and presentations scored using a rubric • Scores on locally designed tests and competency exams accompanied by test “blueprints” describing what is being assessed • Score gains between entry and exit on tests, competency exams and writing samples • Ratings of student skills in the context of class activities, projects and discussions • Portfolios of student work • Scores on nationally- norm instruments (Ewell, 2001)

Types of Measures • Indirect Measures-Provide signs that students are probably learning, but it is less clear exactly what they are learning. • Student grades • Student evaluations and ratings of the knowledge and skills they have gained • Student or graduate satisfaction with their learning in general education competencies • Results of nationally-norm surveys (Ewell, 2001)

Assessment Evaluation Criteria • Reliability • Consistency with which assessment produce measures whatever it is measuring • Validity • Reflects the defensibility of score-based inferences made on the basis of an educational assessment procedure

Assessment Evaluation Criteria • Absence-of-Bias • Degree to which assessments are free of elements that would offend or penalize examinees on the basis of examinees’ gender, ethnicity or other characteristics (Popham, 2002)

Learning Objectives • Define a course in terms of the outcomes the instructor expects students to achieve • Components • A description of what the student will be able to do (verb) • The conditions under which the student will perform the task • The criteria for evaluating student performance

Importance of Learning Objectives Selection of content Development of an instructional strategy Development and selection of instructional materials Construction of tests and other instruments for assessing and then evaluating student learning outcomes Efficient use of instructional time Ongoing improvement and student feedback

Example Objective: Given a set of data the student will be able to compute the standard deviation. Condition - Given a set of data Behavior - the student will be able to compute the standard deviation. Criterion - the number computed will be correct

Course Assessment andEnhancement Model Combs, K., et al.(2008). Enhancing curriculum and delivery: linking assessment to learning objectives. Assessment & Evaluation in Higher Education, Vol. 33, No. 1, pp. 87–102.

Your turn…. • Connecting curriculum elements

Thank you!!!

Designing Valid Assessments of Student Learning