1 / 52

Student Learning Outcomes Assessment Workshop

Student Learning Outcomes Assessment Workshop. College of Science & Mathematics May 9, 2003 Val Whittlesey, Bill Hill, & Ed Rugg Assurance of Learning Council. Goals for an Assessment Plan. To articulate as department members what our expected goals and outcomes are for our graduates

paul
Download Presentation

Student Learning Outcomes Assessment Workshop

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Student Learning Outcomes Assessment Workshop College of Science & Mathematics May 9, 2003 Val Whittlesey, Bill Hill, & Ed Rugg Assurance of Learning Council

  2. Goals for an Assessment Plan • To articulate as department members what our expected goals and outcomes are for our graduates • To evaluate whether we are accomplishing these goals through the curriculum as a whole and within courses that address specific outcomes

  3. Goals for an Assessment Plan • To use assessment data to guide improvements/enhancements to the curriculum and instruction • To report to external stakeholders [students, employers, the Board of Regents, accreditation agencies (e.g., SACS)]

  4. Student Learning is Job 1 “Assessment should be first and foremost about improving student learning and secondarily about determining accountability for the quality of learning produced.” Angelo, 1999

  5. Assessment Should be Student-Centered “Student’s academic, attitudinal, and behavioral outcomes are the primary source of assessment data … (and) provide information about the effectiveness of the curriculum, advising, faculty development, ethnic diversity goals, and external linkages with other segments of education.” Halpern et al., pp. 24-25

  6. Elements of an Effective Assessment Plan • Articulating Student Learning Outcomes • Connecting the Outcomes to the Curriculum • Connecting Outcomes with Assessment Methods • Articulating your Expected Results with Respect to the Outcomes • Articulating the Research/Assessment Plan for Gathering the Data • Collecting & Analyzing the Data • Comparing Actual Results with Expected Results • Using Results for Improvement

  7. Step 1- Articulating Student Learning Outcomes • There are three types of learning outcomes • Knowledge • Skills/Proficiencies • Attitudinally-Based

  8. Step 1- Articulating Student Learning Outcomes • There are two levels of specificity at the program level (the same is true at the course level) • Broad learning outcomes • Specific learning outcomes

  9. Step 1- Articulating Student Learning Outcomes • Where available, use disciplinary or national models when developing learning outcomes; however, we may choose to adapt a national model of learning outcomes to fit our unique niche or mission

  10. Step 1- Articulating Student Learning Outcomes • State Outcomes in measurable terms so that it can be determined if achievement of the learning outcomes has been achieved. • Although broad outcomes can be stated in measurable terms they are still difficult to assess because of their breath. Therefore, because specific outcomes should be the primary focus of assessment, they must be measurable.

  11. Step 1- Articulating Student Learning Outcomes • Applying Bloom’s taxonomy (1956) • Original taxonomy divides thinking skills into lower and higher level knowledge, skills, and attitudes. • Anderson and Krathwohl (2001) have adapted Bloom’s model to more outcome-oriented, contemporary language. • Clark (2002) shows how Bloom’s work can be used to develop specific learning outcomes.

  12. Group Exercise 1Articulating Specific Student Learning Outcomes • Groups receive a broad learning outcome • Each individual should brainstorm as many specific learning outcomes as possible for the broad learning outcome. • Share individual specific learning outcomes and generate consensus list of outcomes. • Record the group’s list of specific learning outcomes on an overhead to be shared.

  13. Some Examples of Learning Outcomes

  14. PsychologyBroad Outcomes Outcome Category 1: Knowledge, Skills, and Values Consistent with the Science and Application of Psychology • Goal 1:Knowledge Base of Psychology • Goal 2: Research Methods in Psychology • Goal 3: Critical Thinking Skills in Psychology • Goal 4: Application of Psychology • Goal 5: Values in Psychology

  15. PsychologyBroad Outcomes Outcome Category 2: Knowledge, Skills, and Values Consistent with Liberal Arts Education that are Further Developed in Psychology • Goal 6: Information and Technological Literacy • Goal 7: Communication Skills • Goal 8: Sociocultural and International Awareness • Goal 9: Personal Development • Goal 10: Career Planning and Development

  16. PsychologyBroad Outcomes Example Broad Learning Outcome: Goal 2. Research Methods in Psychology Understand and apply basic research methods in psychology, including research design, data analysis, and interpretation

  17. PsychologySpecific Outcomes 2.1 Describe the basic characteristics of the science of psychology

  18. PsychologySpecific Outcomes 2.2 Explain different research methods used by psychologists • Describe how various research designs address different types of questions and hypotheses • Articulate strengths and limitations of various research designs • Distinguish the nature of designs that permit causal inferences from those that do not

  19. PsychologySpecific Outcomes 2.3 Evaluate the appropriateness of conclusions derived from psychological research • Interpret basic statistical results • Distinguish between statistical significance and practical significance • Describe effect size and confidence intervals • Evaluate the validity of conclusions presented in research reports

  20. PsychologySpecific Outcomes 2.4 Design and conduct basic studies to address psychological questions using appropriate research methods • Locate and use relevant databases, research, and theory to plan, conduct, and interpret results of research studies • Formulate testable research hypotheses, based on operational definitions of variables • Select and apply appropriate methods to maximize internal and external validity and reduce the plausibility of alternative explanations

  21. PsychologySpecific Outcomes 2.4 Design and conduct basic studies to address psychological questions using appropriate research methods • Collect, analyze, interpret, and report data using appropriate statistical strategies to address different types of research questions and hypotheses • Recognize that theoretical and sociocultural contexts as well as personal biases may shape research questions, design, data collection, analysis, and interpretation

  22. PsychologySpecific Outcomes 2.5 Follow the APA Code of Ethics in the treatment of human and nonhuman participants in the design, data collection, interpretation, and reporting of psychological research

  23. PsychologySpecific Outcomes 2.6 Generalize research conclusions appropriately based on the parameters of particular research methods • Exercise caution in predicting behavior based on limitations of single studies • Recognize the limitations of applying normative conclusions to individuals • Acknowledge that research results may have unanticipated societal consequences

  24. PsychologySpecific Outcomes 2.6 Generalize research conclusions appropriately based on the parameters of particular research methods • Recognize that individual differences and sociocultural contexts may influence the applicability of research findings

  25. MS Primary Care Nurse PractitionerBroad Outcome Example Broad Learning Outcome: Goal 2. Make Clinical Decisions The graduate is able to use critical thinking, clinical judgment skills and ethics to make safe independent and interdependent clinical decisions, incorporating principles of health promotion, awareness, diagnosis and management of client potential and actual health care problems

  26. MS Primary Care Nurse PractitionerSpecific Outcomes 2.1 Provide anticipatory guidance and counseling to promote health, reduce risk factors, and prevent disease and disability

  27. MS Primary Care Nurse PractitionerSpecific Outcomes 2.2 Employ appropriate diagnostic and therapeutic interventions and regimens with attention to safety, cost, invasiveness, simplicity, acceptability, adherence, and efficacy

  28. MS Primary Care Nurse PractitionerSpecific Outcomes 2.3 Formulate an action plan based on scientific rationale, evidence-based standards of care, and practice guidelines

  29. MS Primary Care Nurse PractitionerSpecific Outcomes 2.4 Initiate appropriate and timely consultation and/or referral when the problem exceeds the nurse practitioner’s scope of practice and/or expertise

  30. MS Primary Care Nurse PractitionerSpecific Outcomes 2.5 Assess client’s motivation for learning and maintenance of health related activities using principles of change and stages of behavior change

  31. Group Exercise 2Connecting Learning Outcomes

  32. Step 2 - Connecting Student Learning Outcomes and the Curriculum

  33. Group Exercise 3Constructing an Outcome-Curriculum Matrix An outcome-curriculum matrix allows one to analyze the relationship between a program’s curriculum and its specific expected learning outcomes for the program.

  34. Outcome-Curriculum Matrix for an Academic Program

  35. Outcome-Curriculum Matrix for an Academic Program Some Recommendations: • Focus on Specific Learning Outcomes. • Only list specific required courses or course groupings. • Outcomes may be accomplished in a single or multiple courses.

  36. The KSU Assurance of Learning Council • The Council is asking departments to focus on Steps 1 and 2 for each of their academic programs for the fall of 2003 • Some departments may want to move to other steps in the outcomes assessment plan process during the fall

  37. Assessment as Research & Scholarship Assessment is Research!! • Dependent Variables: Student Learning Outcomes • Independent Variables: The Curriculum • Hypothesis: Articulation of Expected Results • Method: Designing your research through specifying assessment methods (DV measures) and data gathering procedures • Results: Statistical analyses of assessment data • Discussion: Evaluation of your hypothesis (comparison of actual with expected results) and projected use of results for improvement.

  38. Assessment as Research & Scholarship • Assessment in the discipline is an opportunity to engage in applied scholarship. • Assessment is also the scholarship of teaching and learning. • Multiple venues exist to present and publish this scholarship.

  39. Step 3-Connecting Learning Outcomes and Assessment Methods • General Categories of Assessment Methods • Select assessment methods that are appropriate for specific learning outcomes • A matrix can be constructed connecting learning outcomes to assessment methods

  40. The Assessment Method Search “There is no single assessment method appropriate for all departments and institutions. The search for a perfect assessment plan is like the quest for the Holy Grail.” Halpern et al., p. 26.

  41. Step 4-Articulation of Expected Results Prior to the collection of assessment data, the criterion for determining successful performance on each assessment method for each learning outcome should be determined

  42. Step 5-The Research Plan • A plan concerning the collection of assessment data should be done • The plan should address aspects such as: • What you will assess • When and where the data will be collected (a timetable, which may be a multi-year plan with different assessment methods or different learning outcomes emphasized in different years) • Who collects the data • Resources (e.g., materials, reassigned time for the researcher) needed for data collection • Sampling issues

  43. Step 6-Assessment Results and Analysis The data should be collected, analyzed and evaluated

  44. Step 7-Comparison of Data to Expected Results • Comparison of data analysis to the expected results for each learning outcome • A matrix can be constructed comparing the actual data to the expected data

  45. Step 8-Use of the Results for Improvement Several decisions can be made based on the assessment data • No action is needed--student achievement reached or exceeded the expected result • Curriculum changes (program requirements or individual courses) • Use of results: modification of assessment plan (learning outcomes, assessment methods, expected results)

  46. Where Do You Go from Here? • Involve the faculty!! • Allow sufficient time to discuss and agree upon learning outcomes. • See it as a constantly changing and evolving process. It NEVER ends!!! • Provide resources and time to the departmental faculty leading the effort. • Tie this to scholarship.

  47. Role of the Assurance of Learning Council Facilitate preparation for changes in SACS Guidelines (effective 2004) • Switch from “Criteria of Accreditation” to “Principles of Accreditation” • Comprehensive Standard #1 states that “an educational program for which academic credit is awarded should establish and evaluate learning outcomes” • Core Requirement #12 requires institutions to “develop an acceptable Quality Enhancement Plan that improves the university’s learning environment supporting student achievement”

  48. Role of the Assurance of Learning Council Review and provide feedback of program assessment efforts

  49. Degree Program: ___________________________________ Student Learning Outcomes Assessment Evaluation Form Please rate each of the following elements below. I. Student Learning Outcomes Very Weak Weak Satisfactory Strong Very Strong II. Matrix of Learning Outcomes Very Weak Weak Satisfactory Strong Very Strong & the Curriculum III. Matrix of Learning Outcomes & Very Weak Weak Satisfactory Strong Very Strong Assessment Methods IV. Articulation of Expected Results Very Weak Weak Satisfactory Strong Very Strong V. Assessment Results, & Analyses Very Weak Weak Satisfactory Strong Very Strong VI. Actual Results Compared Very Weak Weak Satisfactory Strong Very Strong with Expected Results VII. Use of Results for Improvement Very Weak Weak Satisfactory Strong Very Strong

More Related