1 / 34

Two Levels of Course Assessment to Identify Student Learning

Two Levels of Course Assessment to Identify Student Learning. Carol Lerch, Ph.D lerch@dwc.edu S 52 Friday 11:45 – 12:35. Background. Assess new format – College Algebra Introduced fall 2000 Previous course failure rate +40% Looking at students in one specific major.

maisie-nicholson
Download Presentation

Two Levels of Course Assessment to Identify Student Learning

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Two Levels of Course Assessment to Identify Student Learning Carol Lerch, Ph.D lerch@dwc.edu S 52 Friday 11:45 – 12:35

  2. Background • Assess new format – College Algebra • Introduced fall 2000 • Previous course failure rate +40% • Looking at students in one specific major • Assess new format – College Algebra • Introduced fall 2000 • Previous course failure rate +40% • Looking at students in one specific major • Course description • Purpose of this course is to provide students with a proficiency in algebraic skills necessary for success in future coursework … • Assess new format – College Algebra • Introduced fall 2000 • Previous course failure rate +40% AMATYC 2007

  3. Collect Data • Specifics Fall 2001 – Summer 2007 • Pre-2000 60% pass (anecdotal) • Fall average 85% • Spring average 64% • Summer 80% • Includes Withdraws AMATYC 2007

  4. Analysis • Improvement over previous College Algebra course • Fall pass rates acceptable • Continuing to improve • Improvement over previous College Algebra course • Fall pass rates acceptable • Continuing to improve • Spring pass rates much lower • Needs analysis & new plan • Improvement over previous College Algebra course • Fall pass rates acceptable • Continuing to improve • Spring pass rates much lower • Needs analysis & new plan • Summer needs more data • Online first time summer 07 • Improvement over previous College Algebra course AMATYC 2007

  5. Changes to Course • Added text related course management system • online homework • Interactive textbook • Instant help • Video, animation • Added section JITTs • Just-in-time-tests • Added text related course management system • online homework • Interactive textbook • Instant help • Video, animation AMATYC 2007

  6. Purpose of AssessmentPlan • Pass rates • Pass rates for particular course • Course sequence • Semesters to complete math sequence • Thanks to Jack Rotman • Pass rates • Pass rates for particular course • Course sequence • Semesters to complete math sequence • Thanks to Jack Rotman • Concept assessment • Learning goals/objectives • Pass rates • Pass rates for particular course AMATYC 2007

  7. Semesters to Complete • Semesters to pass through the math sequence, passing MA152 for each semester since fall 03: • 82% of those who started in MA098 passed MA152 in 4 semesters (4 courses) • 93% of those who started in MA122 passed MA152 in 3 semesters (3 courses) AMATYC 2007

  8. Levels of Assessment • Gathering information about students’ achievement (Marzano, 2001, p. 115) • Level I Assessment • General Course information • Pass rates • Success in subsequent math courses • Semesters to complete sequence AMATYC 2007

  9. Levels of Assessment • Level II Assessment • Content specific • Concepts • Learning Map • Summative assessment • Overall course content • Final exam • Item Analysis • Formative assessment • Individual concepts tested AMATYC 2007

  10. Cycle of Assessment AMATYC 2007

  11. Plan • Online homework required as of spring 2006 • Introduced as option for fall 2005 • Daily quiz • 2-3 questions on concepts in each section • Read text • Come to class prepared • Carrot – replace test grade with quiz average AMATYC 2007

  12. Collect Data • Fall pass rate improved overall • 90% • Spring stayed same • 63% AMATYC 2007

  13. Analysis • Fall pass rates acceptable • Online homework helped • Anecdotal evidence • Students like all options available with online homework • Have more opportunities to learn • Spring rates need improvement AMATYC 2007

  14. Plan Plan • Assesses student learning • Concept assessment • Learning goals/objectives • Develop “Learning Map” • Identify major concepts • Identify teaching objectives • Identify student learning objectives • Means of assessment • Set success standard AMATYC 2007

  15. Learning Map AMATYC 2007

  16. Learning Map • 1st step • Major topics of course • Linear equations, quadratic equations, inequalities, absolute value, exponential & log equations • Modified as further developed • 2nd step • What do I want them to learn • Teaching objectives • 1st step • Major topics of course • Linear equations, quadratic equations, inequalities, absolute value, exponential & log equations • Modified as further developed AMATYC 2007

  17. Learning Map • Knowledge level • Cognitive Process • Action verbs AMATYC 2007

  18. Levels of Knowledge • Factual • Basic elements of the discipline • Facts specific to topic • Parts of linear or quadratic equation • Methods to solve quadratic equations • Factual • Basic elements of the discipline • Facts specific to topic • Parts of linear or quadratic equation • Methods to solve quadratic equations • Procedural • Processes of mathematics • Algorithms, techniques, methods • Complete the square • Quadratic formula AMATYC 2007

  19. Levels of Knowledge • Conceptual • Interrelationships • Classifications • Principles, generalizations • Pythagorean Theorem • Properties • Metacognitive • Self knowledge • Students need to reflect to learn • Conceptual • Interrelationships • Classifications • Principles, generalizations • Pythagorean Theorem • Properties AMATYC 2007

  20. Teaching vs. Learning Objectives • Student learning experience • What they will do • Teaching objective • What I will do • What I want the students to know • Student learning objective • “Students will be able to …” AMATYC 2007

  21. Assessment on Map • Each objective has generic question • Indicates how student is assessed AMATYC 2007

  22. Learning Map • Gave to students as part of syllabus • Type of questions included • ‘Truth in advertizing’ • Matched final exam to map • Redesigned map • Concepts tested but not part of map • Redesigned final exam • Some concepts not tested but important • Some concepts tested multiple times AMATYC 2007

  23. Level II Assessment • Assess concepts learned • Allow overall picture of student progress • Shows strengths • Identifies weaknesses • Areas for further development AMATYC 2007

  24. Collect Data • Collected chapter test data • Right/wrong only • Collected final exam item results • 4th semester of data collection • Stats: Percent correct AMATYC 2007

  25. Collect Data • Identified concept tested by semester • Shows what is not tested (over-tested) Results of chapter tests for Fall 06, Spring 07, Summer 07 AMATYC 2007

  26. Analysis - Tests • More Student Learning Objectives • Needed to add very specific objectives • Dropped some that I thought I wanted but don’t test for • More Student Learning Objectives • Needed to add very specific objectives • Dropped some that I thought I wanted but don’t test for • Success measure – 65% or better correct • Results below need attention • More Student Learning Objectives • Needed to add very specific objectives • Dropped some that I thought I wanted but don’t test for • Success measure – 65% or better correct • Results below need attention • 0 - 10% indicates problem with question as well as topic • More Student Learning Objectives • Needed to add very specific objectives • Dropped some that I thought I wanted but don’t test for • Success measure – 65% or better correct AMATYC 2007

  27. Analysis – Final exam • Final used by all sections • Data just my sections • Shows problems with online course on specific concepts AMATYC 2007

  28. Asides • Only list what is being assessed • If can’t assess, don’t put as objective • Have a thick skin • Data show changes made in course • Provides longitudinal look at course • Allows comparison of student results by semester or year • Data show where more emphasis needed in teaching AMATYC 2007

  29. Before Beginning Assessment • Purpose of assessment • What do you want to know? • Why do assessment? • Action leading from assessment • What do you do with results? AMATYC 2007

  30. 5 Basic Questions • Which objectives to assess? • Student learning • What should they know? • How to assess? • Multiple layers • What are results? • Collect/collate data • What is ‘success’? • What to do about results? • Next steps AMATYC 2007

  31. References • Angelo, T. A. (1991). Introduction and overview: From classroom assessment to classroom research. Classroom research: Early lessons from success. T. A. Angelo and K. P. Cross. San Francisco, Jossey-Bass: 7-15. • Brockbank, A. and I. McGill (1998). Facilitating reflective learning in higher education. Philadelphia, Society for Research into Higher Education & Open University Press. • Carson, T. (1990). "What kind of knowing is critical action research?" Theory Into PracticeXXIX(3): 167-173. • Cochran-Smith, M. and S. L. Lytle (1993). Inside outside: Teacher research and knowledge. New York, Teachers College. • Elliott, J. (1991). Action research for educational change. Philadelphia, PA, Open University Press. • Leibowitz, M. (1997). Instruction for process learning. Supporting the spirit of learning: When process is content. A. Costa and R. M. Liebmann. Thousand Oaks, CA, Corwin Press, Inc.: 47-54. • Marzano, R. J. (2001). Designing a new taxonomy of educational objectives. Thousand Oaks, CA, Corwin Press. • Mezirow, J. (1990). Fostering critical reflection in adulthood: A guide to transformative and emancipatory learning. San Francisco, Jossey-Bass. • Vygotsky, L. S. (1978). Mind in society: The development of higher psychological processes. Cambridge, MA, Harvard University. AMATYC 2007

  32. Participation • Gather into groups • Pick one course you teach • List the major concepts • Very few topics • Pick one or two to use • Identify Teaching Objective • Identify Student Objective • Identify knowledge levels • Can be more then one • Identify possible means of assessment • Identify success measures AMATYC 2007

  33. Share AMATYC 2007

  34. Two Levels of Course Assessment to Identify Student Learning Carol Lerch, Ph.D lerch@dwc.edu S 52 Friday 11:45 – 12:35

More Related