1 / 22

Evolution of Institutional Capacity to support the Assessment-Change-Effectiveness Cycle

Evolution of Institutional Capacity to support the Assessment-Change-Effectiveness Cycle. Course Content & Design Implementation Examples Assessment Strategies by Dr. Greg Witteman, U. Guam. The big questions. What are students’ abilities before taking the class?

parker
Download Presentation

Evolution of Institutional Capacity to support the Assessment-Change-Effectiveness Cycle

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Evolution of Institutional Capacity to support the Assessment-Change-Effectiveness Cycle Course Content & Design Implementation Examples Assessment Strategies by Dr. Greg Witteman, U. Guam

  2. The big questions • What are students’ abilities before taking the class? • What Knowledge, Skills, or Abilities (KSAs) should the students enter the class with? • What are the major objectives of the class? • What KSAs should the students leave the class with? • What is the role of assessment in course design or delivery? • Can outcomes be improved? (and how would we know?)

  3. Type of Course Content • Self contained Content (no prerequisites) • Examples: Anatomy and Physiology I, and II • Prerequisite knowledge needed • Examples: Conservation Biology and Comparative Vertebrate Biology labs  Abilities (student) Objectives (Course) Assessment (results)

  4. Self contained course content • Anatomy and Physiology I & II • Assumes no prior knowledge • Each part of the course builds on knowledge gained earlier in the course (hierarchy)  Abilities (student) Objectives (Course) Assessment (results)

  5. Course content assumes KSAs Cons. Bio. • e.g. Without general biology, math and ecology species population size, distribution and probability of extinction can not be meaningfully presented Previous courses  Abilities (student) Objectives (Course) Assessment (results)

  6. Course Design and Delivery Strategies • Course content • (what type of subject matter and students?) • Course goals • (what do we hope to accomplish?) • What should students be able to do to finish the course? • What should they know after the course ends? • Delivery and Assessment methods • (how and when do we know if we accomplished our goals ?)

  7. No Knowledge Have Knowledge limited from: H.S., background College, other Lower Division non-Sci. Majors Lower Division Science Majors Upper Division Science Majors The Ranges of KSA’s and goals for Science Courses Entry KSAs Courses Course goals  Abilities (student) Objectives (Course) Assessment (results)

  8. Example 1: Anatomy and Physiology • Non-science majors (freshmen/sophomore) • Extreme range of educational and cultural backgrounds • Goal: “Remember and Understand”: • Content heavy (terminology for structures must be learned before subsequent structures, function & concepts can be presented) • Two semester course, concurrent lab. (one grade) • Large lecture hall, no blackboard (PowerPoint and overheads only). Typical lab facilities. • No pre-requisites or assumed knowledge  Abilities (student) Objectives (Course) Assessment (results)

  9. Original Course Delivery model Concepts build on those previously presented in class. Students must leave course with knowledge and ability in every topic presented (which will be needed for subsequent courses and professional certification) Abilities (student) Objectives (Course) Assessment (results)

  10. Advantages and Disadvantages(assessing this approach) Pro: consistent delivery of material (Year-to-year) Con: can not adapt to differences in student learning (can leave gaps in knowledge) Abilities (student) Objectives (Course) Assessment (results)

  11. Problems with this approach Changes in content delivery (availability & access): Addition of supplemental materials on web and in library, adopted new texts.  Abilities (student) Objectives (Course) Assessment (results)

  12. Addressing these problems(adding adaptive assessment TO THIS approach) • Weekly adaptive quizzes • Goal = insure that content is covered and retained by “recycling” or re-emphasizing poorly understood topics (subsequent content adapts to knowledge gaps indicated by objective measures of student performance). • Pre-post tests (entry assessments, midterms and final) • Goal = overall course improvement by evaluating: successful presentations of individual topics, retention of material after completion of course (i.e.:of first semester material in second semester)  Abilities (student) Objectives (Course) Assessment (results)

  13. Quiz 1 Results 1 30 Neurons 30 70% Needs 40 Neuroglia 50 80% Review review? 30 Function 20 60% Quiz 2 40 Neurons 40 60 Neuroglia 40 0 (N. Function) 20 Adaptive Assessment Abilities (student) Objectives (Course) Assessment (results)

  14. Assessment “feedback” loop Pro: an adaptive approach can be used to close the gaps in student knowledge Con: takes time and effort by instructor to implement. Note: this can be used for both self contained and pre-requisite knowledge courses  Abilities (student) Objectives (Course) Assessment (results)

  15. Course Changes: Technology classroom (NIH-RISE) • Interactive media & Video on demand • Dissections, models, background & supplemental information • View PowerPoint lectures • Print lecture notes and outlines • Automated practice quizzes and tests • (Web-based, secure (CGI),with feedback (in TC only) • Equal access to course materials

  16. Example 2: Comparative Vertebrate Biology • Upper division biology majors • Juniors/Seniors (pre-med, pre-vet, pre-graduate school) • “Remember, Understand & Analyze” • Content and concept intensive • Lecture & lab (20 students) • Assumes basic understanding of lower-division general biology, basic laboratory skills, understanding of correct terminology.  Abilities (student) Objectives (Course) Assessment (results)

  17. Course design & delivery (lab) NOTE: Student self-assessment/feedback is within lab prior to tests • Lab exercises with questions • (non-adaptive, “you must know this…” and “how is this structure similar to …”). Assumes good study habits and skills. • Student presentations of lecture summaries in lab • Identify difficult concepts with need for more in-depth coverage (identify gaps in knowledge) • Create participatory classroom dynamic • Lab documentation projects • Students create class materials (self documenting review materials) • “problem based learning”  Abilities (student) Objectives (Course) Assessment (results)

  18. Course changes: NIH-Rise-TC • Visual review (Digital Video) of in-lab procedures, dissections, and specimens • A picture is worth a thousand words, and a video is worth a thousand pictures! • Equal access to information resources • (“go find everything you can about…” on the web.)

  19. Learner-centered approaches to assessment/improvement implemented • Human Anatomy and Physiology • Use of adaptive weekly quizzes to “fine-tune” delivery of individual lectures (year-to-year), guarantee a sufficient knowledge foundation on which to build additional concepts (within the semester) • Entry and Exit assessments provide the information necessary to measure success in meeting the course goals • Comparative Vertebrate Anatomy Lab • Use of self and peer assessment to identify and fill gaps in knowledge, skills and abilities before they are formally tested.

  20. Application of NIH-Rise Technology Classroom capabilities • Increases in information resources availability • Online resources (web) • Class notes, outlines and presentations. • Dynamic content capabilities • Self serve video (TB server capable of 600+ hours of MP1 video) • Interactive content (practice exams, data exploration) • Provide all the Information Technology tools needed for “Problem based learning” • …and, let’s not forget Technology Development support and training for faculty

  21. The bad news • There is no free lunch • Incorporating adaptive assessment (in A&P) requires an additional 3 hours each week to quiz and lecture preparation and delivery (50% increase after initial development of course materials) • Adding self and peer assessment (2 hours additional planning and setup for TC use).

  22. Still to be addressed... • Why did I do this? • Student outcomes poor relative to objective standards • Why should others do this? • How can others incorporate these types of assessment features in their courses? • Drink more coffee • Give up sleep • Conclusion: still some implementation obstacles

More Related