The Development of an Evaluation and Tracking System for Learning Objects

1. The Development of an Evaluation and Tracking System for Learning Objects Vicki S. Freeman, Ph.D. Carol Larson MSEd.

2. CLS Accessibility Project • Partners • University of Texas Medical Branch, Galveston, TX • University of Nebraska Medical Center, Omaha, NE • Evaluation Consultant • David Holcomb Ed.D., Baylor College of Medicine, Houston, TX

3. Team Members • UTMB: • Dr. Vicki Freeman, Co-Project Director • Dr. Jean Brickell, Faculty Content Expert • Dr. Michelle Kanuth, Faculty Content Expert • Mary Donna Piazza, Graphic Design Artist • Toby Smith, Programmer • UNMC: • Carol Larson, Co-Project Director • Sandy Latshaw, Faculty Content Expert

4. Objectives • Define the 3 levels of learning objects and the uses for these learning objects at each level. • Describe the development of an evaluation and tracking system to be used to determine the cost/benefit of learning objects. • Discuss the evaluation studies of the impact of LOs in terms of cost-effectiveness and accessibility.

5. Levels of Learning Objects • Learning objects will be grouped into 3 categories based on their granularity. • Level 1 – images, illustrations • Level 2 – video, animation (non-interactive) • Level 3 – animation, interactive activities

6. What are Learning Objects? • Small, reusable components of instructional media that allow: • creation of instructional components that can be reused numerous times in different learning contexts • delivery over the Internet and access by a number of individuals simultaneously, with minimal effort • faculty to reassemble into their own lesson format to support their individual instructional goals • increased speed and efficiency in instructional development and a decrease faculty preparation time

7. Examples of Level 1 LO’s

8. Examples of Level 2 LO’s • Animation of the construction of a protein molecule • Video of a positive catalase test

9. Example of Level 3 LO • Interactive lesson of Indole Test • Interpretation • Test principle • Used to identify bacteria

10. Progress • Learning Objects developed • Level 1 (pictures, graphics) = 213 • Level 2 (animation, video) = 111 • Level 3 (interactive, flash) = 39 • Public status = 264 Goals: LevelYear 1Year 2Year 3 1 20 30 25 2 10 25 20 3 5 15 25

11. Web-Accessible LO Database http://webcls.utmb.edu/lo/

12. Web-Accessible LO Database

13. Learning Object Properties

15. View Effort Expended per Learning Object

16. Effort Expended Detail

17. CLS Accessibility Project Evaluation Active Faculty/Student Usage/Acceptance High Quality Learning Objects CLS Accessibility Project Summative Benchmarks of Success Comprehensive Web-based Dissemination Resource Cost Effective/ Sustainable Formative Assessments of Success • Accuracy / clarity / currency • Interactivity / durability • Three levels of learning • Replicability Learning Objects Faculty/Student Usage Web-based/Dissemination Cost Effectiveness/ Sustainability

18. Evaluation Database

19. Evaluation Database

20. Evaluation Database

21. Time Analysis

22. CLS Accessibility Project Evaluation Active Faculty/Student Usage/Acceptance High Quality Learning Objects CLS Accessibility Project Summative Benchmarks of Success Comprehensive Web-based Dissemination Resource Cost Effective/ Sustainable Formative Assessments of Success Learning Objects Faculty/Student Usage Web-based/Dissemination Cost Effectiveness/ Sustainability

23. CLS Accessibility Project Evaluation Active Faculty/Student Usage/Acceptance High Quality Learning Objects CLS Accessibility Project Summative Benchmarks of Success • Faculty training activities • Faculty usage strategies • Student usage / performance • Faculty / student critiques of • learning objects Comprehensive Web-based Dissemination Resource Cost Effective/ Sustainable Formative Assessments of Success Learning Objects Faculty/Student Usage Web-based/Dissemination Cost Effectiveness/ Sustainability

24. User Report

25. User Report

26. User Report

27. CLS Accessibility Project Evaluation Active Faculty/Student Usage/Acceptance High Quality Learning Objects • Web-accessible data base • Linkages with related national and regional sites • Promotion among national and regional CLS groups • Tracking system CLS Accessibility Project Summative Benchmarks of Success Comprehensive Web-based Dissemination Resource Cost Effective/ Sustainable Formative Assessments of Success Learning Objects Faculty/Student Usage Web-based/Dissemination Cost Effectiveness/ Sustainability

28. User Report

29. CLS Accessibility Project Evaluation Active Faculty/Student Usage/Acceptance High Quality Learning Objects CLS Accessibility Project Summative Benchmarks of Success Comprehensive Web-based Dissemination Resource Cost Effective/ Sustainable • Development cost analyses • Benefits analyses • Marketing results • Licensing agreement • Institutionalization Formative Assessments of Success Learning Objects Faculty/Student Usage Web-based/Dissemination Cost Effectiveness/ Sustainability

30. CLS Accessibility Project Evaluation Active Faculty/Student Usage/Acceptance High Quality Learning Objects CLS Accessibility Project Summative Benchmarks of Success Comprehensive Web-based Dissemination Resource Cost Effective/ Sustainable Formative Assessments of Success Learning Objects Faculty/Student Usage Web-based/Dissemination Cost Effectiveness/ Sustainability Accuracy/clarity/currency Faculty training activities Web-accessible data base Development cost analyses Linkages with related national and regional sites Interactivity/durability Faculty usage strategies Benefits analyses Three levels of learning Student usage/performance Marketing results Promotion among national and regional CLS groups Faculty/student critiques of learning objects Replicability Licensing agreement Tracking system Institutionalization

31. Questions / Comments

32. Thank you!!