1 / 27

E-Outcomes Assessment Project: Technology Linking Assessment & Learning

E-Outcomes Assessment Project: Technology Linking Assessment & Learning. Luvai Motiwalla, Steven Tello & Kathryn Carter College of Management UNIVERSITY OF MASSACHUSETTS LOWELL. Presentation at The NERCOMP Conference Mar 20 th 2007, Worcester, MA. Agenda. Assessment Challenge

crwys
Download Presentation

E-Outcomes Assessment Project: Technology Linking Assessment & Learning

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. E-Outcomes Assessment Project: Technology Linking Assessment & Learning Luvai Motiwalla, Steven Tello & Kathryn Carter College of Management UNIVERSITY OF MASSACHUSETTS LOWELL Presentation at The NERCOMP ConferenceMar 20th 2007, Worcester, MA.

  2. Agenda • Assessment Challenge • Project Overview • Program-Level Assessment Process • Systems Integration • Progress to Date • Future Directions

  3. The Assessment Challenge • Increased calls for accountability in higher education • The Spelling Report, Secretary of Educationcalls for “evidence-based” assessment measures • Regional and Professional Accreditation agencies • Widespread interest in higher-educationinstitutions to define and assess student learning objectives • No resolution on how to define, measure, and achieve quality learning • Different approaches for outcomes assessment • student learning within courses and academic programs • student learning within specific fields or majors • Pre and Post learning assessments, surveys, etc.

  4. The Assessment Challenge • What is Assessment?Many different perspectives • Administrator, Faculty, Student, Agency • “Assessment is the systematic collection, review, and use of information about educational programs undertaken for the purpose of improving student learning and development.”(Palomba & Banta, 1999)

  5. The Assessment Challenge Problems with Assessment • Tedious • Costly • Untimely • Painful • Useless • A labor intensive, paper-based process driven by accreditation agency visits & timelines

  6. A Possible Solution… Design and develop an electronic outcomes assessment process • which captures the course or student activities (e.g., assignments, discussions, or exams) within a learning management systems (LMS) • categorize it relative to the college or university’s stated learning goals or outcomes.

  7. Project Overview • Two-year, grant-funded project • UMass Lowell & Davis Educational Foundation • Use LMS to organize and report student progress toward program-level learning outcomes • Interdisciplinary focus • 6 Business Core Courses, 6 General Education • Two operational foci • Faculty Development • Systems Development

  8. Project Overview - Goals • Improve the learning environment for students by increasing student access to content, instructors and peers. • Improve content and assessment consistency across multiple disciplines. • Facilitate student development of academic e-portfolios. • Reduce overall administrative costs through online paperless assessment, reporting, tracking.

  9. Assessment Process - Learning Outcomes I II III IV V VI VII VIII Breadth of Knowledge UniversityGeneral EducationOutcomes Clear Communication InformationLiteracy Ethics Self-Direction/ Collaboration Critical Thinking Diversity InformationCommunication Technologies COMLearning Outcomes Quantitative & Qualitative Skills Ethics & Values Communication Skills Decision-making&Problem solving Diversity Impact Team Leadership & Membership GlobalImpact

  10. Assessment Process – Self Reporting Current Process Manual, Paper-based, Self-report

  11. Assessment Process: New Approach Proposed Process Automated, Menu-driven, Drill down Assessments Discussion Topic A Assignment 1 Exam 1 & 3 Assignment 2 Group Tool Usage Group Project Excel Assignment Case Assignment Courses CoM Learning Outcomes Course A Course B Course C Course N COMMUNICATION SKILLS QUANTITATIVE SKILLS TEAM PARTICIPATION IT USAGE ETHICS DIVERSITY IMPACT GLOBAL IMPACTS

  12. Assessment Process: Future Reporting Possibilities Digital Dashboard

  13. Systems Integration • Use LMS (WebCT Vista) as Assessment Collection Tool • Organize & present course assessment activities • Present outcomes to students • Collect assessment artifacts • System limitations require external database • To map course activities to program-level outcomes • To organize & report progress toward outcomes

  14. Assessment Tool Chat Tool CourseContent Files DiscussionTool Grade Book Syllabus Tool Assignment Tool Groups Systems Integration • Use the activity tools in WebCT Vista • Train faculty in specific use of these tools

  15. Systems Integration • Use PowerSight to report off Grade book • Train faculty to use grade book

  16. Systems Integration • EOutcomes Database • My SQL, Perl, Web Interface • Export CSV from WebCT, Import to EOutcomes • Faculty map outcomes to specific course activities in EOutcomes Database • Administrators then report on progress toward outcomes through EOutcomes

  17. EOutcomes System V1 Systems Integration

  18. Learning Outcomes Menu Systems Integration

  19. Progress to Date – Year 1 • Conducted Summer Training Workshop • Seven courses offered in Fall 2006 / Spring 2007 • Developed a prototype EOutcomes database and application

  20. Goal 1: Improve learning environment by increasing student access… • 7 Courses offered Fall2006, 385 students • Each Web-Enhanced to varying degrees.

  21. Goal 1: Improve learning environment by increasing student access…

  22. Goal 2: Improve content and assessment consistency across multiple disciplines • EOutcomes Training Institute, June 2006 (June, 2007 as well) • Learning Outcomes • Assessment Matrix in Excel • Assessment Tools in WebCT • Reporting Tools in WebCT • Respondus Exam Tool • EOutcomes Database • Tablet PC Demo • Training Materials online • http://comwebserver.uml.edu/eoutcomes/ • EOutcomes WebCT Course

  23. Goal 3: Facilitate student development of web-based academic portfolios. • ePortfolios provide ability to link sample student work to activity to learning outcomes • Very limited success due to WebCT issues and manual nature of effort • ePortfolios often focused on student need to build a portfolio, rather than administrative need to measure achievement • Continue to review possible tools within WebCT and external • Possibly EOutcomes Database

  24. Goal 4: Reduce overall administrative costs • Possible savings include: • Supply savings – less scantron sheets and paper for exams • Proctor savings – less time spent scanning, reviewing and reporting exam grades (less room for error) • Faculty time savings when: • grading exams, many are auto-scored • grading quizzes, many are auto-scored • filling out assessment spreadsheets • collecting student work samples • making copies • Savings are balanced by initial development effort on part of faculty, and maintenance

  25. Emerging Research Questions • Does the use of a LMS in a web-enhanced course increase student awareness of program-level learning outcomes? • Do students perceive a benefit in the use of a LMS for course-level assessment activities? • Do faculty perceive a benefit in the use of a LMS for course-level assessment activities? • Does the use of a LMS facilitate the organization and reporting of program-level assessment data for accreditation purposes? • Does the LMS in a web-enhanced course have any positive impact on students’ involvement in a course or satisfaction with the course or their learning?

  26. Next Steps • Develop EOutcomes Reporting System tools • Incorporate six more courses in 2007-2008 academic year • Conduct training institutes in Summer 2007 • Measure Project Achievements • Integrate Research Agenda

  27. Thank you • Prof. Luvai MotiwallaLuvai_Motiwalla@uml.edu • Prof. Steven TelloSteven_Tello@uml.edu • EOutcomes Websitehttp://comwebserver.uml.edu/eoutcomes/

More Related