A Comparison of Learning Management System Accessibility:Blackboard, Desire2Learn,Moodle, SAKAI CSUN 2013
Presenters • HadiRanginWeb Design and IT Accessibility SpecialistUniversity of Illinois at Urbana-Champaign • Marc Thompson, Ph.D.Instructional DesignerOnline & Continuing EducationUniversity of Illinois at Urbana-Champaign • Dan HahneLearning ProfessionalCITES Academic Technology ServicesUniversity of Illinois at Urbana-Champaign
Overview • Background on LMS Comparison and why we did fresh evaluation • Categories and criteria • LMS side-by-side comparisons • Closing remarks • Contacts and FAQ
Vendor Collaboration • Working with vendors since 2005 • WebCT/Blackboard, Desire2Learn, Ebsco Publishing, Elsevier, Ex Libris, BB Collaborate, Qualtrics, many more • Very positive response from companies and good results • Increasing designers’/developers’ knowledge of accessible design and techniques • VPAT & Certificates • Accessible design vs. accessibility bandage
Disclaimer • All statements/opinions in this report are those of the authors. • This report does not represent the view or opinion of the presenters' respective institutions. • Our testing does not reflect a full or complete accessibility evaluation. • As we tested only selected tools, it is possible we did not test for some relevant functional accessibility features or tools. • It is possible there are errors in our testing and evaluation results. • Evaluation was performed at different times in 2012, so data does not reflect the newest enhancements (e.g., BB has revised its Discussion Board since our evaluation; thus, it is not included in the version we tested).
About the Evaluation and Results • Our report is an accessibility/usability evaluation. • We focus on coding practice, accessibility techniques, and overall usability. • Each of these LMS is unique and powerful. • Same tools but with different capabilities and complexity. • Results could be unfair to vendors offering more capable tools. • Results do not say which tool is better as a whole. • If you have suggestions for improving the testing protocol, please send your feedback to email@example.com.
Goals of this Project • Introduce accessibility/usability features that enhance users’ experience • Offer stakeholders independent accessibility-related information • Educate the community that accessibility is much more than alt tags for graphics and labels for form controls • Acknowledge and recognize vendors' efforts in accessible design and collaborative approaches to development.
LMS Compared • Blackboard 9.1 Service Pack 6 & 8 • Desire2Learn 10 • Moodle 2.3 • SAKAI 2.8
Testing & Evaluation Categories Login, Configuration, and Compatibility Testing Personalization and Customization Navigation Forms Help and Documentation Common Student-facing Modules/Tools Authoring Tools and Content Creation Features Unique to LMS that Affect Accessibility (not scored)
Common Student-facing Modules/Tools • Announcements • Discussion • E-mail • Chat • Assignments, Activities, Course Content, Learning Modules • Grade Book • Quizzing/Testing Components
Authoring Tools & Content Creation • File Uploading • Authoring Tool • Grade Book (Instructor Level) • Multimedia Content Handling
Evaluation Rationale • Performing a task easily and reliably • Performing a task independently & effectively • Error handling & recovery • Accessing help & documentation • Adaptation of UI to users’ needs
Evaluation Criteria • Functional vs. Technical Evaluation: • Kinds of questions we ask • Can users accomplish particular tasks?” • Can users post to a particular forum/thread? • Can users find out how much time is remaining for their quiz? • Kinds of questions we don’t ask • How did you code this? • Did you code the input element correctly for the subject line of the email? • Accessible technique is required, but not sufficient.
Which Disabilities Did We Test For? • Visual • Mobility • Learning • Cognitive
Methodology • Each criterion is tested and scored for fundamental accessibility features. • Testing for keyboard, screen reader, low-vision, learning disabilities. • Criteria have different weights depending on how they impact accessibility. • Related criteria in each category are organized into groups (subcategories). • Total score of a category is comprised of total points for the criteria in that category, including all subcategories.
Comparative Data Table • Shows points received for each criterion • Shows average score for each criterion • Shows how each LMS is doing in each category • Each LMS also has its own specific results page • Data are linked to corresponding comments in specific LMS results page
Rationale:Login, Configuration, and Compatibility Testing • First point of contact/interaction with the LMS. • LMS depends on external software/plug-ins, browser settings, etc. • Users must be informed about the required/optional software and optimal configuration before attempting to login. • Users must be able to download/install required/optional software and change necessary settings independently.
Rationale:Personalization and Customization • Users have different needs and ways of viewing and interacting with applications. • Users can adapt the system to their needs instead of adapting to the application. • Personalization improves the user’s experience.
Rationale for Navigation • Navigation is the most important element for accessibility. • Often there is no visual clue or description of the layout. • Users need to obtain necessary information to make navigation decisions. • Users must be able to navigate effectively and with certainty.
Rationale for Forms • Real interaction starts with forms. • Users must enter data easily and with certainty. • Data format must be easy to understand. • Instruction must be clear and understandable. • Users must be notified about possible errors and be able to fix them. • Users must be notified about successful submissions.
Rationale for Help and Documentation • LMS are complex applications and require a learning curve. • Main functions of tools must be clearly explained and easily visible. • Step-by-step instructions must be provided before or during the help process. • Supported accessibility features must be documented and made available inside the LMS, as well as outside the LMS.
Rationale:Common Student-facing Modules/Tools • User needs to interact with various tools in LMS. • Most essential and common modules/tools have been selected for evaluation. • Criteria depend upon the functionality of the tools.
Rationale for Authoring Tools and Content Creation • Content is the main component in online learning. • Authoring tools play a critical role in creating accessible content. • They can guide/educate content creators to create accessible content. • They can test for accessibility before upload.
Closing Remarks • Administrators: demand accessibility features and consider them on par with security & privacy • Accessibility is not a "nice feature" to have; it is a "must feature” • Need for continued collaboration • LMS are more accessible than ever • Build a culture that has accessibility as part of basic design considerations
How Can You Get Involved? • How can we become part of the solution? • Collaboration between product developers, end-users, and accessibility specialists. • Collaboration culture: commercial vs. open-source community. • All these products are significantly more accessible than a few years ago. • Blackboard Accessibility Interest Grouphttp://collaborate.athenpro.org/group/bb/ • Desire2Learn Accessibility Interest Grouphttp://collaborate.athenpro.org/group/d2l/ • Moodle Docs Accessibility Pagehttp://docs.moodle.org/dev/AccessibilityMoodle Accessibility Trackerhttps://tracker.moodle.org/browse/MDL/component/10083 • SAKAI Accessibility Working Group Pagehttps://confluence.sakaiproject.org/display/2ACC/Accessibility+Working+Group
Credits & Acknowledgments • Ken Petri, Ohio State University • Joe Humbert, Indiana University • Joanna Hunt (BB) • Sandra Earl & Janna Cameron (D2L) • Peter Jeffels, • Cheryl Todd, • Gaby Venatta, University of Illinois • Brian Richwine, Indiana University
Questions? • Contacts • Hadi Rangin, firstname.lastname@example.org • Ken Petri, email@example.com • Marc Thompson, firstname.lastname@example.org • Dan Hahn email@example.com • Joe Humbert firstname.lastname@example.org