Usability Testing of a Medical Library Portal Anamarija Rožić-Hristovski, M.D., M.Sc.Central Medical Library Iztok Humar, B.Sc., M.Sc.Faculty of Electrical Engineering
Presentation Overview • Introduction • CMK portal • Case study goals • Usability studies • Methods • Results • Conclusions
Introduction • A Web portal or gateway aggregates library resources and services through a single access point. • Medical library Web interfaces have become increasingly complex. • There is a fundamental need for the systematic evaluation of the usability of these Web based systems.
CMK Portal • CMK decided to extend its Web site with information portal • CMK portal based on MyLibrary system at NC State University • Customisation to the needs of CMK users was needed • MyLibrary functionality is enabled
MyLibrary CMK functionality • Consideration of the CMK organisation, role and function • Slovene-English communication with MyLibrary • Customisation of text and content • Personalization of users’ profiles • Slight modification of the portal design to suit the rest of the CMK Web site
Case Study Goals • Formal usability case study for the evaluation of the pilot portal • Review of applications of usability testing to library Web sites • Selection of the most appropriate usability methods • Application of selected methods to evaluate CMK portal
Usability & usability testing • ISO defines usability as “the extent to which a product can be used by specified users to achieve specific goals with effectiveness, efficiency, and satisfaction in a specified context of use” • In the world of WWW this means that the interface of a portal must be easy to learn, remember and use and must lead to as few errors as possible • In usability testing, users are observed while using the portal, to perform given task or achieve a set of predefined goals
Most common methods for evaluating the web site usability • Objective performance • Subjective user preferences • Experimental evaluation • Direct observation
“Think aloud” method • The most systematic and valid direct observation method • The users are asked to perform specific task • A relatively small sample size (5-6 users) is needed for identifying the majority of the usability problems • During their work, they are required to verbalize their problems • User verbalizations and keystrokes are recorded on audio tapes, observations are written to notebooks
“Think aloud” method: properties • This method provides answers: • How easy a user can learn and use the system • How flexible the system is • Why users have made mistakes • “Critiques of this method goes to: • Presence of the experimenter: • disturbance for the user • The need for verbalization: • depends on the user’s character
Methods- I • “Think aloud” method was selected as the most appropriate • Designing the test: • 12 test exercises • Instructions to observer • Tape-recording of sessions • Observations written to notebooks
Methods –II • Test participants • Before instruction • 3 students • 3 faculty • After instruction • 3 students • 3 faculty
Usability Testing of MyLibrary@CMK • Login to MyLibrary@CMK portal! • On your personal page, select the library links that you intend to use at your work. Return to MyLibrary! • On your personal page, select the university links that you intend to use at your work. Return to MyLibrary! • On your personal page, select the quick searchers that you intend to use at your work. Return to MyLibrary! • On your personal page, select the databases that you intend to use at your work. Return to MyLibrary! • On your personal page, select the e-texts that you intend to use at your work. Return to MyLibrary! • Identify all the resources about AIDS available in MyLibrary! Return to MyLibrary! • Find all books from the field of dermatology, available at the Central Medical Library! Return to MyLibrary! • Find journal Neurology! Return to MyLibrary! • Find a journal article about headache! Return to MyLibrary! • Find new books from the field of public health of the last three months! Return to MyLibrary! • Switch the language for communication with MyLibrary!
Results – III:Summary of Usability Problems • Note on length of password • Inconsistent performance of buttons • Complaints about registration • Unclear terminology • The role and content of MyLibrary • The role and content of OPAC • The role and content of databases • No idea how to find an article • No distinction between the title of an article and the title of an journal
Conclusions • Students performed tasks much better than faculty • More familiar with internet and concept of digital library • Role of education • No influence on success rate • Less time needed for finishing the tasks • Identification of main usability problems
Further Work Redesign of library portal considering results of usability study New usability study to verify appropriateness of changes Updating the content of portal Public use of portal