Checklist for Digital/Hybrid Library Web Sites

1. Checklist for Digital/Hybrid Library Web Sites • Based on Pitchmann's (2001) article • Includes content as well as interface design and underlying technology • Expandable to metrics which relate content, operations/processes and systems/technological performance E. O’Brien Digital Libraries Term Project Due May 13, 2002

2. Libraries Evaluated: • MIT – http://libraries.mit.edu • University of Delaware – http://www.udel.edu • UIUC – http://www.library.uiuc.edu • Rice (Fondren) – http://www.rice.edu/frondren • Trinity U. (Coates) – http://lib.trinity.edu • Georgetown U. – http://www.georgetown.edu/dml • Rutgers U. – http://www.libraries.rutgers.edu E. O’Brien Digital Libraries Term Project Due May 13, 2002

3. Sample Criteria - Content • Is there one formal selection policy statement for the whole library on each website? • Is there any statement on the web site that helps users understand that the services offered are selective and quality controlled? • Does the library identify whether or not the collection is: • Comprehensive • Research • Study • Basic • Minimal E. O’Brien Digital Libraries Term Project Due May 13, 2002

4. Sample Criteria – Operations/Processes • Is the research process leading to the creation of the content described? • Is there evidence that the intended audience has been identified? • Does the site adhere to recognized standards and appropriate technologies? • What level of user support is available? (FAQs only, interactive support, help screens, contact information) E. O’Brien Digital Libraries Term Project Due May 13, 2002

5. Sample Criteria – Systems/Technological Performance • Are there process or technical criteria? (Technical features that measure the integrity of the site and availability of content reported to be provided) • Information integrity (Age of site, frequency of updates, frequency of revisions, version control, link maintenance) • System integrity (Server stability, frequency of downtime, response time, traffic volume and delays, downtime announcements, projected availability time and date announcements) • What metadata are used at each web site? E. O’Brien Digital Libraries Term Project Due May 13, 2002

6. Content, process, technology and metrics • Components of electronic libraries (Lakos, 1997) • Infrastructure • Services • Contents • Management • Ownership • Performance measurement compares what library is doing (actual performance) with what it means to do (mission) and what it wants to achieve (goals) E. O’Brien Digital Libraries Term Project Due May 13, 2002

7. Connecting the components: Collection development policy statement Acquisition Cataloging Digitizing Linking Using Retrieving Serving E. O’Brien Digital Libraries Term Project Due May 13, 2002

8. Results:

9. Highlights: • Rice/UIUC had excellent collection development policy descriptions • Only Trinity U describes mission, objectives and progress assessment • UIUC documents library staff operations yr/yr and compares data to other libraries. • Only GU and RU did not have info about quality control and selectivity of services • Only RU announced anticipated downtimes • All libraries described how they identified different user populations • MIT, UDEL, UIUC had process criteria to measure integrity of site and availability of content E. O’Brien Digital Libraries Term Project Due May 13, 2002

10. Conclusions: • Pitchmann's criteria can be adapted to evaluation of academic digital libraries • There may be better sources for evaluation (Brophy, 1998 & Lakos, 1997, others) • Libraries don’t plan for evaluation & performance measurement • Plenty of room for future work in this area but it can be done! E. O’Brien Digital Libraries Term Project Due May 13, 2002

11. Specific recommendations for Rutgers University Libraries are: • Define AAU criteria that libraries are to help RU meet. State how libraries will contribute. Include measurements of progress toward goals and when and to whom results will be reported. • Create mission or vision statement for libraries that includes mission, goals, current performance. (See UIUC, MIT, Trinity U. for examples.) • Develop metrics for infrastructure, services, content, management and ownership of libraries that link the impacts of performance in each area. (See Lakos article.) • Provide information on RU libraries’ web site about service and collection quality. (See MIT, UDEL, Rice and Trinity U. as examples.) • Update all pages on web sites that deal with mission statement, metrics, collection development. • Add information to web sites about how interests and needs of users are explored and how often their input is sought. (See MIT, UIUC as examples.) E. O’Brien Digital Libraries Term Project Due May 13, 2002