Technological Tools for Assessment

1. Technological Tools for Assessment Jonathan D. Sousa Technical Applications Development Manager Association of Research Libraries New Ways of Listening to Library Users Workshop Washington, DC November 4 - 5, 2005 old.libqual.org

2. Who I Am • Jonathan D. Sousa (jonathan@arl.org) • Technical Applications Development Manager • Responsible for the development of online applications for ARL’s Statistics and Measurement Program • LibQUAL+™, StatsQUAL™, DigiQUAL™, ARL Statistics Survey (new version), MINES for Libraries • 4 years at ARL • 8 years in application development • Worked 2 years in an ARL library • Mucking about online since Gopher was new and 14.4 Kbps was fast old.libqual.org

3. What I Want to Cover Today • The technological history of LibQUAL+™ • Nicely representative of how technology and assessment in libraries have grown together • Current assessment offerings using technology • StatsQUAL™ project, DigiQUAL™, MINES for Libraries • Tools to help with other types of analysis • Future directions (with online demonstration) • Whatever you want • I’m happy to digress • But one thing you should know: old.libqual.org

4. old.libqual.org

5. The LibQUAL+™ Online Survey Process • Web-based survey hosted at a single location but delivered all over the world • Each participating institution receives a URL with a unique code embedded • When individuals take the survey, they see a version tailored to their institution’s language and type and further customized by the survey liaison using our online administration tools • Results are compiled and analyzed by the LibQUAL+™ team and made available for download at the end of the survey run old.libqual.org

6. LibQUAL+™ – The Technological Beginnings • Texas A&M – Pre-2000: • Simple piece of rough ColdFusion code and a flat-file database • Experimental • Few thoughts of scaling the project • Innovative but localized old.libqual.org

7. Pilot Program with ARL – 2000-2001: • Some new techniques were attempted • Numerous obstacles were encountered in moving beyond one institution • Production of the results was a laborious process, particularly in 2001 • An attempt at gathering data on library users across institutions with a new protocol using a relatively new medium • The full power of LibQUAL+™ was starting to appear in its standardized core: cross-institutional analysis old.libqual.org

8. LibQUAL+™ – A Shift in Development – 2002 • The project development moved to ARL • Code base was entirely rewritten • Database was shifted to a scalable relational structure • An interface for institution liaisons was added • Administrative pages were created old.libqual.org

9. LibQUAL+™ – Refinement and Growth – 2003-2005 • The site was rewritten again in 2003 • Improve the interface • Simplify the process • Increase customization options • Add additional languages and institution types • Automation of results production • What the public sees is now the tip of the iceberg: nearly all administration of the project takes place through restricted pages on the website • Survey instrument refined: the core is now 22 questions, 3 dimensions old.libqual.org

10. Making LibQUAL+™ Mundane • Faced with many people now coming to LibQUAL+™ without first being familiar with ARL • Different types of institutions coming onboard, many with very limited staff and few hours to devote to assessment • How LibQUAL+™ has responded: • Simplified all steps as much as we are able, while still allowing flexibility • Accept credit cards and consortial payment arrangements • Direct download of results instead of mailing them around the world: faster and it keeps the price of the survey low • Excel format of results for those without access to SPSS • Available all year long—2 sessions to accommodate the Southern Hemisphere • Moved to professional hosting facility for assured uptime old.libqual.org

11. LibQUAL™ – Rapid Growth old.libqual.org

12. 9 Languages Afrikaans American English British English Dutch Dutch English French (Canada) French (Europe) Swedish Swedish English 14 Types of Institutions Academic Health Sciences Academic Law Academic Military College or University Community College Electronic European Business Family History Hospital Natural Resources Library Public Research Centers (FFRDC) Libraries State University/TAFE Rapid Growth in System Complexity old.libqual.org

13. LibQUAL+™ as a Mirror of Assessment in Libraries • Started in isolation • Expanded but suffered growing pains • Grant funded or recipient of a one-time allocation of funds • Began to prove its worth • Became part of library’s and institution’s decision making processes • Shifted from a single event to an ongoing process (a culture of assessment) • Now frequently treated as a necessary part of the library’s mission as demonstrated by employees and committees dedicated to assessment old.libqual.org

14. How LibQUAL+™ Technology Concerns Have Changed • 2 years ago the things I talked about: • Network issues • Either things are getting better or we are all more familiar with how the Internet works… and breaks • Caching • Software on the networks and on computers is finally doing its job without getting in the way • Less concern about bandwidth and network speed and fewer people looking to caching as a solution • Getting accurate email lists • Many more institutions now see keeping an up-to-date email list to be critical to their functioning • Now we need to worry about keeping email out of aggressive spam filters • Browser compatibility • This one hasn’t changed: there are still holdouts who continue to use browsers from before 1997 old.libqual.org

15. Online Tools for Assessment • DigiQUAL™ • Short survey for digital libraries using the 2-column SERVQUAL format • Math Forum, MERLOT, Utopia, etc. • MINES for Libraries • Survey on the purpose of use of electronic resources • Run at individual libraries without a centralized server • Mandatory for certain time periods • ARL Statistics Survey • A long-running survey of collections, expenditures, staffing, and service activities at ARL institutions • The collection tool was completely rewritten this year while the questions remained the same old.libqual.org

16. Other Technological Tools • Usability studies • Camtasia • http://www.techsmith.com/ • Captivate • http://www.macromedia.com/ • Web log analysis • Urchin • http://www.urchin.com/ • WebTrends • http://www.webtrends.com/ old.libqual.org

17. Future Projects • StatsQUAL™ • Unify the various assessment tools under one system with a single login • Share datasets between surveys • Interactive Statistics • Beta version demonstration old.libqual.org

18. What We’re Focusing On • Data integration • Dynamic analysis • Web standards • New assessment solutions old.libqual.org

19. Online Resources • StatsQUAL™ Website:http://www.statsqual.org • LibQUAL+™ Website:http://www.libqual.org • Publications:http://www.libqual.org/publications • Events and Training:http://www.libqual.org/events • LibQUAL+™ Bibliography:http://www.coe.tamu.edu/~bthompson/servqbib • LibQUAL+™ Procedures Manual:http://www.libqual.org/Manual/index.cfm old.libqual.org

20. LibQUAL+™ Contacts • Martha Kyrillidou • Director, Statistics & Measurement Program • martha@arl.org • Amy Hoseth • LibQUAL+™ Communications Coordinator • amyh@arl.org • Richard Groves • Research Assistant - Statistics & Measurement • richard@arl.org • Jonathan D. Sousa • Technical Applications Development Manager • jonathan@arl.org old.libqual.org