Making Library Assessment Work: Practical Approaches for Developing and Sustaining Effective Assessment Phase I Update

1. Making Library AssessmentWork:Practical Approaches for Developing and Sustaining Effective AssessmentPhase I Update ARL VISITING PROGRAM OFFICERS 2004-06 Steve Hiller Library Assessment Coordinator, University of Washington Jim Self Director, Management Information Services, University of Virginia & Martha Kyrillidou ARL, Director of Statistics and Measurement Program

2. ARL New Measures 1999- Key Areas • User Satisfaction/Service Quality • LibQUAL+™ • Measuring Library Service Quality Lyceum • Market Penetration • Ease and Breadth of Access • Library Impact on Teaching and Learning • Project SAILS, Learning Outcomes • Library Impact on Research • Cost Effectiveness • Library Facilities/Space • E-Metrics • MINES for libraries • ARL supplementary statistics

3. Barriers to Management Information and Using Data Effectively in Libraries • Organizational culture/leadership support • Time/Staff/Resources • Data issues – Too much, compatibility, validity • Establishing priorities • Knowing what to measure and methods to use • Inexperience, perceived lack of skills and expertise • Understanding, presenting and knowing what to do with the results Hiller, S. and Self, J. (2004). From Measurement to Management: Using Data Wisely for Planning and Decision-Making. Library Trends.

4. Making Library AssessmentWork:Practical Approaches for Developing and Sustaining Effective Assessment • 2 year ARL project to assist libraries with moving assessment efforts forward • Led by Steve Hiller (UW) and Jim Self (UVa) • Recognition that libraries understand value of assessment but have trouble sustaining efforts and using results • Site visits conducted to evaluate assessment and develop practical approaches to effective local practices • Identify common barriers and facilitators to assessment • 7 libraries participating in Phase I Spring 2005; 12-14 during 2005-06 academic year; final report in 2006

5. Phase I Participants • University of Arizona February 2005 • Arizona State University February 2005 • University of Connecticut March 2005 • University of Illinois at UC May 2005 • New York University March 2005 • University of Notre Dame June 2005 • University of Oregon May 2005

6. Phase I Process • Each library chooses a contact person • Survey sent to contact person 2 months before visit • Assessment activities, motivators, organizational structure • What’s worked/hasn’t, specific areas to address, expectations • Schedule of meetings established • Site visit • Short report completed within 30 days • Specific recommendations made on moving forward • Library evaluates process after final report received

7. Sample Site Visit Schedule • Meet with University Librarian/Contact person • Presentation on effective assessment • 90 minutes to 2 hours with Q&A and discussion • Group Meetings • with management/administrative group • assessment-related group (if formed • different departments • functional areas (e.g. info literacy) • Wrap-up

8. ARL ProjectPreliminary Observations after Phase I Visits • No “cookie-cutter” libraries; diverse organizational cultures • Sustainability seen as resource issue • Management information system seen as crucial underpinning • Integrating data use effectively into library management • Many internal statistics (non-ARL related) kept • Few libraries allocate sufficient resources (staff and funding) for assessment or management information • Facilities renovation as catalyst to good assessment practices • Good assessment work being done but not published/ disseminated or coordinated • Need for basic/practical training on assessment methods/analysis • Little understanding of what’s happening outside the library in data warehousing/management

9. Phase I Feedback from Participants • One day is too short • More resource materials would be helpful • More examples presented • Follow-up activity would maintain momentum • Establishing an assessment-related “community” of practitioners would assist ongoing communication and development

10. Phase II Changes • Site visit time increased from 1 day to 1 1/2 days • Appropriate resource materials provided • Consulting on a follow-up activity • Assessment plan implementation • Specific assessment effort • Individual meeting at a professional conference • Group meeting at a professional conference • Stronger project Web presence

11. Confirmed Cornell Emory Kansas Louisville Pennsylvania State Southern California Texas Tech Wayne State Western Ontario Willing to commit (not confirmed) Boston Public Library Colorado State Nebraska Purdue Interest pending (maybe) Alabama Alberta Marquette Phase II Participants

12. Future Reporting • Upcoming Presentations • Northumbria International Conference on Performance Measurement, Durham, England, August 22-25 • 3rd International Evidence-Based Librarianship Conference, Brisbane, Australia, October 16-19 • ARL Membership Meeting, Washington, D.C., October 25-28 or in May 2006 • Making Library Assessment Work Web Site • http://www.arl.org/stats/newmeas/VPO_Hiller_Self.html • Final Report late summer 2006 • ARL Newsletter • Conference in Fall 2006?