Assessment for Impact: Turning Data into Tangible Results

1. Assessment for Impact: Turning Data into Tangible Results Laura Miller Tim Morton Paul Rittelmeyer Library Assessment Conference Seattle, WA August 5, 2008

2. Where Does Data Come From? • Balanced Scorecard • LibQUAL+ • Surveys • Users • Staff • Focus Groups • Follow-up Interviews • Statistics (circ, ILL, gate counts, purchase requests, etc.)

3. What Does Data Do? • Improve Library Services • Inform Collections Decisions • Influence Satisfaction in the Workplace • Support Budget Requests • Contribute to Staff Development

4. Improve Library Service Rush Order (aka Purchase Express) Metric U.4.b. Turnaround time for new book and ILL requests (external resources). Target1: Satisfy the turnaround targets 75% of the time.Target2: Satisfy the turnaround targets 50% of the time. Method: New Books: Fill requests in 7 days.  A sample of in-print US titles requested by University-affiliated patrons will be drawn from the request database by Management Information Services. Turnaround time from patron request to availability will be measured.ILL: Fill requests in 7 days. Turnaround time will be measured by ILS from patron request to notification that the book or article is available.

5. Improve Library Service AUL OK Acquisitions (order to receipt) 2-4 days

6. Improve Library Service AUL OK OK Acquisitions (order to receipt) 2-4 days Cataloging (receipt to shelf prep to notifying patron) 1 day

7. Improve Library Service AUL OK OK Acquisitions (order to receipt) 2-4 days Cataloging (receipt to shelf prep to notifying patron) 1 day Balanced Scorecard

8. Improve Library Service AUL OK OK Acquisitions (order to receipt) 2-4 days Cataloging (receipt to shelf prep to notifying patron) 1 day Patron request/selector review/placing order ? Balanced Scorecard

9. Improve Library Service AUL OK OK Acquisitions (order to receipt) 2-4 days Cataloging (receipt to shelf prep to notifying patron) 1 day Patron request/selector review/placing order ? Balanced Scorecard • Selector training • Back ups for email/order placement • Funding for expedited shipping

10. Inform Collections Decisions LibQUAL+ and satisfaction with journals

11. Inform Collections Decisions LibQUAL+ and satisfaction with journals

12. Inform Collections Decisions Targeted Interviews Following up with faculty • Focused on low-scoring areas • Approached a diverse group of faculty • Asked for specific needs, wants and suggestions • Kept the interviews brief

13. Inform Collections Decisions What We Learned? • Dissatisfaction was not caused by single factor – no ‘smoking gun’ • Perceived access was a problem – in one area we already subscribed • to all but one of the requested journals • Lack of foreign titles • Missing or incomplete backfiles

14. Inform Collections Decisions The Results • To address access, we instituted a major, ongoing re-design of our website • We invested in a electronic resource management system (Serials Solutions 360) • We prioritized the purchase of science backfiles (Wiley, Elsevier) and new publisher packages • In the arts, we put significant funds into the purchase of quality images, or, if they could not be bought, we digitized them ourselves

15. Influence Staff Satisfaction • Library Worklife Survey • Conducted in even-numbered years • Available to all staff via online form with email reminders • Response Rates: 2004 – 60%, 2006 – 52.2% • Two-Step Analysis/Response Process • Online survey • Concerns identified • Focus groups, one-on-one meetings • What can be done to address concerns?

16. Influence Staff Satisfaction • Impacts Seen2006 2004 Change • Library staff in other departments are familiar with what I do. 28.3% 17.3% 11.0% • The Library administration is willing and available to listen to 52.3% 37.3% 15.0% • my concerns, comments, and recommendations. • The Library administration effectively communicates to staff. 36.9% 36.6% 0.3% • Collaboration is encouraged and supported in my job. 69.0% 75.8% -6.8% • I get cooperation from other departments when we work together. 71.2% 74.0% -2.8%

17. Influence Staff Satisfaction • Impacts Seen 2006 2004 Change • I have adequate resources and materials to complete assignments. 59.5% 49.2% 10.3% • I feel safe in my work environment. 87.9% 81.5% 6.4% • The Library is concerned about and addresses my ergonomic needs 67.0% 76.3% -9.3% • I am physically comfortable in my work environment • (temperature, light, noise). 56.4% 44.6% 11.8%

18. Influence Staff Satisfaction • Impacts Seen 2006 2004 Change • My salary and benefits are reasonable for the work I do. 43.9% 21.5% 22.4% • The salary I receive is equitable when compared to co-workers • in similar grade levels or ranks. 49.0% 30.9% 18.1% • My work at the Library helps me achieve my career goals. 55.4% 35.9% 19.5% • There are opportunities for me to advance at the Library. 36.8% 28.1% 8.7% • There are adequate staff development opportunities • provided by the Library. 64.2% 57.3% 6.9%

19. Influence Staff Satisfaction • The Overall Impact • 2006 2004 Change • My work at the Library helps me achieve my career goals. 55.4% 35.9% 19.5% • I am satisfied with my job. 63.2% 58.5% 4.7% • Overall Job Satisfaction Category 65.5% 58.0% 7.5%

20. Support Budget Requests • 80% of Music faculty responded to the 2004 Faculty Survey. • Of these respondents, 88% chose sound recordings as their • highest priority, making it the highest priority item for the • department. • While every Music faculty respondent rated the resource • “sound recordings” on a 1-5 scale, the category received one • of the lower ratings for satisfaction, a 3.88. • The rating for sound recordings among faculty as a whole was • 3.89.

21. Support Budget Requests Music Library Budget

22. Challenges Ahead Promoting a Culture of Assessment • Raising awareness of traditional practices as assessment tools • (ref counts, gate counts, circ data, ILL data) • Need to review, revise, reassess our BSC metrics and other • tools. Assessment is ongoing. • Managers and all staff need to learn how to better use data • and assessment results in daily operations. • Continue to move assessment out of MIS only. Position MIS • as consultant/coordinator or assessment done throughout • the organization. • Assessment is for everyone!