How to Identify Your Impact: The Value of Libraries

1. How to Identify Your Impact:The Value of Libraries Joe Matthews June 10, 2013

2. Megan Oakleaf

3. Outline • Performance Measures • Value • Value of Information • Value of Information Services

4. Outline • Value of a Library • Personal value • Direct measures • Indirect measures • Organizational value • Financial value

5. Few libraries exist in a vacuum, accountable only to themselves. There is thus always a larger context for accessing library quality, that is, what and how well does the library contribute to achieving the overall goals of the parent constituencies? Sarah Pritchard

6. There is no systematic evidence collected which shows the value of academic libraries for teaching and research staff. Claire Creaser and Valerie Spezi

7. Performance Measures Library Services Society Individual Input Process Output Outcomes Outcomes Resources Capability Use Beneficial effects Effectiveness Efficiency Impact Cost Effectiveness Cost benefit VALUE

8. Start with the end in mind: work backwards Refocus from the activity to the impact

9. Impact

10. Library Control

11. Library & Customers Decide

12. Customers Decide

13. Live by the numbers, ….

14. Challenges • Lack of consensus about what should be measured and how • Lack of understanding of performance measurement and metrics • Organizational structural issues • Lack of precision in measuring performance, and alignment issues • Determining the “bottom line” is too far away • Majority of stakeholders are too far away • Library staff find it difficult to see the “big” picture

15. And the survey said ….

16. Lack of a Connection • Budget and outputs (and outcomes) are separated • No “bottom line” measure for libraries • Decision-making process is bigger than the library • Library has neither champions nor foes • Library benefits are not widely self-evident

17. Orr’s Fundamental Questions • How good is the library? • What good does the library do? • How well is the library managed?

18. We should be a bit wary of the “little library” …For when it is good, it is very, very good and when it is bad, it’s a “pretty good library for a town this size.” Eleanor Jo Rodger

20. Levels of Assessment … • Individual student • Course • Departmental/Program • College or University

21. Types of Measures • Direct • Provide tangible, visible and self-explanatory evidence of what students have & have not learned • Indirect • Capture students’ perceptions of their knowledge & skills; supplement direct measures; sometimes called surrogates

22. Qualitative Tools • Focus groups – open ended • Biography • Phenomenology – capture the “Aha!” moment • Grounded theory • Ethnography • Case study

23. Qualitative Assessment • Provides in-depth understanding of user responses and interactions • Represents part of a long-term strategy of formative evaluative

24. Quantitative Tools • Surveys • Transaction logs • Statistics from systems • Observations (count)

25. Quantitative Assessment • Analyses to determine library impacts on academic performance, retention rates • Describe retention rates and GPAs in defined populations over semesters and users • Compare users & non-users of library services while adjusting for academic preparation and background differences • Conduct quasi-experimental designs employing multivariate analysis of covariance & hierarchical regression techniques

26. Useful Assessment

27. Be cautious about cause-and-effect relationships

28. The Issue • Is it: Use library resources/services and you will get better grades. • Or: I want to do well and so I work hard to achieve better grades - and one way I do that is to use library resources/services.

29. “Not surprisingly, librarians are keen to show that the use of expensive, scholarly materials positively correlates with higher grades, although they cannot prove that this is so.” Deborah Goodall & David Pattern

30. “There is growing pressure on all academic library managers to be more accountable for how they use limited resources and to achieve institutional outcomes perceived as important by college and university stakeholders….” ElizabethMezick

34. Value of Information • Expect value-in-use • Library’s collection reflects a “potential value” • Collection also reflects a “future value” • Value of local collection is declining

35. Valuable is not about our professional values; in the paradigm of the value of public libraries, we are the producers, not the consumers of services. Our sense of what is valuable really doesn’t matter much at all unless it matches that our our customers. Eleanor Jo Rodger

36. Fundamental Changes Libraries have changed more in the past two decades than in the prior two centuries. Technology is the major driver . . . We need to recognize that all this change has only begun, and that change is irreversible.

37. Increasingly it is important to remember that libraries provide few unique services.

38. Information is woven into our lives

39. Quality of Information This fast food approach to information consumption drives librarians crazy. “Our information is healthier and tastes better too” they shout. But nobody listens. We’re too busy Googling.” Peter Morville

40. Key Characteristics of Information

41. Criteria for Judging Value

42. Collections are disrupted Atoms to bits

44. Nature of Information is Changing Information is …. Information was …. All around us Cheap or free Shaped by consumers Designed for sharing, participation & feedback Immediate Embedded in our worlds Scare, controlled Expensive Shaped by elites One-way, mass consumption Slow moving External to our worlds

46. Value of the Academic Library

47. If the physical proximity of print collections had a demonstrable impact on researcher productivity, no university would hesitate to allocate prime real estate to library stacks.

48. Traditional Value Proposition Without a great library, there can be no great university. David Kinly, President of The University of Illiniois 1929

49. Universities Provide • Private goods & services • Courses exchanged for tuition • Research completed for funding • The value proposition The value to an individual or an organization determines whether payment is made for the service

50. Academic Libraries Provide • Public goods and services Print and online resources are shared by all, usually without the exchange of payment • Value proposition The collective value of all users must be estimated to determine if a good or service should be continued