1 / 65

Demonstrating Value and Creating Value: Evidence-Based Library Management through MINES for Libraries™

This article discusses the MINES methodology for library management, which includes a web-based survey to measure electronic resource usage. It also explores the impact of technology on librarian roles and the credibility of libraries in the digital age.

angiew
Download Presentation

Demonstrating Value and Creating Value: Evidence-Based Library Management through MINES for Libraries™

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Demonstrating Value and Creating Value:Evidence-Based Library Managementthrough MINES for Libraries™ Martha Kyrillidou Director of the ARL Statistics and Measurement Program Association of Research Libraries Brinley Franklin Vice Provost for University Libraries University of Connecticut Scholars Portal Forum, Ontario Council of University Libraries February 1 2006 Toronto http://www.minesforlibraries.org www.arl.org/stats/

  2. Today is Tomorrow - Peter Melinchok expressing dismay at his parents’ distorted sense of time http://www.minesforlibraries.org

  3. Bangor University considers removing librarians posted by Blake on Thursday January 27, @07:30AM -753 hits   Ms Information writes "News from the University of Wales Bangor in the UK. senior management no longer feel that subject librarians / academic liaison librarians are needed in the modern academic library. They have made restructuring proposals which include removing all bar one of the subject librarians and a tier of the library management, including the Head of Bibliographic Services. The university management thinks that technology has 'deskilled' literature searching. As far as I know, this proposal is unprecedented in the United Kingdom.In essence, there will remain 4 professional librarians serving a 'research-led' university of 8,000 plus FTEs and with 8 library sites. These will be the university librarian, cataloguing librarian, acquisitions librarian and Law librarian.Has anything like this happened anywhere that you know of? If so, what have been the effects? http://www.minesforlibraries.org

  4. http://www.minesforlibraries.org

  5. http://www.minesforlibraries.org

  6. http://www.minesforlibraries.org

  7. http://www.minesforlibraries.org

  8. Rhodes University – Anne Moon http://www.minesforlibraries.org

  9. Total Circulation Note. M. Kyrillidou and M. Young. (2003). ARL Statistics 2002-03. Washington, D.C.: ARL, p.8. http://www.minesforlibraries.org

  10. Reference Transactions Note. M. Kyrillidou and M. Young. (2003). ARL Statistics 2002-03. Washington, D.C.: ARL, p.8. http://www.minesforlibraries.org

  11. ARL Overall http://www.minesforlibraries.org

  12. Libraries Remain a Credible Resource in 21st Century 98% agree with statement, “My … library contains information from credible and known sources.” Note. Digital Library Federation and Council on Library and Information Resources. (2002). Dimensions and Use of the Scholarly Information Environment. http://www.minesforlibraries.org

  13. Changing Behaviors Recent Survey: Only 15.7% agreed with the statement “The Internet has not changed the way I use the library.” Note. Digital Library Federation and Council on Library and Information Resources. (2002). Dimensions and Use of the Scholarly Information Environment. http://www.minesforlibraries.org

  14. The Internet Goes to College Early data from ethnographic interviews • “I use Google because I heard it searches for more things” (than other sources). • “I believe I can find anything on the Internet. There hasn’t been anything I haven’t been able to find.” • “Because I’m lazy.” • Books have “so much information that no one can go through it all.” • I use “the Internet first is because it is more convenient.” • I go to the library “because that’s what teachers like.” • “Google has gotten me through college.” Source: Steve Jones, The Internet Goes to College, ARL Talk http://www.minesforlibraries.org

  15. … a revolution in making Il est plus nécessaire d'étudier les hommes que les livres —FRANÇOIS DE LA ROCHEFOUCAULD (1613–1680) http://www.minesforlibraries.org

  16. Into the future … • StatsQUAL+™ • ARL Statistics • E-Metrics • LibQUAL+™ • DigiQUAL+™ • MINES for Libraries™ http://www.minesforlibraries.org

  17. MINES for Libraries The MINES Survey Methodology Brinley Franklin Vice Provost for University Libraries University of Connecticut brinley.franklin@uconn.edu http://www.minesforlibraries.org www.arl.org/stats/

  18. What is MINES? • Action research • Set of recommendations for research design • Set of recommendations for web survey presentation • Set of recommendations for information architecture in libraries • Plan for continual assessment of networked electronic resources http://www.minesforlibraries.org

  19. MINES for LibrariesTM • MINES is a transaction-based research methodology consisting of a web-based survey form and a random moments sampling plan. • MINES typically measures who is using electronic resources, where users are located at the time of use, and their purpose of use. • MINES was adopted by the Association of Research Libraries (ARL) as part of the “New Measures” toolkit in May, 2003. • MINES is different from other electronic resource usage measures that quantify total usage (e.g., Project COUNTER, E-Metrics) or measure how well a library makes electronic resources accessible (LibQual+TM). http://www.minesforlibraries.org

  20. MINES for LibrariesTM Survey Form Five Questions and a Comment Box http://www.minesforlibraries.org

  21. Questions Addressed By MINES for Libraries™ for the OCUL Scholars Portal • How extensively do sponsored researchers use OCUL’s Scholars Portal? How much usage is for non-funded research, instruction/education, student research papers, and course work? • Are researchers more likely to use the Scholars Portal from inside or outside the library? What about other classifications of users? • Are there differences in Scholars Portal based on the user’s location (e.g., in the library; on-campus, but not in the library; or off-campus)? • Could MINES, combined with usage counts, provide an infrastructure to make Scholars Portal usage studies routine, robust, and easily integrated into OCUL’s administrative decision-making process for assessing networked electronic resources? http://www.minesforlibraries.org

  22. OCUL/MINES Methodological Considerations • The sampling plan was determined at the outset. Surveys were conducted once a month for two hours a month between May, 2004 and April, 2005 • The selection of the monthly survey periods were weighted based on usage counts by time of day and were chosen randomly. • Participation was mandatory, negating on-respondent bias, was based on actual use in real-time, and was brief (to minimize user inconvenience). • OCUL designed the local questions, mounted the survey, collected data and sent it to ARL for tabulation in aggregate and by individual institution. • If more than one search was conducted by a user, the survey form was auto-populated with initial responses as the default. http://www.minesforlibraries.org

  23. OCUL/MINES Methodological Considerations (continued) • Each participating library explained the survey and its confidentiality provisions to their local constituency. • Research ethics officers and/or Ethics Review Boards, where necessary, reviewed and approved the survey instrument and methodology. • OCUL determined that individual institutions and their institution-specific data collected during the survey periods would not be disclosed. Individual data was anonymous. • The mandatory nature of the survey required discussion on some campuses and caused one OCUL member library to withdraw from the study. • Two institutions pre-tested the survey in January, 2004. Data collection programming and configurations/links had to be revised in February and March, 2004. • After completing the survey, users were connected to their desired Scholars Portal networked electronic resource. http://www.minesforlibraries.org

  24. Issues with web surveys • Research design • Coverage error • Unequal access to the Internet • Internet users are different than non-users • Response rate • Response representativeness • Random sampling and inference • Non-respondents • Data security http://www.minesforlibraries.org

  25. MINES strategy • A representative sampling plan, including sample size, is determined at the outset. Typically, there are 48 hours of surveying over 12 months at a medical library and 24 hours a year at a main library. • Random moment/web-based surveys are employed at each site. • Participation is usually mandatory, negating non-respondent bias, and is based on actual use in real-time. • Libraries with database-to-web gateways or proxy re-writers offer the most comprehensive networking solution for surveying all networked services users during survey periods. http://www.minesforlibraries.org

  26. MINES strategy (continued) • Placement • Point of use • Not remembered, predicted or critical incident • Usage rather than user • What about multiple usages • Time out ? • Cookie or other mechanism with auto-population • Distinguish patron association with libraries. • For example, medical library v. main library. • But what if the resources are purchased across campus for all. Then how to get patron affiliation? http://www.minesforlibraries.org

  27. Web Survey Design Guidelines • Web survey design guidelines that MINES followed: • Presentation • Simple text for different browsers – no graphics • Different browsers render web pages differently • Few questions per screen or simply few questions • Easy to navigate • Short and plain • No scrolling • Clear and encouraging error or warning messages • Every question answered in a similar way - consistent • Radio buttons, drop downs • ADA compliant • Introduction page or paragraph • Easy to read • Must see definitions of sponsored research. • Can present questions in response to answers – for example if sponsored research was chosen, could present another survey http://www.minesforlibraries.org

  28. Quality Checks • Target population is the population frame – surveyed the patrons who were supposed to be surveyed - except in libraries with outstanding open digital collections. • Check usage against IP. In this case, big numbers may not be good. May be seeing the survey too often. • Alter order of questions and answers, particularly sponsored and instruction. • Spot check IP against self-identified location • Spot check undergraduates choosing sponsored research – measurement error • Check self-identified grant information against actual grants • Content validity – discussed with librarians and pre-tested. • Turn-aways – number who elected not to fill out the survey • Library information architecture -- Gateway v. HTML pages – there is a substantial difference in results. http://www.minesforlibraries.org

  29. Mandatory – UConn Libraries (3 months) http://www.minesforlibraries.org

  30. Optional – UConn Libraries (3 months) http://www.minesforlibraries.org

  31. Issues with web surveys:brief bibliography • Cook, Colleen; Heath, Fred; and Russell L. Thompson. 2000 (December). “A Meta-Analysis of Response Rates in Web- or Internet-Based Surveys.” Educational and Psychological Measurement 60(6): 821-836. • Couper, Mick P.; Traugott, Michael W.; and Lamias, Mark J. 2001. "Web Survey Design and Administration," Public Opinion Quarterly, 65 (2): 230-253. • Covey, Denise Troll. . 2002. Usage and Usability Assessment: Library Practices and Concerns. CLIR Publication 105. Washington DC: Council on Library and Information Resources. • http://www.clir.org/pubs/reports/pub105/contents.html • Dillman, D.A. 2000 (December). Mail and Internet Surveys, The Tailored Design Method. 2nd Ed. New York: John Wiley & Sons. • Gunn, Holly. 2002. “Web-based Surveys: Changing the Survey Process.” FirstMonday 7(12). • http://firstmonday.org/issues/issue7_12/gunn/index.html • LIBQUAL+ ™ Spring 2004 Survey. 2004. Cook, Colleen, and others. • http://www.libqual.org/documents/admin/ARL_Notebook2004.pdf • Schonlau, Matthias; Fricker Jr., Ronald D.; and Elliott, Marc N. 2002. Conducting Research Surveys via E-Mail and the Web. Santa Monica, CA: RAND. • Tenopir, Carol, with the assistance of Brenda Hitchcock and Ashley Pillow. 2003 (August). Use and Users of Electronic Library Resources: An Overview and Analysis of Recent Research Studies. Washington DC: Council on Library and Information Resources. • http://www.clir.org/pubs/reports/pub120/contents.htmls • Thomas, Susan J. 2004. Using Web and Paper Questionnaires for Data-Based Decision Making: From Design to Interpretation of the Results. Thousand Oaks, Corwin Press. • Thompson, Bruce.; Cook, Colleen.; Thompson, Russell L. 2002. Reliability and Structure of LibQUAL+™ scores: Measuring Perceived Library Service Quality. portal: Libraries and the Academy.3-12. http://www.minesforlibraries.org

  32. MINES for Libraries OCUL MINES What do the data tells us? Martha Kyrillidou Director of the ARL Statistics and Measurement Program Association of Research Libraries http://www.minesforlibraries.org http://www.minesforlibraries.org www.arl.org/stats/

  33. Questions Addressed By MINES for Libraries™ for the OCUL Scholars Portal • How extensively do sponsored researchers use OCUL’s Scholars Portal? How much usage is for non-funded research, instruction/education, student research papers, and course work? • Are researchers more likely to use the Scholars Portal from inside or outside the library? What about other classifications of users? • Are there differences in Scholars Portal based on the user’s location (e.g., in the library; on-campus, but not in the library; or off-campus)? • Could MINES, combined with usage counts, provide an infrastructure to make Scholars Portal usage studies routine, robust, and easily integrated into OCUL’s administrative decision-making process for assessing networked electronic resources? http://www.minesforlibraries.org

  34. MINES for LibrariesTM Survey Form Five Questions and a Comment Box http://www.minesforlibraries.org

  35. Analysis • Web deliverables: • Crosstabulations in html for all OCUL data • Interactive crosstabs for all OCUL and institutions • Print deliverables: • summary tables for OCUL • summary tables for each institution • Final report http://www.minesforlibraries.org

  36. Add them up and down … http://www.minesforlibraries.org

  37. http://www.minesforlibraries.org

  38. OCUL Scholars Portal UsageAffiliation http://www.minesforlibraries.org

  39. http://www.minesforlibraries.org

  40. http://www.minesforlibraries.org

  41. http://www.minesforlibraries.org

  42. Affiliation by Purpose of Use http://www.minesforlibraries.org

  43. User Status by Purpose of Use http://www.minesforlibraries.org

  44. Location by Purpose of Use http://www.minesforlibraries.org

  45. Reason for Use http://www.minesforlibraries.org

  46. http://www.minesforlibraries.org

  47. http://www.minesforlibraries.org

  48. http://www.minesforlibraries.org

  49. How extensively do sponsored researchers use OCUL’s Scholars Portal? How much usage is for non-funded research, instruction/education, student research papers, and course work? MINES for Libraries™shows that the Scholars Portal resources are heavily used by faculty and students in all OCUL. The majority of the use is from the sciences and the medical field and particularly in those fields the majority of the use is for sponsored research purposes. http://www.minesforlibraries.org

  50. Are researchers more likely to use the Scholars Portal from inside or outside the library? What about other classifications of users? Most faculty, graduate professionals and undergraduates uses of the Scholars Portal are from outside the library building. Undergraduates though do show many uses of the Scholars Portal from within the library as they are probably becoming more exposed to these resources by having more physical contact with the library. http://www.minesforlibraries.org

More Related