1 / 27

Analyzing Data, Getting Results Some practical, not-too-burdensome tips and tricks

In this talk at the SCELC Research Day, Jenn Riley from the University of North Carolina at Chapel Hill explores how libraries can leverage evidence-driven decision-making to improve operations. Drawing on practical tips and techniques, she emphasizes the importance of collecting and analyzing data to inform purchasing decisions, evaluate service efficacy, and project future needs. With a focus on user-centered design and satisfaction metrics, Riley aims to empower library professionals to make informed, impactful changes, enhancing both service delivery and resource management.

lanza
Download Presentation

Analyzing Data, Getting Results Some practical, not-too-burdensome tips and tricks

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Analyzing Data, Getting ResultsSome practical, not-too-burdensome tips and tricks Jenn Riley University of North Carolina at Chapel Hill

  2. Evidence-driven decisions are a powerful guide for library operations. SCELC Research Day

  3. “The plural of anecdote is not data.” SCELC Research Day Sometimes attributed to Frank Kotsonis. After a quote with the opposite meaning, by Raymond Wolfiger.

  4. “There are three kinds of lies – lies, damned lies, and statistics.” SCELC Research Day Mark Twain, perhaps after Benjamin Disraeli.

  5. Using data for planning library operations Designing web sites and other online resources Existence/hours of service points Materials to buy/license/accept/digitize/keep/preserve Effectiveness of/satisfaction with procedures/services SCELC Research Day Evaluating a pilot service or project Projecting future expenditures

  6. Both cost and value are key ALCTS Heads of Technical Services in Large Research Libraries Interest Group, Task Force on Cost/Value Assessment of Bibliographic Control (2010) Proposes definitions of value for cataloging: SCELC Research Day Discovery success Use Display understanding Data interoperability Support for FRBR user tasks Throughput/timeliness Support administrative goals

  7. Example studies • By Joyce Chapman, then at North Carolina State University • Benefits of manually enhanced metadata for images • Comparing effort to utility for specific EAD elements SCELC Research Day See Chapman, Joyce. “Metrics & Management: Cost & value of metadata workflows.” SAA 2011. http://www.academia.edu/1708422/Return_on_Investment_Metadata_metrics_and_management

  8. Some common analyses Predicting impact of a change Cost per unit produced Change over time SCELC Research Day Identifying unmet needs Error/problem rate

  9. Back to library scenarios SCELC Research Day

  10. Existence/hours of service points • Who is using what and when? • How can we most effectively staff them? • Costs • Staff time • Facilities management costs • Benefits • Number and type of visitors, and how they use it • Service transactions completed • Specific services used at the location • Other data to collect • Usage by time of day • Calculate cost per transaction SCELC Research Day

  11. Materials to buy/license/accept/digitize/keep/preserve • Should we acquire, make more accessible, or keep this? • Costs • Initial purchase/license • Ongoing license/maintenance • Staff for cataloging/processing/digitizing/ingesting/preserving • Software • Hardware/storage • Benefits • Current and predicted future use • Opportunity for transformative use SCELC Research Day

  12. Evaluating a pilot service or project • Is the cost/benefit ratio appropriate? • What is the raw cost? • But it’s not all about cost/benefit: • Is the pilot achieving its aims? • Does this [whatever] do what we thought it would? • What collateral effects will it have? • Were the assumptions we made correct? • Data collection will be varied for this task SCELC Research Day

  13. Designing web sites and other online resources • A/B testing • User-centered design • Satisfaction surveys with previous iterations, similar sites, or prototypes • Web stats for previous iterations or similar sites • Task-based usability testing • Don’t forget the cost of sustaining it once you have it up! SCELC Research Day

  14. Effectiveness of/satisfaction with procedures/services • What parts of our current service are users most and least happy about? • What are the ineffieciences in our procedure for [whatever]? • Some data collection ideas • User surveys • Ratio of potential to actual users • Ratio of returning to non-returning users • Error/failure rates • Time from request to delivery • Time tracking during staff activity SCELC Research Day

  15. Projecting future expenditures • Equipment • Define its lifecycle • Amortize purchase cost • Add in maintenance costs • Compare to use as context • Staff • Educated guess at raises, turnover, benefit costs changes • Consider: • Inflation • Past trends • Upcoming sea changes SCELC Research Day

  16. Strategies for getting data that can be analyzed SCELC Research Day

  17. Tracking use • Circulation • COUNTER/SUSHI • Physical visitors • Web hits • Social media engagement • Attendance at events/sessions SCELC Research Day

  18. Tracking time • Can be effective when collected as a representative snapshot • Options for data collection • Clipboard next to a clock • Spreadsheet • Free time tracking apps • Make it as simple as possible SCELC Research Day

  19. Calculating costs • Staff time • 2080 hours per year is full time • Standard benefit percentages • Materials (including software) • Initial purchase • Maintenance contracts for big-ticket items • Amortize big costs over time in service • Overhead • Universities typically have standard rates SCELC Research Day

  20. Calculating error rates • Both objective and subjective criteria • Typically best when done as a sample • Consider both automated and manual means to locate errors for study SCELC Research Day

  21. Categorization • Putting things into like groups • Compare size of groups to one another • Compare effort spent on one group to another • Compare priority/value of one group to another • Can be done at time of data collection, or afterwards • Good idea to have some sense of categories at the beginning of the study SCELC Research Day

  22. Calculating benefit • Change in knowledge or status • Over time • After an interaction • Survey – ask about knowledge level before and after • Pre- and post-tests • Indirect measures • Number of people reached • Use SCELC Research Day

  23. Additional data analysis strategies SCELC Research Day

  24. Mechanics • Code qualitative data to make it processable • Make sure you pick a representative and consistent sample • Extrapolate based on known data when you need to • ALWAYS do a sanity check • Spreadsheets are your friend SCELC Research Day

  25. More advice • Context is key • Don’t be paralyzed by a perceived need for perfection • Know your basic analysis plans before you collect/identify data • Utilize pilot projects to generate data where there is none • Use the right tool for the job • Document your assumptions • It’s OK to use “napkin math” SCELC Research Day

  26. Get in the habit of collecting data.It will make your next decision easier. SCELC Research Day

  27. Thank you!Questions and discussion SCELC Research Day jennriley@unc.edu

More Related