Crunching Numbers: OPAC Log Analysis of WebVoyage

1. 2007 SCVUGM, Stillwater, OK Crunching Numbers: OPAC Log Analysis of WebVoyage Bennett Claire Ponsford Digital Services Librarian Texas A&M University Libraries

2. Overview • Why analyze your log files • How to do it • What we found • The changes we made • What the latest logs say • What next?

3. Why Analyze? • To see how your users search when you’re not watching • To resolve internal disagreements over default searches, limits, etc. • To see whether changes to WebVoyage really improved search results • As a counterpoint to task-based user testing

4. C.S. Lewis Lewis, C. S. (Clive Staples) LION WITCH? LION, WITCH? LION, WITCH, AND WARDROBE? Lewis, C. S. (Clive Staples)

6. Issues to Think About • Does Voyager capture the data you need? • Privacy concerns • Does your network organize data the way you need? • Staff vs. public IP addresses • Do you want all searches or a sample?

7. How To • Read the documentation • Technical Manual, Chapter 15, Popacjob • Begin logging your data • Extract data into Access database • Clean up data as needed • Run queries • Scratch head and contact Tech Support

9. Data Fields

10. Data Fields (cont.)

11. Data Fields (cont.)

13. SQL for Count of Search Type • SELECT Fall_2007_OPAC_log.Search_type, Count(Fall_2007_OPAC_log.Search_type) AS CountOfSearch_type1 • FROM Fall_2007_OPAC_log • WHERE (((Fall_2007_OPAC_log.Hyperlink)="N") AND ((Fall_2007_OPAC_log.Search_tab)="1") AND ((Fall_2007_OPAC_log.Client_type)="W")) • GROUP BY Fall_2007_OPAC_log.Search_type • ORDER BY Fall_2007_OPAC_log.Search_type;

14. Results

15. June 2006 (Voyager 5)

16. September 2006 (Voyager 5) • Changed interface • Defaults • Kept Tab at Simple Search • Changed Search to Keyword (CMD* with javascript) • Changed result sort to by relevance

17. Fall 2006 • Preparing to upgrade to Voyager 6.1 • New keyword searches with ^ to automatically AND words together • Some people unhappy with recent changes • Default search • Search results sort order • Decided to look at the data

18. Decisions upgrading to V6 • Basic data • Where are our searchers • What search tab are they using • How are they searching • Default search • Order of title searches • Simple limits

19. Where Are Our Searchers?

20. What Search Tab Used?

21. Default Search: Discussion • Title search (TALL) • What we traditionally had used • Reference’s preference • General keyword search (new GKEY^*) • What users are used to in a Google world • More forgiving search

24. Default Search: Decision • General keyword search (new GKEY^*) • User preference • Fewer No Hit results

25. First Title Search: Discussion • Left anchored title (TALL) • Preferred by Reference • Title keyword (new TKEY^*) • More forgiving

26. Title Search (TALL): Problems

27. Title Search: Decision • Title keyword • Left-anchored title had too many problems

28. Simple Limits • Several additional location limits requested • Concern that too many would be confusing

29. Search Limits Used

30. Simple Limits: Decision • Added new limits and will evaluate with more data

32. Analysis of Voyager 6 Logs • Improved differentiation library staff and public IP addresses

33. Where are our users?

34. Keyword and Subject Searches

35. Author Searches

36. Title Searches

37. Location Limits Used

38. Comparison of Limits

39. Have Changes Helped? • Search frequency • No hits percentage

40. Search Frequency

41. No Hit Percentages

42. Detailed No Hit Percentages

43. Analysis of No Hit Searches • Do we have the title? • Why did the search not find it? • What can we do to help?

44. No Hit Title Searches: Do We Own Them?

45. No Hit Title Searches: Problems

46. What Next? • Continued analysis of searches with no hits • Analysis of search repair strategies • Word counts

47. Improvements: Spelling • Spellchecking • Automatic searching of variant spellings • “&” or “and” • British vs. American spellings • Numbers • Abbreviations • Did you mean? Suggestions based on field • Working on using ASPELL to create spellchecker

48. Improvements: Help • More granular no hits help • Specific search types • Any search with “conference” or “proceedings” in it • Journal title searches including “vol.”, “no.”, or a number • Searches with more than 4 or 5 words • More granular help for too many hits

49. Improvements: Specific Searches • Keyword searches • Automatic stemming • Ignore punctuation and spacing • Ignore stop words • Title searches • Ignore initial article

50. More Information • Jansen, Bernard J. “Search log analysis: What it is, what’s been done, how to do it,” Library & Information Science Research, 28 (2006) 407-432. • Yu, Holly and Margo Young, “The impact of Web search engines on subject searching in OPAC”, Information Technology and Libraries, 23 (2004) 168-180.