What Do Exploratory Searchers Look at in a Faceted Search Interface?

1. What Do Exploratory Searchers Look at in a Faceted Search Interface? Bill Kules and Matthew Banta The Catholic University of AmericaSchool of Library and Information Science

2. Outline • 60-second demo • Our goals • Research questions • Experimental design • Results • Future work

4. Our Goals • Improve understanding of how faceted interfaces affect searcher actions and tactics • Develop and validate a methodology for creating exploratory search tasks for evaluations of search systems

5. Research Questions • How long do searchers look at the major elements (facets, results, query box, breadcrumbs, etc.) of the interface? • In what order do searchers look at the major elements in a faceted search interface?

6. Experimental Design • N=18 successful sessions • From 21 subjects recruited • 1x2 within-subjects design • Task types • Exploratory tasks (n=4) • Known item tasks (n=2) • Counterbalanced within task type

7. Procedure • Introduction • 90 second training video • Calibrate eye tracker • Conduct 6 searches • Questionnaire after each search • Retrospective verbal report • Video of two searches with gaze data overlaid • Final questionnaire

8. Six Tasks • Exploratory • Feminism in the United States • Textile industry on three continents • Great Britain and the colonies in the 20th century • History of the Olympic games • Known item: find a book • Firefly Encyclopedia of Trees • Harry Potter and the Goblet of Fire

9. Exploratory Task Imagine you are taking a class called “Feminism in the United States”. For this class you need to write a research paper on some aspect of the U.S. feminist movement, but have yet to decide on a topic. Use the catalog to find two possible topics for your paper. Then use the catalog to find three books for each topic so that you might make a decision as to which topic to write about.

10. Equipment Configuration • NCSU Catalog Research Testbed • Tobii 2150 remote eye tracker • 21” monitor • 50 Hz sampling rate • Resolution 1024x768 • Tobii Clearview v2.7.1 • Gaze fixations • Minimum 100 ms • 30 pixel radius • Manually segmented AOIs

11. Interface with Areas of Interest (AOIs) Query Breadcrumbs Facets Results

12. How long did searchers look at the major elements of the interface?

13. Exploratory Search Tasks Known ItemSearch Tasks

14. For exploratory search tasks

15. In what order did searchers look at the major elements?

16. Transitions in Attention Between AOIs

17. Interface with Areas of Interest (AOIs) Query Breadcrumbs Facets Results

18. Post-search Interviews • “The subject thing worked. I don’t normally do subject searches.” • “I needed a subject and I didn’t want to look through 2000 books.” • “shopping around” - selecting facets and then looking to see what was available for a particular subject

19. Limitations • Training was provided • Researcher-provided tasks in lab setting • One high-level scenario • Tasks constructed were focused on this study

20. Future Work • Additional factors • Training – Next study supported by OCLC/ALISE grant and CUA purchase of an eye tracker • Number and size of facets • Domain, search knowledge • Additional measures • Gaze behavior – e.g. fixation counts • “Traditional” measures – e.g. clicks • Refine procedure for exploratory search task generation

21. Conclusions • Facets played a major role in exploratory searches • Fixation time about ½ as much as on results • On first page about equal • Facet-result & result-breadcrumb ~ ½ of all transitions

22. Acknowledgements • Rob Capra, Tito Sierra, Jason Casden, and Joe Ryan • Doug Oard and members of the UMD HCIL – for the use of their facilities and eye-tracker • This research was supported by a grant from Catholic University and in part by a grant from the NSF/Library of Congress (IIS 0455970).