Revisitation Patterns and Disorientation Eelco Herder Session on Empirical Studies ABIS Workshop 2003 Karlsruhe

1. Revisitation Patternsand Disorientation Eelco Herder Session on Empirical Studies ABIS Workshop 2003 Karlsruhe PALS Research Partners anywhere

2. Presentation Overview Introduction Prediction of Disorientation Description of Pilot Study Experimental Results Discussion Revisitation Patterns and Disorientation

3. Becoming Disoriented in a Web Site Symptoms Where am I How did I come here Where can I go to • Due to • non-linear structure of web sites • lack of context information • unfamiliarity with site or domain Introduction

4. Context Information Prevents Disorientation • Contextual links • e.g. menus, indexes and site maps • define users’ spatial or temporal context • are needed to understand the site structure • differ from site to site • Associative links • embedded in the content • interlink semantically related concepts • do not expose the site structure Contextual Navigation Support

5. Presentation Overview Introduction Prediction of Disorientation Description of Pilot Study Experimental Results Discussion Revisitation Patterns and Disorientation

6. Task Efficiency and Disorientation Users may be exploring the site or looking for specific information. Users that are exploring a site take some disorientation for granted, similar to tourists in a city center. We want to find out when people perceive their sense of disorientation as a problem. Goal Directed Exploring Navigation Patterns

7. Site Graph and Navigation Graph Web sites are commonly modeled as directed graphs User navigation paths can be seen as an overlay of the site graph 1 2 3 4 5 6 Prediction of Disorientation

8. Characteristics of User Navigation Navigation graphs describe browsing behavior Path length and linearity Revisitation (cycles and clustering) Analysis of navigation patterns helps in answering questions: Is the user exploring or is he lost? Does the user understand the site structure? Prediction of Disorientation

9. Revisitation and Lostness Contradictory experimental results found in literature P.A. Smith Users who cannot find desired information are lost. Lostness measure based on number of revisits: the more revisits, the more likely the user is lost J.E. McEneaney Well-performing users return often to navigational landmarks. The more linear the path, the worse the performance Prediction of Disorientation

10. Presentation Overview Introduction Prediction of Disorientation Description of Pilot Study Experimental Results on Revisitation Discussion Revisitation Patterns and Disorientation

11. Experimental Design (1) Objectives Find relations betweennavigation patterns and user satisfaction Subjects 30 randomly selected Dutch students(15 from Utrecht, 15 from Twente) Session Length Complete session lasted 2.5 hour Navigation session lasted only 40 minutes Description of Pilot Study

12. Experimental Design (2) Data Gathering Methods Questionnaires (motivational factors, expertise, evaluation) Performance tests (working memory, spatial capacity, episodic memory) Observation and interview Thinking aloud protocols(recorded on tape) Navigation data capturing (Scone) Description of Pilot Study

13. Web Sites Used and Example Tasks What are the definitions of personal budget and financial goal as they are presented within this website? Calculate how much the real value of £ 7.826 will decrease in 5 years given an annual inflation of 3%. Description of Pilot Study

14. Access Tracking Software path visualizations site metrics navigation metrics inference methods Analysis software Graph Package Description of Pilot Study

15. A Navigation Graph Description of Pilot Study

16. Presentation Overview Introduction Prediction of Disorientation Description of Pilot Study Experimental Results Discussion Revisitation Patterns and Disorientation

17. Revisitation Patterns • Percentage of Revisits • About 40% for personal finance tasks,about 25% for browsing on Amazon • (Tauscher: revisitation rate of 58% on long term) • Use of Back Button • On average 9,2 % of navigation actions. • Large differences between sites (3,3 – 15,8 - 7,8) • User tendencies are comparable Experimental Results

18. Revisitation: ‘I will find my way back later’ Average Connected Distance (ACD) The expected length of the path betweenany connected two pages x and y in the navigation graph. What Does It Mean? A higher ACD indicates that users will visit more pages before they return to a certain pageby a direct link, instead of by multiple clicks on the back button ACD and working memory positively correlated Experimental Results

19. ACD vs Number of Revisits ‘I will find my way back later’ means less revisitation Experimental Results

20. Return Rate Return rate The average number of times that a user will revisit a page. Calculated by averaging the number of visits on all pages that have been visited at least twice. What Does It Mean? A higher return rate means that users returnto a specific set of pages, most likely with a specific goal in mind. Experimental Results

21. ACD, Return Rate and Disorientation People with high Average Connected Distance have low revisitation rates – and low back button rates They also make more intensive use of landmarks (once a page is revisited once, they come back more often) Intensive use of landmarks is negatively correlated with disorientation. Experimental Results

22. Average Time Spent on Pages Median View Time The average time a user spends on a page while browsing. In contrast, the average view time is overly influenced by a small set of ‘high content’ pages. Our Findings Users who feel disoriented, spend more time on pages while browsing.The correlation is not very strong, though. Experimental Results

23. Presentation Overview Introduction Prediction of Disorientation Description of Pilot Study Experimental Results Discussion Revisitation Patterns and Disorientation

24. Revisitation Patterns and Disorientation Combined measures on user navigation help in predicting users becoming disoriented. Revisitation patterns are more accurate thanthe amount of revisitation. We are still far away from perfection.Most likely uncertainty cannot be eliminated completely. Discussion

25. Leveraging Lostness • Even with slightly imperfect prediction of disorientation it is still possible to offer adaptive navigation support. • Examples of adaptive contextual navigation aids: • personalized site maps • visualization of navigation history • direct guidance • link annotation • In order to do that, we must be able to distinguish between the various kinds of disorientation. Discussion

26. Limitations of this Study The site structure has not been taken into account, due to its complexity.We have evidence that the structure is an important factor. We haven’t yet been able to analyze the relations between the various measures in detail. This will involve time-consuming visual analysis, combined with more specific tasks and more detailed evaluation. Discussion

27. Revisitation Patterns and Disorientation Time for Questions email: herder@cs.utwente.nl PALS Research Partners anywhere