1 / 43

Windows of the Soul? What Eye Movements Tell Us About Software Usability

Windows of the Soul? What Eye Movements Tell Us About Software Usability. University of Virginia Charlottesville, Virginia August 24, 1998. Investigators. Steve Ellis - Principal Investigator

gail-wiley
Download Presentation

Windows of the Soul? What Eye Movements Tell Us About Software Usability

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Windows of the Soul?What Eye Movements Tell UsAbout Software Usability University of Virginia Charlottesville, Virginia August 24, 1998

  2. Investigators • Steve Ellis - Principal Investigator • Bell Labs / Lucent Technologies, Room 1K-411, 200 Laurel Ave, Middletown NJ 07748, shellis@lucent.com • Ron Candrea • Bell Labs / Lucent Technologies, Room 2D-425, candrea@lucent.com • Jason Misner • Systems Engineering, University of Virginia, Charlottesville, VA, 22903, jdm4s@virginia.edu • Christopher Sean Craig • University of Virginia, csc3b@virginia.edu • Christopher P. Lankford • University of Virginia, cpl2b@virginia.edu • Thomas E. Hutchinson • University of Virginia, teh@virginia.edu

  3. Goals Of Our Investigation • Assess the value of eye tracking technology for user interface evaluation and usability testing • Identify eye tracking technologies most suitable for the usability lab • Identify improvements in current eye tracking system capabilities for use in the usability lab • Develop guidelines for purchase and use of commercial eye tracking systems

  4. Previous Eye Movement-BasedPsychological Research and Applications • Reading Research (Just & Carpenter, 1980; Rayner, 1978) • Studies of Cognition and Elementary Information Processing (Just & Carpenter, 1976; Rayner, 1984) • Social Psychology/Interpersonal Communications (Kleinke, 1986) • Military Applications (Harris & Christhilf, 1980; Birkmire et al, 1993) • Marketing, Advertising (Krugman et al, 1994) • Usability Studies (Benel, Ottens & Horst, 1991; Card, 1984) • Other Ergonomic Studies (Wierda & Maring, 1993) • Bioengineering, Control (Jacob, 1991)

  5. Criteria for Choosing an Eye Tracking System for the Usability Lab • Unobtrusive • Easy and convenient to use • Few or no restrictions on user population • Gaze direction accuracy of 0.5º or better (i.e., 1 cm at 20 in). • Effective tracking range of 30º or better • High sampling rate • Inexpensive • Work with off-the-shelf applications • Powerful and flexible set of tools for data collection and analysis

  6. The Human Eye

  7. Subjective Comparison ofEye Tracking Techniques Lo Med Hi

  8. Pupil-Center/Corneal Reflection Method ERICA System

  9. ERICA System - Continued A view of the infrared image processed by ERICA. The relative position vector between the center of the bright-eye and glint determines the position of the user’s eye-gaze.

  10. A Pilot Study of Web Page Design • Evaluate eye tracking data as a user interface evaluation and design tool • Apply to a representative usability study of Web page design • Understand how best to collect, analyze and interpret the eye tracking data

  11. 4 tasks performed by all testers 1. Do xyz 2. Do abc 3. Do re me 4. Do nada Pilot Study Methodology 4 testers / page design; 1 page design / tester 4 Web page designs 16 independent testers

  12. Original Web Page Design • 12 items in web page. • Each item ranged from3 -12 lines of text with 5 - 9 words/line. • About half the graphics/images were animated. • Scroll about 5 times to see entire page. Testers saw only 2.5 items per screen.

  13. Many Hyperlinks Design • Tripled number of text hyperlinks.

  14. No-Image Web Page Design • Replaced graphics / images with text labels. • Original hyperlinks.

  15. Dense Text Design • Book-like, full-width layout. • Eliminate graphics / images, two-column appearance and bordered rows. • Scroll about 2 times to see entire page. Testers saw 5 of the 12 items on one screen.

  16. Tested Web Page Designs

  17. Data Collection and Analysis • Playback • Gaze Points - collected raw sampled data • Gaze Trails • Dynamically defined analysis measures • Fixation Points • LookZones • Pupil Diameter

  18. Playback • Real-time “video playback” • Tracks scrolling events

  19. Gaze Point Analysis —Raw Sampled Data • About 10 -12 samples/sec.

  20. Gaze Trails

  21. Gaze Trails with Gaze Points • Overlay the sampled gaze points on the gaze trails • Gaze trails optional • Pop-up details on mouse-over each gaze point

  22. Fixation Points - Example 1 • User defined during analysis, after capture. Parameters include: • Duration • Area • Minimum # gaze points • More accurate indication of tester’s viewing & processing

  23. Fixation Points - Example 2

  24. Fixation Point Parameter Setting

  25. Determining Fixation Points

  26. LookZones

  27. LookZone Analysis Notice that the subject, searching for the Wireless links, skipped over most pictures and focused on scanning the text blocks for targets.

  28. Bitmap Screen Analysis Options

  29. 3D LookZone Analysis In Bitmaps

  30. Vary Gaze Trail WidthBy Pupil Diameter

  31. Data Analysis - Review of Tasks • Task 1: Free Browse • Task 2: Directed Search • What is the name of Lucent’s new on-line technology magazine (“e-zine”)? • Task 3: Directed Search • What is the name of the software that lets you search for patterns in large masses of data? • Task 4: Directed Search and Judgment • How many links on this site will take you to pages dealing with “Wireless” technologies, such as that used in cellular phones.

  32. Comparison of TasksMeanTime to Complete Tasks

  33. Review of Web Site Designs • Four Screen Layouts • Original Text Block/Graphics Format • Many Hyperlinks • No Images • Dense Text

  34. Comparison of Web DesignsMean Time to Complete Tasks& * Note: Testers of “Original” design were least experienced group and probably exaggerated the times on this simpler design compared with testers using the busier “Many Hyperlinks” page. & Average of e-zine and wireless search tasks.

  35. Mean Time to First Fixationin Target LookZone& & E-zine search task P-value = 0.03

  36. Mean Number of Fixations& & Wireless search task P-value = 0.37

  37. Mean Average Fixation Duration& & Wireless search task P-value = 0.02

  38. Total Fixations inNon-Target LookZones& & E-zine search task P-value = 0.13

  39. Subjective Comparisonof Web Designs Highest

  40. Results and Conclusions • Correlation among most results • Dense text design elicited best performance • Eye movement data confirmed and provided insights into testers performance • Dense text design led to fewer, shorter fixations • Results should NOT be viewed as conclusive • Too few subjects, groups differed in skill level • Some experimental design flaws, confounds

  41. Next Steps • Replicate the study • Eliminate the potential confounds • (Systematic group differences, scrolling, etc.) • Benchmark subjects’ reading speed • Expand the study to see if results are generalizable • Extend eye tracking methodology to other software applications

  42. References • Benel, D. C. R., Ottens, D. & Horst, R. (1991). Use of an eyetracking system in the usability laboratory. Unpublished paper written for The Human Factors Society 35th Annual Meeting. San Francisco, CA • Birkmire, D. P., Karsh, R., Barnette, R. D., Pillalamarri, R. & DeBastiani, S. (1993). Eye movements in search and target acquisition. In Proceedings of the Human Factors and Ergonomics Society 37th Annual Meeting (pp. 1305-1309). Santa Monica, CA: Human Factors and Ergonomics Society • Card, S. K. (1984). Visual search of computer command menus. In H. Bouma & D. G. Bouwhuis (Eds.). Attention and Performance X: Control of Language Processes (pp. 67-96). London: Erlbaum • Harris, Sr. R. L. & Christhilf, D. M. (1980). What do pilots see in displays? In Proceedings of the Human Factors Society 24th Annual Meeting (pp. 22-26). Santa Monica, CA: Human Factors and Ergonomics Society • Jacob, R. J. K. (1991). The use of eye movements in human-computer interaction techniques: What you look at is what you get. ACM Transactions on Information Systems, 9 (April), 152-169 • Just, M. A. & Carpenter, P. A. (1976). The role of eye-fixation research in cognitive psychology. Behavior Research Methods and Instrumentation, 8, 139-143 • Just, M. A. & Carpenter, P. A. (1980). A theory of reading: From eye fixations to comprehension. Psychological Review, 87, 329-354 • Kleinke, C. L. (1986). Gaze and eye contact: A research review. Psychological Bulletin, 100, 78-100 • Krugman, D. M., Fox, R. J., Fletcher, J. E., Fischer, P. M. & Rojas, T. H. (1994). Do adolescents attend to warnings in cigarette advertising? A eye-tracking approach. Journal of Advertising Research, 34, 39-52 • Rayner, K. (1978). Eye movements in reading and information processing. Psychological Bulletin, 85, 618-660 • Rayner, K. (1984). Visual selection in reading, picture perception, and visual search: A tutorial review. In H. Bouma & D. G. Bouwhuis (Eds.), Attention and Performance X: Control of Language Processes (pp. 67-96). London: Erlbaum • Wierda, M. & Maring, W. (1993). Interpreting eye movements of traffic participants. Chapter 21 in D. Brogan, A. Gale & K. Carr (Eds.), Visual Search 2 (pp. 287-300)

More Related