1 / 28

Analyzing the Integration of Figures and Text Using Eye Movements

Analyzing the Integration of Figures and Text Using Eye Movements. Jason Rubinstein With Eileen Kowler, Nick Ross, Elio Santos, Min Zhao University of Delaware and Rutgers. Introduction.

penha
Download Presentation

Analyzing the Integration of Figures and Text Using Eye Movements

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Analyzing the Integration of Figures and Text Using Eye Movements Jason Rubinstein With Eileen Kowler, Nick Ross, Elio Santos, Min Zhao University of Delaware and Rutgers

  2. Introduction The L–D differences in the ventilatory and metabolic variables occurred even between 15-min epochs of identical activity levels, whether these were below or above the daily mean (Fig. 5). VE/VO2 in the D remained higher than in the L, by 13% when activity was low and 9% at the higher activity level. The results of this analysis did not depend on the animal ’s activity immediately preceding the epoch analyzed, as there was no correlation between the L –D difference in VE for iso-activity epochs and the level of activity prior to the epochs analyzed. • Figures + Text • Same “page” vs. different “page” • Semantic integration of information over long timescales • Building up a knowledgebase • Between saccades • “rapid movements of the eyes between positions of rest” • Between “page turns” • Meaning • People tend to limit memory load when possible

  3. Ballard, Hayhoe, & Pelz, 1995

  4. Epelboim & Suppes, 2001

  5. Question What strategies do people use to extract meaning from figures and text (in combination)?

  6. Why? • Real world • Textbooks • Educational material • Standardized tests • SATs • Newspapers • Websites • Instruction manuals • Captions

  7. The percentage statements during which the participant looked at the person was used as an indicator of gaze seeking. There was not an effect of gaze cueing and no interaction between gaze cueing and full or partial instructions. For full instructions, the person was fixated less often when he provided gaze cues, than when he did not provide gaze cues. For partial instructions, gaze cues were often sought.

  8. The percentage statements during which the participant looked at the person was used as an indicator of gaze seeking. There was not an effect of gaze cueing and no interaction between gaze cueing and full or partial instructions. For full instructions, the person was fixated less often when he provided gaze cues, than when he did not provide gaze cues. For partial instructions, gaze cues were often sought.

  9. Which of the following is true? A. The percentage of fixations on the person was high for both full and partial instructions independently of whether the gaze cues were provided by the person or not. B. Fixations on the person were more frequent following full instructions with or without gaze cues. C. Gaze cues increased the percentage of fixations on the person for both full and partial instructions. D. For partial instructions, fixations on the person were rare whether gaze cues were provided or not.  

  10. Question • What strategies do people use to extract meaning from figures and text (in combination)? • Patterns in • Rule-governed transitions • Revisits • “Effector” (motor action) • Eye (simultaneous) or arm (sequential) vs

  11. VIDEO

  12. Results • Revisit figure after reading key words in text • Words that refer to the legend, specific aspects of figure, etc

  13. Results • Revisit figure after reading key words in text • Words that refer to the legend, specific aspects of figure, etc • On average… • Switch more in simultaneous condition (!)

  14. Number of visits to text Number of visits to figure

  15. Results • Revisit figure after reading key words in text • Words that refer to the legend, specific aspects of figure, etc • On average… • Switch more in simultaneous condition (!) • Though “easier” (cognitively) in alternating • Less “work”

  16. Results • Revisit figure after reading key words in text • Words that refer to the legend, specific aspects of figure, etc • On average… • Switch more in simultaneous condition (!) • Though “easier” (cognitively) in alternating • Less “work” • Periphery -- planning

  17. Results • Revisit figure after reading key words in text • Words that refer to the legend, specific aspects of figure, etc • On average… • Switch more in simultaneous condition (!) • Though “easier” cognitively in alternating • Less “work” • Periphery -- planning • Spend more time in text in both conditions • Longer visits + more time overall

  18. Proportion of time spent in figure Proportion of time spent in text

  19. Results • Revisit figure after reading key words in text • Words that refer to the legend, specific aspects of figure, etc • On average… • Switch more in simultaneous condition (!) • Though “easier” cognitively in alternating • Less “work” • Periphery -- planning • Spend more time in text in both conditions • Longer visits + more time overall • However, proportion of overall time between simultaneous and alternating about the same

  20. Average length of visits to figure (s) Average length of visits to text (s)

  21. Results • Revisit figure after reading key words in text • Words that refer to the legend,specific aspects of figure, etc • On average… • Switch more in simultaneous condition (!) • Though “easier” cognitively in alternating • Less “work” • Periphery -- planning • Spend more time in text in both conditions • Longer visits + more time overall • However, proportion of overall time between simultaneous and alternating about the same • Reading… • Images vs. words?

  22. What next? • MORE • Participants • Stimuli • Shorter trials/harder questions • Accuracy of various strategies • Specifics of transitions • Word in text → which part of figure? • Vice versa • Model revisits • Simultaneous trials • Saccade triggers stimuli • Periphery

  23. ?

  24. Thank you! • Eileen Kowler • Nick Ross, Elio Santos, Min Zhao • Matthew Stone • Joe, Henry, Avery, Rachel, Lara • The Academy • Everyone else

More Related