gaze contingent displays review and current trends n.
Skip this Video
Download Presentation
Gaze-Contingent Displays: Review and Current Trends

Loading in 2 Seconds...

play fullscreen
1 / 24

Gaze-Contingent Displays: Review and Current Trends - PowerPoint PPT Presentation

  • Uploaded on

Adaptive Displays Conference 2004. Gaze-Contingent Displays: Review and Current Trends. Andrew T. Duchowski. Acknowledgements. National Science Foundation NASA Ames DoD / Navy / SPAWAR Charleston Students Nathan Cournia Hunter Murphy Scott Gibson (Summer Research Internship). Overview.

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation

PowerPoint Slideshow about 'Gaze-Contingent Displays: Review and Current Trends' - malise

Download Now An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
  • National Science Foundation
  • NASA Ames
  • DoD / Navy / SPAWAR Charleston
  • Students
    • Nathan Cournia
    • Hunter Murphy
    • Scott Gibson (Summer Research Internship)
  • Brief review of Gaze-Contingent Displays (GCDs)
  • “Tuning” of GCDs via real-time metrics:
    • update rates
    • display / interface design
  • Current trends
    • GPU-programmable image processing
  • Eye tracking technology: state-of-the-art
gaze contingent displays
Gaze-Contingent Displays
  • Motivation:
    • minimize display or user’s attentional bandwidth
    • balance displayed information against visual information processing capacity (via eye tracking)
  • Generally, partition display in two distinct regions:
    • high-resolution foveal region
    • low-resolution peripheral surround
  • Critical concerns:
    • how, what, and when to display in the periphery
gcd strategies
GCD Strategies
  • Three approaches (low- to high-level):
    • screen-based displays (manage pixels)
    • model-based displays (manage graphics objects)
    • Attentive User Interfaces (manage interface)
  • Basic assumption:
    • observer’s real-time focus of attention coincides with fixation point (eye movements)
    • measured unobtrusively by eye tracker
attentive user interfaces
Attentive User Interfaces

Fig.1: AUIs; (a) eyeCONTACT sensor, (b) Light fixture with eyeCONTACT sensor, (c) eyePROXY, (d) attentive TV

(Courtesy Roel Vertegaal, see Shell et al. (2003))

model based approach
Model-Based Approach
  • Model-Based Gaze-Contingent Applications:
    • manipulate geometry just prior to display
    • degrade resolution of peripherally located objects
  • Not as much progress in this area (vs. screen-based) despite recent advancements in Level Of Detail (LOD) modeling techniques
  • Most relevant established technique is that of isotropic LOD management, as originally proposed by Clarke (1976)
model based approach1
Model-Based Approach
  • Isotropic LOD management:
    • pre-computed fine-to-coarse object hierarchies
    • resolution is uniformly degraded as object recedes from view (based on projected pixel area coverage)
  • Isotropic LOD management not always desirable, especially when viewing large objects close up
  • Due to advancements in multiresolution modeling, it is now becoming feasible to extend LOD approach to nonisotropic object rendering
model based examples
Model-Based Examples
  • Luebke et al.’s (2002) gaze-contingent LOD graphics
  • O’Sullivan et al.’s (2002) temporal graphics degradation

Fig.2: gaze-contingent graphics

model based examples1
Model-Based Examples
  • Our work:
    • Gaze-contingent terrain generation (AAAI Symposium 2000)
    • Gaze-contingent graphics modeling (EuroGraphics 2001)

Fig.3: gaze-contingent terrain, model

model based metrics
Model-Based Metrics
  • Key questions: is it worth it?
    • how much to degrade, where?
    • does degradation reduce rendering time?
    • how does degradation impact user?
  • Parhurst and Niebur (2004) discuss classic speed / accuracy tradeoff:
    • degradation improves display speed (interaction)
    • reaction times GCDs impede target indentification
screen based approach
Screen-Based Approach
  • Strategy:
    • manipulate framebuffer just prior to display
    • periphery is often masked or smoothed to compress image information (bits-per-pixel)
  • Good deal of progress in this area since design of early eye-slaved flight simulators
  • Research is rooted with classic Psychology research on reading perception and “moving window” paradigm (McConkie and Rayner 1975)
screen based image coding
Screen-Based Image Coding

(a) Recorded scanpath

(b) Reconstructed image

Fig.4: HVS-matching wavelet coefficient scaling

(Haar wavelets emphasizing degradation effects)

screen based video coding
Screen-Based Video Coding
  • Bergström (2003) incorporates MAR-based visual acuity model into transform coders

Fig.5: The Barbara image DCT coded (left) and MDCT coded (right). The focus point is in the centre of the MDCT coded image.

Fig.6: Barbara coded by the DWT coder (left) and the MDWT coder (right).

screen based metrics
Screen-Based Metrics
  • Loschky and McConkie (2000) found that to be imperceptible, image changes must start within 5 ms following end of saccade
  • Parkurst et al. (2000) found visual search performance with 5º window comparable to uniform resolution display

Fig.7: Gaze-contingent multi-resolution displays

current trends
Current Trends
  • Good deal of work recurring in screen-based approaches:
    • introduction of arbitrary visual maps
    • hardware-accelerated processing
    • new applications (fisheye displays)
  • New research facilitated by rapid progress in eye tracking technology
arbitrary visual maps
Arbitrary Visual Maps
  • Perry and Geisler (2002) introduced arbitrary visual fields
  • An important advancement since degradation function of any shape can be created
  • Simulations of visual dysfunctions possible (e.g., glaucoma)

Fig.8: Arbitrary Visual Fields

hardware accelerating imaging
Hardware-Accelerating Imaging
  • GPU programs allow per-fragment resolution selection from mipmap pyramid
  • For image-based GCDs, image processing bottleneck effectively eliminated
  • Peripheral color degradation now possible

Fig.9: GPU-programmable mipmap lookup

new applications
New Applications
  • PDT lens is a type of focus plus context screen with focal area magnified
  • Gaze-Contingent applications as yet unexplored

Fig.10: Pliable Display Technology (PDT) lens

eye tracking technology
Eye Tracking Technology
  • 1st generation: plaster-of-paris, scleral coils
    • eye-in-head measurements; invasive
  • 2nd generation: photo- and video-oculography
  • 3rd generation: video-based corneal reflection
  • 4th generation: digital video, DSP computer vision algorithms (for face, eye-in-head detection)
    • much easier to use, still as accurate and fast
    • still not auto-calibrating, but getting closer
state of the art
  • State-of-the-art eye trackers

facilitate rapid application dev.

  • Key GCD research question:
    • perception or performance?
  • Real-time applications:
    • simulation, telecommunication, etc.
  • Other directions:
    • diagnostic (off-line) uses, testing purposes, etc.

Fig.11: Tobii eye tracker


Bergström, P. (2003). Eye-Movement Controlled Image Coding. PhD Dissertation (No. 831), Institute of Technology, Linköping University, Linköping, Sweden.

Clarke, J. H. (1976). Hierarchical Geometric Models for Visible Surface Algorithms. Communications of the ACM 19, 10, 547-554.

Danforth, R., Duchowski, A., Geist, R., McAliley (2000). A Platform for Gaze-Contingent Virtual Environments. In Smart Graphics (Papers from the 2001 AAAI Spring Symposium, Technical Report SS-00-04), Menlo Park, CA, AAAI, pp. 66-70.

Luebke, D., Reddy, M., Cohen, J., Varshney, A., Watson, B., and Huebner, R. (2002). Level of Detail for 3D Graphics. Morgan-Kaufmann Publishers, San Francisco, CA.

McConkie, G. W. and Rayner, K. (1975). The Span of the Effective Stimulus During a Fixation in Reading. Perception & Psychophysics 17, 578-586.


Murphy, H. and Duchowski, A. (2001). Gaze-Contingent Level Of Detail. In EuroGraphics (Short Presentations), Manchester, UK, EuroGraphics.

O’Sullivan, C., Dingliana, J., and Howlett, S. (2002). Gaze-Contingent Algorithms for Interactive Graphics. In The Mind’s Eye: Cognitive and Applied Aspects of Eye Movement Research, J. Hyöna, R. Radach, and H. Duebel, Eds., Elsevier Science, Oxford, England.

Parkhurst, D., Culurciello, E., and Niebur, E. (2000). Evaluating Variable Resolution Displays with Visual Search: Task Performance and Eye Movements. In Eye Tracking Research & Applications Symposium p.105-109. Palm Beach Gardens, FL.

Parkhurst, D. J., & Niebur, E. (2004). A Feasibility Test for Perceptually Adaptive Level of Detail Rendering on Desktop Systems. In Applied Perception and Graphics Visualization (APGV). ACM, Los Angeles, CA, to appear.


Perry, J. S. and Geisler, W. S. (2002). Gaze-Contingent Real-Time Simulation of Arbitrary Visual Fields. In Human Vision and Electronic Imaging, San Jose, CA, SPIE.

Shell, J. S., Selker, T., and Vertegaal, R. (2003). Interacting with Groups of Computers, Communications of the ACM 46, 3 (March), 40-46.