Gaze contingent displays review and current trends
This presentation is the property of its rightful owner.
Sponsored Links
1 / 24

Gaze-Contingent Displays: Review and Current Trends PowerPoint PPT Presentation


  • 170 Views
  • Uploaded on
  • Presentation posted in: General

Adaptive Displays Conference 2004. Gaze-Contingent Displays: Review and Current Trends. Andrew T. Duchowski. Acknowledgements. National Science Foundation NASA Ames DoD / Navy / SPAWAR Charleston Students Nathan Cournia Hunter Murphy Scott Gibson (Summer Research Internship). Overview.

Download Presentation

Gaze-Contingent Displays: Review and Current Trends

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Gaze contingent displays review and current trends

Adaptive Displays Conference 2004

Gaze-Contingent Displays:Review and Current Trends

Andrew T. Duchowski


Acknowledgements

Acknowledgements

  • National Science Foundation

  • NASA Ames

  • DoD / Navy / SPAWAR Charleston

  • Students

    • Nathan Cournia

    • Hunter Murphy

    • Scott Gibson (Summer Research Internship)


Overview

Overview

  • Brief review of Gaze-Contingent Displays (GCDs)

  • “Tuning” of GCDs via real-time metrics:

    • update rates

    • display / interface design

  • Current trends

    • GPU-programmable image processing

  • Eye tracking technology: state-of-the-art


Gaze contingent displays

Gaze-Contingent Displays

  • Motivation:

    • minimize display or user’s attentional bandwidth

    • balance displayed information against visual information processing capacity (via eye tracking)

  • Generally, partition display in two distinct regions:

    • high-resolution foveal region

    • low-resolution peripheral surround

  • Critical concerns:

    • how, what, and when to display in the periphery


Gcd strategies

GCD Strategies

  • Three approaches (low- to high-level):

    • screen-based displays (manage pixels)

    • model-based displays (manage graphics objects)

    • Attentive User Interfaces (manage interface)

  • Basic assumption:

    • observer’s real-time focus of attention coincides with fixation point (eye movements)

    • measured unobtrusively by eye tracker


Attentive user interfaces

Attentive User Interfaces

Fig.1: AUIs; (a) eyeCONTACT sensor, (b) Light fixture with eyeCONTACT sensor, (c) eyePROXY, (d) attentive TV

(Courtesy Roel Vertegaal, see Shell et al. (2003))


Model based approach

Model-Based Approach

  • Model-Based Gaze-Contingent Applications:

    • manipulate geometry just prior to display

    • degrade resolution of peripherally located objects

  • Not as much progress in this area (vs. screen-based) despite recent advancements in Level Of Detail (LOD) modeling techniques

  • Most relevant established technique is that of isotropic LOD management, as originally proposed by Clarke (1976)


Model based approach1

Model-Based Approach

  • Isotropic LOD management:

    • pre-computed fine-to-coarse object hierarchies

    • resolution is uniformly degraded as object recedes from view (based on projected pixel area coverage)

  • Isotropic LOD management not always desirable, especially when viewing large objects close up

  • Due to advancements in multiresolution modeling, it is now becoming feasible to extend LOD approach to nonisotropic object rendering


Model based examples

Model-Based Examples

  • Luebke et al.’s (2002) gaze-contingent LOD graphics

  • O’Sullivan et al.’s (2002) temporal graphics degradation

Fig.2: gaze-contingent graphics


Model based examples1

Model-Based Examples

  • Our work:

    • Gaze-contingent terrain generation (AAAI Symposium 2000)

    • Gaze-contingent graphics modeling (EuroGraphics 2001)

Fig.3: gaze-contingent terrain, model


Model based metrics

Model-Based Metrics

  • Key questions: is it worth it?

    • how much to degrade, where?

    • does degradation reduce rendering time?

    • how does degradation impact user?

  • Parhurst and Niebur (2004) discuss classic speed / accuracy tradeoff:

    • degradation improves display speed (interaction)

    • reaction times GCDs impede target indentification


Screen based approach

Screen-Based Approach

  • Strategy:

    • manipulate framebuffer just prior to display

    • periphery is often masked or smoothed to compress image information (bits-per-pixel)

  • Good deal of progress in this area since design of early eye-slaved flight simulators

  • Research is rooted with classic Psychology research on reading perception and “moving window” paradigm (McConkie and Rayner 1975)


Screen based image coding

Screen-Based Image Coding

(a) Recorded scanpath

(b) Reconstructed image

Fig.4: HVS-matching wavelet coefficient scaling

(Haar wavelets emphasizing degradation effects)


Screen based video coding

Screen-Based Video Coding

  • Bergström (2003) incorporates MAR-based visual acuity model into transform coders

Fig.5: The Barbara image DCT coded (left) and MDCT coded (right). The focus point is in the centre of the MDCT coded image.

Fig.6: Barbara coded by the DWT coder (left) and the MDWT coder (right).


Screen based metrics

Screen-Based Metrics

  • Loschky and McConkie (2000) found that to be imperceptible, image changes must start within 5 ms following end of saccade

  • Parkurst et al. (2000) found visual search performance with 5º window comparable to uniform resolution display

Fig.7: Gaze-contingent multi-resolution displays


Current trends

Current Trends

  • Good deal of work recurring in screen-based approaches:

    • introduction of arbitrary visual maps

    • hardware-accelerated processing

    • new applications (fisheye displays)

  • New research facilitated by rapid progress in eye tracking technology


Arbitrary visual maps

Arbitrary Visual Maps

  • Perry and Geisler (2002) introduced arbitrary visual fields

  • An important advancement since degradation function of any shape can be created

  • Simulations of visual dysfunctions possible (e.g., glaucoma)

Fig.8: Arbitrary Visual Fields


Hardware accelerating imaging

Hardware-Accelerating Imaging

  • GPU programs allow per-fragment resolution selection from mipmap pyramid

  • For image-based GCDs, image processing bottleneck effectively eliminated

  • Peripheral color degradation now possible

Fig.9: GPU-programmable mipmap lookup


New applications

New Applications

  • PDT lens is a type of focus plus context screen with focal area magnified

  • Gaze-Contingent applications as yet unexplored

Fig.10: Pliable Display Technology (PDT) lens


Eye tracking technology

Eye Tracking Technology

  • 1st generation: plaster-of-paris, scleral coils

    • eye-in-head measurements; invasive

  • 2nd generation: photo- and video-oculography

  • 3rd generation: video-based corneal reflection

  • 4th generation: digital video, DSP computer vision algorithms (for face, eye-in-head detection)

    • much easier to use, still as accurate and fast

    • still not auto-calibrating, but getting closer


State of the art

State-of-the-Art

  • State-of-the-art eye trackers

    facilitate rapid application dev.

  • Key GCD research question:

    • perception or performance?

  • Real-time applications:

    • simulation, telecommunication, etc.

  • Other directions:

    • diagnostic (off-line) uses, testing purposes, etc.

Fig.11: Tobii eye tracker


References

References

Bergström, P. (2003). Eye-Movement Controlled Image Coding. PhD Dissertation (No. 831), Institute of Technology, Linköping University, Linköping, Sweden.

Clarke, J. H. (1976). Hierarchical Geometric Models for Visible Surface Algorithms. Communications of the ACM 19, 10, 547-554.

Danforth, R., Duchowski, A., Geist, R., McAliley (2000). A Platform for Gaze-Contingent Virtual Environments. In Smart Graphics (Papers from the 2001 AAAI Spring Symposium, Technical Report SS-00-04), Menlo Park, CA, AAAI, pp. 66-70.

Luebke, D., Reddy, M., Cohen, J., Varshney, A., Watson, B., and Huebner, R. (2002). Level of Detail for 3D Graphics. Morgan-Kaufmann Publishers, San Francisco, CA.

McConkie, G. W. and Rayner, K. (1975). The Span of the Effective Stimulus During a Fixation in Reading. Perception & Psychophysics 17, 578-586.


References1

References

Murphy, H. and Duchowski, A. (2001). Gaze-Contingent Level Of Detail. In EuroGraphics (Short Presentations), Manchester, UK, EuroGraphics.

O’Sullivan, C., Dingliana, J., and Howlett, S. (2002). Gaze-Contingent Algorithms for Interactive Graphics. In The Mind’s Eye: Cognitive and Applied Aspects of Eye Movement Research, J. Hyöna, R. Radach, and H. Duebel, Eds., Elsevier Science, Oxford, England.

Parkhurst, D., Culurciello, E., and Niebur, E. (2000). Evaluating Variable Resolution Displays with Visual Search: Task Performance and Eye Movements. In Eye Tracking Research & Applications Symposium p.105-109. Palm Beach Gardens, FL.

Parkhurst, D. J., & Niebur, E. (2004). A Feasibility Test for Perceptually Adaptive Level of Detail Rendering on Desktop Systems. In Applied Perception and Graphics Visualization (APGV). ACM, Los Angeles, CA, to appear.


References2

References

Perry, J. S. and Geisler, W. S. (2002). Gaze-Contingent Real-Time Simulation of Arbitrary Visual Fields. In Human Vision and Electronic Imaging, San Jose, CA, SPIE.

Shell, J. S., Selker, T., and Vertegaal, R. (2003). Interacting with Groups of Computers, Communications of the ACM 46, 3 (March), 40-46.


  • Login