Techniques of perception with x3d
This presentation is the property of its rightful owner.
Sponsored Links
1 / 44

Techniques of Perception with X3D PowerPoint PPT Presentation


  • 56 Views
  • Uploaded on
  • Presentation posted in: General

Techniques of Perception with X3D. Nicholas F. Polys Luciano Pereira Soares. Session Outline. Polys Perception for Design Visualization Soares Immersion and Perception Polys Information-Rich Environments. Perception for Design.

Download Presentation

Techniques of Perception with X3D

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Techniques of perception with x3d

Techniques of Perception with X3D

Nicholas F. Polys

Luciano Pereira Soares

Web3D Symposium 2005


Session outline

Session Outline

  • Polys

    • Perception for Design

    • Visualization

  • Soares

    • Immersion and Perception

  • Polys

    • Information-Rich Environments

Web3D Symposium 2005


Perception for design

Perception for Design

  • Using our understanding of the human perceptual systems to guide design

    • Visual system

    • Auditory system

    • Vestibular system

  • Leverage pre-attentive facilities

  • Reduce cognitive overhead

Web3D Symposium 2005


Background information psychophysics

Background:Information Psychophysics

  • Edward Tufte, Envisioning Information

    (1983, 1990)

  • Jaques Bertin, Semiology of Graphics

    (1983)

  • Donald Norman, Cognitive Engineering

    (1986)

  • Joseph Goguen, Semiotic Morphisms

    (2000)

  • Colin Ware, Perception for Design

    (2003)

Web3D Symposium 2005


Pre attentive processing

Pre-attentive Processing

  • Involuntary, do not require conscious attention

  • Parallel

  • Efficient

  • Resistant to instruction

Web3D Symposium 2005


Visual fields

Visual Fields

Fovea

Useful Field of View

Field of View

Field of Regard

Web3D Symposium 2005


Frame rate

Frame Rate

  • Threshold for perceiving continuity:

    • flicker < 50 Hz

    • > 24 fps looks smooth & plenty interactive

  • Flicker & Attention can lead to change blindness (Simmons, 2000)

  • Browser.getCurrentFrameRate()

  • Implementing X3DPerFrameObserverScript

    - public void prepareEvents (){}

Web3D Symposium 2005


Graphics and vision

Graphics and Vision

Constructivism:

  • Bottom-up : internal representations are built out of simple external properties

  • Top-down : hypotheses, expectations

    Perceptual processes are influenced by both; top-down processes take over for ambiguous or degraded stimuli

Web3D Symposium 2005


Graphics and vision bottom up

Graphics and Vision: bottom-up

  • Objects – integration, recognition, working memory and reasoning

  • Patterns – contours, regions, groups; Gestalt

  • Features – color, texture, stereoscopic depth, shape & form, some aspects of motion

Web3D Symposium 2005


Features color

Features: Color

  • Luminance channel

    (3x spatial accuity)

  • Red / Green channel

  • Yellow / Blue channel

    The spectrum is not a

    perceptually linear sequence

    (not pre-attentive)!

    (Keller 1993; Ware, 2000)

Web3D Symposium 2005


Shapes appearances

Shapes & Appearances

  • Appearance {} and Materials {} : specular, emissive, and diffuse Colors in RGB, shininess, transparency, ambientIntensity

  • creaseAngle: shading across polygons edges of the mesh

  • normals (for shape-dependent lighting control)

  • colorPerVertex

Web3D Symposium 2005


Rgb material

RGB Material {}

diffuseColor

0.678, 0.169, 0.07

specularColor

shininess

Web3D Symposium 2005


Textures

Textures

  • ImageTexture {} with (or without) alpha channels can be applied and mapped to geometry as fixed or animated maps. 

    • Standard formats: .png, .jpg,

  • MovieTexture {}

  • TextureTransform {} …

  • PixelTexture {}

Web3D Symposium 2005


Multitexture

MultiTexture {}

Blending operations specified via

mode field

Base Texture

+ Lightmap                  

= Result

Web3D Symposium 2005


Lighting

Lighting

Lighting Nodes: on, intensity, ambientIntensity, color

  • Pointlight {attenuation}

  • DirectionalLight {}

  • Spotlight {direction, beamWidth, cutOffAngle}

  • AMD 1: SFBool global

Web3D Symposium 2005


Features depth

Features: Depth

  • Occlusion

  • Motion Parallax

  • Linear Perspective

    • Relative size

    • Texture & shade gradients

  • Stereoscopy

  • Oculormotor cues

  • Transform {translation rotation}

  • Head-Up-Display / Imageplane

Web3D Symposium 2005


Patterns grouping

Patterns & Grouping

  • Gestalt principles

  • Also: continuation, closure, common fate

  • Guiding Law of Pragnanz (simplest, most stable configuration)

Web3D Symposium 2005


Gestalt principles

Gestalt principles

  • Palmer & Rock, 1990– review & update principles; grouping based on perceived proximity in 3D space (not 2D proximity on retina)

  • Quinlan & Wilton, 1998 – study involving Gestalt conflict; proposed resolution mechanisms

Web3D Symposium 2005


Objects

Objects

  • Feature Binding – putting the streams together for internal represenation

    • color, form, motion

    • Just in time?

  • 2.5 D sketch (Marr, 1982)

  • Geons (Biederman, 1993)

Web3D Symposium 2005


Attention

Attention

  • Pop out effects ‘stand out’ in some simple dimension (conjunctions don’t):

    • Rapid visual search

    • Form, color, simple motion/blinking, spatial stereo depth, shading, position

12987621909023748594329087065483940560248595437289009890509874632234

Web3D Symposium 2005


Techniques of perception with x3d

Web3D Symposium 2005


Fundamental data types

Fundamental Data Types

  • Spatial / perceptual data:

    geometry, colors, textures, lighting

  • Abstract data / world & object attributes: nominal, ordinal, quantitative

  • Temporal data / behaviors:

    states, dynamics

Web3D Symposium 2005


Visual markers

Data Type

Quantitative

Ordinal

Nominal

Graphical Representation

position

length

angle / slope

area

volume

color / density

(Cleveland and McGill, 1980)

position

density

color

texture

connection

containment

length

angle

slope

area

volume

(Mackinlay, 1986)

position

color

texture

connection

containment

density

shape

length

angle

slope

area

volume

(Mackinlay, 1986)

Visual Markers

Web3D Symposium 2005


Techniques of perception with x3d

Web3D Symposium 2005


Auditory perception

Auditory Perception

  • Sound {}

  • AudioClip {}

  • MovieTexture {}

    • pitch

    • intensity

    • Spatialized Audio (doppler effect)

    • Standard formats: .wav, .midi, .mp3, mpeg-1

Web3D Symposium 2005


Existential perception

Existential Perception

What is my relation to this environment?

What can I do in this world?

What do my senses tell me?

  • Viewpoint {fieldOfView}

  • NavigationInfo {avatarSize, headlight, visibilityLimit, type, speed}

  • Timesensor {cycleInterval }

Web3D Symposium 2005


Environmental effects

Environmental effects

  • Background {}: colors and textures give a context for the environment

  • TextureBackground {transparency }

  • Fog {type color visibilityRange}

  • LocalFog {} &

    FogCoordinate {}

Web3D Symposium 2005


Immersion

Immersion

  • Luciano

Web3D Symposium 2005


Information rich virtual environments irves

Information-Rich Virtual Environments (IRVEs)

We need to understand:

  • How spatial/perceptual information and abstract information can be combined and displayed

  • What makes the combinations effective

  • What makes them usable and

  • How users think and act when using them

Web3D Symposium 2005


Multimedia comprehension

Multimedia & Comprehension

  • Co-references between text and images

    (Chandler & Sweller, 1990; Faraday & Sutcliffe, 1997, 1998)

  • Task Knowledge Structure

    (Sutcliffe & Faraday, 1994; Sutcliffe 2003)

  • Meaningful Learning: troubleshooting, redesigning, deriving principles

    (Mayer, 2002)

Web3D Symposium 2005


Visualization annotation

Visualization & Annotation

  • Challenge to maintain perceptual fidelity

    • Scientific Visualization overloads color, texture to show abstract information

    • IRVEs attempt to maintain perceptual/spatial fidelity

  • Grouping, registering temporal and abstract information to referents

    • Visual

    • Interactive

Web3D Symposium 2005


Layout space locations

Layout Space (Locations)

The layout space of abstract information in IRVEs is described by the coordinate system it is resident in:

  • Object

  • World

  • User

  • Viewport

  • Display

Web3D Symposium 2005


Object space

Object Space

Object space is relative to an object’s location in the environment

(e.g. Semantic Objects).

Web3D Symposium 2005


World space

World Space

World space is relative to an area, region, or location in the environment.

Web3D Symposium 2005


User space

User Space

User space is relative to the user’s location but not their viewing angle.

Web3D Symposium 2005


Viewport space

Viewport Space

Viewport space-is the image plane where HUDs or overlays may be located.

Web3D Symposium 2005


Display space

Display Space

Display layout space where abstract visualizations are located outside the rendered view in some additional screen area.

Web3D Symposium 2005


Working memory

Working Memory

How are the products of perception linked with memory and reasoning?

A number of models exist:

  • Baddeley’s componentized WM

  • Barnards’s Interacting Cognitive Subsystems

Web3D Symposium 2005


Working memory baddeley 2003

Central Executive

Visuospatial

sketchpad

Episodic buffer

Phonological

Loop

Fluid system

Visual semantics Episodic LTM Language

Crystallized system

Working Memory (Baddeley, 2003)

Web3D Symposium 2005


Visuospatial wm

Visuospatial WM

  • Capacity 3-5 ‘items’

  • Functional units & chunking

    • Objects & features (Vogel et al, 2001)

    • Visual indices & dynamic feature binding (Saiki, 2003)

  • Subsystems: form & color, space & movement (Logie 1995)

  • Relation to Central Executive

    (Miyake et al. 2001)

Web3D Symposium 2005


Model ics interacting cognitive subsystems

Model ICS: Interacting Cognitive Subsystems

Visual

Auditory

Object

Implicational

Propositional

Articulatory

Motor

Web3D Symposium 2005


Irves perception

IRVEs & Perception

  • Integrating information types via

    • Depth cues

    • Gestalt assocation cues

  • Recent studies examined search and comparison tasks:

    • the Viewport vs. Object space interfaces

    • The Display vs. Object space interfaces

  • Future work will investigate supporting pattern and trend recognition tasks

Web3D Symposium 2005


References

References

Bertin, J., Semiology of Graphics. 1983, Madison, WI: University of Wisconsin Press.

Baddeley, A., Working memory: Looking back and looking forward. Nature Reviews Neuroscience, 2003. 4: p. 829-839.

Barnard, P., May, J., Duke, D., and Duce, D., Systems, interactions, and macrotheory. ACM Trans. on Computer-Human Interaction, 2000. 7(2): p. 222-262.

Biederman, I., and Gerhardstein, P.C., Recognizing depth-rotated objects: Evidence for 3-D viewpoint invariance. Journal of Experimental Psychology: Human Perception and Performance, 1993(19): p. 1162-1182.

Card, S., J. Mackinlay, and B. Shneiderman, Information Visualization: Using Vision to Think. 1999, San Francisco: Morgan Kaufmann.

Chandler, P., and Sweller, J., Cognitive Load Theory and the Format of Instruction. Cognition and Instruction, 1991. 8: p. 293-332.

Cleveland, W.S., and McGill, R., Graphical perception: Theory, experimentation and application to the development of graphical methods. J. Am. Stat. Assoc, 1984. 79(September).

Faraday, P., and Sutcliffe, Alistair. Designing Effective Multimedia Presentations. in CHI. 1997. Atlanta, GA: ACM.

Goguen, J., Information Visualizations and Semiotic Morphisms. 2000, UCSD: http://citeseer.ist.psu.edu/goguen00information.html.

Keller, P.R., Visual Cues: Practical Data Visualization. 1993, Piscataway, NJ: IEEE Computer Society Press.

Logie, R.H., Visuo-spatial working memory. 1995, Hove, UK: Psychology Press.

Mackinlay, J., Automating the Design of Graphical Presentations of Relational Information. ACM Trans. on Graphics, 1986. 5(2): p. 111-141.

Marr, D., Vision : a computational investigation into the human representation and processing of visual information. 1982, San Francisco: W.H. Freeman.

Mayer, R.E., Cognitive Theory and the Design of Multimedia Instruction: An Example of the Two-Way Street Between Cognition and Instruction. New Directions for Teaching and Learning, 2002. 89: p. 55-71.

Norman, D.A., Cognitive Engineering, in User Centered System Design, D.A. Norman, and Draper, S.D., Editor. 1986, Lawrence Erlbaum Associates: Hillsdale, NJ. p. 31-61.

Web3D Symposium 2005


Techniques of perception with x3d

References

Pickett, R.M., Grinstein, G., Levkowitz, H., Smith, S., Harnessing Preattentive Perceptual Processes in Visualization, in Perceptual Issues in Visualization, G. Grinstein, and Levkoitz, H., Editor. 1995, Springer: New York.

Quinlan, P.T., and Wilton, R.N., Grouping by proximity or similarity? Competition between the Gestalt principles in vision. Perception, 1998. 27: p. 417-430.

Rock, I., and Palmer, S., The legacy of Gestalt psychology. Scientific American, 1990(December): p. 48-61.

Simmons, D.J., Attentional capture and inattentional blindness. Trends in Cognitive Sciences, 2000. 4: p. 147-155.

Sutcliffe, A., and Faraday, P. Designing Presentation in Multimedia Interfaces. in CHI. 1994: ACM Press.

Tufte, E., The Visual Display of Quantitative Information. 1983, Cheshire, CT: Graphics Press.

Tufte, E., Envisioning Information. 1990, Cheshire, CT: Graphics Press.

Ware, C., Information Visualization: Perception for Design. 2000, New York, NY: Morgan Kauffman.

http://webvision.med.utah.edu/VisualCortex.html

Thanks to the ‘Rev.’ Bob Cripsen on his early instruction on VRML lighting!

Web3D Symposium 2005


  • Login