slide1 n.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
Tae Soo Yun Dept. of Digital Contents Dongseo University Fall 2002 based on notes from Soon Ki Jung, KNU Wohn, KAIST …… PowerPoint Presentation
Download Presentation
Tae Soo Yun Dept. of Digital Contents Dongseo University Fall 2002 based on notes from Soon Ki Jung, KNU Wohn, KAIST ……

Loading in 2 Seconds...

play fullscreen
1 / 95

Tae Soo Yun Dept. of Digital Contents Dongseo University Fall 2002 based on notes from Soon Ki Jung, KNU Wohn, KAIST …… - PowerPoint PPT Presentation


  • 103 Views
  • Uploaded on

Advanced Topics in Virtual Reality. Tae Soo Yun Dept. of Digital Contents Dongseo University Fall 2002 based on notes from Soon Ki Jung, KNU Wohn, KAIST ……. Table of Contents. Introduction : What is VR ? Psychological and Cognitive Issues VR System Anatomy Virtual Perception

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

Tae Soo Yun Dept. of Digital Contents Dongseo University Fall 2002 based on notes from Soon Ki Jung, KNU Wohn, KAIST ……


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
slide1

Advanced Topics in Virtual Reality

Tae Soo Yun

Dept. of Digital Contents

Dongseo University

Fall 2002

based on notes from

Soon Ki Jung, KNU

Wohn, KAIST

……

table of contents
Table of Contents
  • Introduction : What is VR ?
  • Psychological and Cognitive Issues
  • VR System Anatomy
  • Virtual Perception
  • Interaction
  • Virtual Worlds: Representation, Creation and Simulation
  • Virtual Worlds: Rendering
  • Networked VR Systems and Shared Virtual Environements
  • Image-based Virtual Reality
  • Augmented Reality
conceptual model of vr
Conceptual Model of VR

Virtual environment

Human

H-sensor

perception

cognition

motion control

H-effector

V-sensor

P-effectorv

L-effector

virtual

object

sensing

avatar

action

virtual

agent

V-effector

L-sensor

P-sensor

Logical devices

for

displacements,

angles,

events.

chapter 3 vr system anatomy
Chapter 3. VR System Anatomy

3-1. System Functional Overview

3-2. System-level Metaphors

3-3. Physical Sensors

3-4. Physical Effector: Visual Display

3-5. Physical Effector: Auditory Display

3-6. Physical Effectors: Haptic and Motion Display

3-7. Physical Effectors: Olfactory Display

3-8. Computing System

3-9. Case Study: Existing VR Systems

3 1 system functional overview
3-1. System Functional Overview

Topics to be discussed

  • Typical setup for a VR system
  • Conceptual model
  • Functional model -- a schematic diagram
  • Functional modules
conceptual model of vr1
Conceptual Model of VR

Virtual environment

Human

H-sensor

perception

cognition

motion control

H-effector

V-sensor

P-effectorv

L-effector

virtual

object

sensing

avatar

action

virtual

agent

V-effector

L-sensor

P-sensor

Logical devices

for

displacements,

angles,

events.

functional diagram
Functional diagram

rendering

(Chap. 7)

displaying

(Sec. 3-4,5,6,7)

simulation

(Sec. 6-3,4)

VW

DB

(Sec. 6-1)

Virtual

perception

(Chap. 4)

interaction

(Chap. 5)

Sensing

(Sec. 3-3)

VW

Authoring

(Sec. 6-2)

(Sec. 3-8)

modules
modules
  • sensing
  • virtual perception
  • interaction
  • simulation
  • rendering
  • displaying
  • virtual world DB
sensing and virtual perception
sensing and virtual perception
  • sensing
    • handles the physical sensors (and its controller) of the conceptual model
  • virtual perception
    • handles the transition from the physical sensors to the logical sensors.
slide11

Virtual environment

Human

H-sensor

perception

cognition

motion control

H-effector

V-sensor

P-effectorv

L-effector

virtual

object

sensing

avatar

action

virtual

agent

V-effector

L-sensor

P-sensor

Logical devices

for

displacements,

angles,

events.

slide12

Glove

Controller

Hand gesture

recognition

Video camera

Image processor

Body gesture

recognition

Video camera

Image processor

Facial/face expression

recognition

Microphone

Signal processor

Speech recognition

Sensing

Virtual perception

interaction
Interaction

rendering

(Chap. 7)

displaying

(Sec. 3-4,5,6,7)

simulation

(Sec. 6-3,4)

VW

DB

(Sec. 6-1)

Virtual

perception

(Chap. 4)

interaction

(Chap. 5)

Sensing

(Sec. 3-3)

VW

Authoring

(Sec. 6-2)

(Sec. 3-8)

slide14

Virtual environment

Human

H-sensor

perception

cognition

motion control

H-effector

V-sensor

P-effectorv

L-effector

virtual

object

sensing

avatar

action

virtual

agent

V-effector

L-sensor

P-sensor

Logical devices

for

displacements,

angles,

events.

slide15
Avatar의 정의, 모델링.
    • geometrical model
    • interaction model
      • interaction가능한 body parts?
      • avatar는 가상환경과 어떤 interaction을 할 수 있는가?
  • Logical sensor의 정의.
  • Logical sensor로부터 avatar(virtual effector)로의 mapping.
  • Avatar(virtual sensor)로부터 logical effector로의 mapping.
  • Run-time시 실제 일어나는 interaction결정.
simulation
Simulation

rendering

(Chap. 7)

displaying

(Sec. 3-4,5,6,7)

simulation

(Sec. 6-3,4)

VW

DB

(Sec. 6-1)

Virtual

perception

(Chap. 4)

interaction

(Chap. 5)

Sensing

(Sec. 3-3)

VW

Authoring

(Sec. 6-2)

(Sec. 3-8)

slide17

Virtual environment

Human

H-sensor

perception

cognition

motion control

H-effector

V-sensor

P-effectorv

L-effector

virtual

object

sensing

avatar

action

virtual

agent

V-effector

L-sensor

P-sensor

Logical devices

for

displacements,

angles,

events.

slide18
Interaction에 따라 virtual world를 변화시킴
  • Virtual world를 운영
  • Kinematic simulation
  • dynamic simulation
  • behavioral simulation
  • behavioral simulation of autonomous agents
    • cognitive, emotional, physical
rendering
Rendering

rendering

(Chap. 7)

displaying

(Sec. 3-4,5,6,7)

simulation

(Sec. 6-3,4)

VW

DB

(Sec. 6-1)

Virtual

perception

(Chap. 4)

interaction

(Chap. 5)

Sensing

(Sec. 3-3)

VW

Authoring

(Sec. 6-2)

(Sec. 3-8)

slide20

Virtual environment

Human

H-sensor

perception

cognition

motion control

H-effector

V-sensor

P-effectorv

L-effector

virtual

object

sensing

avatar

action

virtual

agent

V-effector

L-sensor

P-sensor

Logical devices

for

displacements,

angles,

events.

slide21
Simulation된 결과를 rendering.
  • Perceptualization of virtual worlds
  • image rendering
  • sound rendering
  • haptic rendering
  • others
displaying
Displaying

rendering

(Chap. 7)

displaying

(Sec. 3-4,5,6,7)

simulation

(Sec. 6-3,4)

VW

DB

(Sec. 6-1)

Virtual

perception

(Chap. 4)

interaction

(Chap. 5)

Sensing

(Sec. 3-3)

VW

Authoring

(Sec. 6-2)

(Sec. 3-8)

slide23

Virtual environment

Human

H-sensor

perception

cognition

motion control

H-effector

V-sensor

P-effectorv

L-effector

virtual

object

sensing

avatar

action

virtual

agent

V-effector

L-sensor

P-sensor

Logical devices

for

displacements,

angles,

events.

slide24

Stereo image

display

Stereo image

generation

Headphone

3-D sound

generation

Haptic display

Haptic signal

generation

virtual world db
Virtual World DB

rendering

(Chap. 7)

displaying

(Sec. 3-4,5,6,7)

simulation

(Sec. 6-3,4)

VW

DB

(Sec. 6-1)

Virtual

perception

(Chap. 4)

interaction

(Chap. 5)

Sensing

(Sec. 3-3)

VW

Authoring

(Sec. 6-2)

(Sec. 3-8)

slide26
Representation of virtual worlds
  • “scene graph”
  • Big issue
    • expressiveness vs. efficiency
3 2 system level metaphors
3-2. System-level Metaphors
  • How is a virtual world presented to the user?

1. On-the-table (desktop, fish tank, work bench,…)

2. Through-the-window (inside-the-vehicle, …)

3. Immersive

    • encumbered (body suit, HMD,…)
    • CAVE, Dome
    • Neural-connection

4. Third person (mirror, reflexive)

slide31
CAVE

“A Review of Tele-Immersive Applications in the CAVE Research Network”, J. Leigh, et.al., Proc. IEEE Virtual Reality ’99, pp. 180-187, March 1999.

third person reflexive mirror
Third person (reflexive, mirror)

Camera

Avatar

VR system

Large Display

Participant

slide35

Chroma-keying

3rd Person VR

(2D)

developments

3rd Person VR

(3D)

Virtual Studio

slide36
Advantages
    • full-body interaction
    • unencumbered interface
    • easily shared virtual experience
  • Disadvantages
    • slow/coarse/poor interactions
    • (early systems) 2D VE, interactions
    • must watch the VE face to face.
  • Applications
    • entertainment, education
    • broadcasting, film making
3 3 physical sensors
3-3. Physical Sensors

Virtual environment

Human

H-sensor

perception

cognition

motion control

H-effector

V-sensor

P-effectorv

L-effector

virtual

object

sensing

avatar

action

virtual

agent

V-effector

L-sensor

P-sensor

Logical devices

for

displacements,

angles,

events.

slide38
1. Conventional 2D Interfaces

2. Position trackers

3. Gloves

4. Others

position trackers
Body parts to be tracked

hand, finger

eye

head

whole body

Performance specifications

static accuracy

dynamic accuracy

latency

update rate

signal to noise ratio

registration

Position trackers
tracking methods
Tracking methods
  • mechanical
  • magnetic
  • optical
    • stereo, multiple cameras
    • time-of-flight, phase
    • structured light
  • acoustic
  • inertial
  • GPS
  • Research issues
    • tracking and prediction
    • sensor fusion
gloves
Gloves
  • Finger flexions :
    • Optical fiber sensors [VPL dataglove]
    • Hall Effect Sensor [Exos Dexterous Hand Master]
    • Strain gauges based on flexible plastic

with a constant resistive surface [Mattel’s Power Glove]

    • Resistive bend-sensing technology [Virtex Cyberglove]
  • Orientation / position
    • Electromagnetic tracking system [Polhemus Isotrak]
    • Ultrasonic transducer / receiver [Mattel’s Power Glove]
    • Electromechanical, acoustic, optical tech..

[VPL dataglove]

[Power Glove]

others
Others
  • 3D mouse
  • wand
  • treadmill
  • bicycle, wheel chair, shopping cart
  • Neural interface
    • EMG (electro-myograph) -- muscle
    • EEG (electro-encephalograph) -- brain
    • EOG (electro-oculograph) -- eye
3 4 5 6 7 physical effectors
3-4,5,6,7. Physical Effectors

Virtual environment

Human

H-sensor

perception

cognition

motion control

H-effector

V-sensor

P-effectorv

L-effector

virtual

object

sensing

avatar

action

virtual

agent

V-effector

L-sensor

P-sensor

Logical devices

for

displacements,

angles,

events.

slide44
Physical effectors
    • visual display
    • auditory display
    • haptic display
    • motion display
    • olfactory display
3 4 physical effector visual display
3-4. Physical Effector: Visual Display
  • “ideal” display to match 20/20 vision
    • resolution: 5,000 x 5,000
    • fov: 90 degrees
    • update rate: 40 ~ 50 fps
slide46
Objectives
  • To develop the hardware (visual devices)
  • To develop the algorithm for the hardware

in such a way that the visual display match the characteristics of the human visual system.

Research issues

  • methods for perfecting display devices (e.g., calibration,...)
  • psychological issues (e.g., sensory adaptation, depth perception, …)
  • human perception-based displaying techniques
slide47
Topics to be covered

1. Image generation: principle

  • device-independent issues

2. Image generation: technology

  • various types of displays

3. Head mounted display

3 5 physical effector auditory display
3-5. Physical Effector: Auditory Display
  • Research issues
    • Principle of 3-D sound generation
    • 3-D sound generation techniques
    • Psychological effects of altered auditory environments
3 6 physical effector haptic and motion displays
3-6. Physical Effector:Haptic and Motion Displays
  • Force, pressure, tactile feedbacks
  • Haptic interface requires sensing capability.
  • Research issues
    • Haptic science (studies on human haptics)
    • Tool-hand system
    • Creating the haptic illusion
    • Interaction interface of haptic and vision
    • Texture, temperature devices
2 force feedback display
2. force feedback display
  • worn-type (e.g., exoskeleton)
  • held-type (e.g., universal hand controller)
  • encountered type
4 motion display
4. motion display
  • Motion
    • whole body motion
      • passive motion -- e.g., motion platform
      • active motion -- e.g., locomotion
    • part body motion
      • passive motion
      • active motion
slide58
Motion cues
    • vestibular system
    • motor
    • visual
    • auditory
    • tactile
    • proprioceptive / kinesthetic -- muscle
  • Motion displays
    • inertial system: moves the body (e.g., treadmill, platform)
    • non-inertial system: simulates motion
3 7 physical effectors olfactory displays
3-7. Physical Effectors:Olfactory Displays

http://www.wired.com/wired/archive/7.11/digiscent_pr.html

3 8 computing system
3-8. Computing System

General requirements

  • frame rate
  • latency
  • uniform performance

1. Hardware challenges

2. Software challenges

3. Computational challenges: A new time model

4. Computational challenges: Time-critical computing

5. Software architecture

1 hardware challenges
1. Hardware challenges
  • AI, scientific computing, … and now VR.

(1) graphics architecture

    • parallelization of the graphics pipeline

(2) computation

    • simulation of very large physical systems

(3) data management

    • large data sets generated from the virtual world
1 graphics pipeline
(1) Graphics Pipeline

Application-specific

processing

Scene

processing

Polygon

processing

Pixel

processing

Scene graph

Display list

Frame

buffer

Depth

buffer

Texture

buffer

- per primitive (polygon processing)

- rasterization (included in pixel processing)

- per fragment (pixel processing)

parallelism
parallelism

per primitive

rasterization

per pixel

primitive

crossbar

(SGI RE,

PixelPlane)

pixel image

crossbar

(PixelFlow)

fragment

crossbar

(E&S)

2 computation
(2) computation
  • simulation - computation-bound
  • visualization - data-bound
  • directions
    • vectorized architecture
    • massively parallel processors
    • distributed computing
3 data management
(3) data management
  • large data sets
  • sources of data
    • virtual perception
    • data to be rendered
  • bus with an ultra-high bandwidth
2 software challenges
2. Software challenges
  • Reference
    • “VR as a Forcing Function: Software Implications of a New Paradigm”, A. van Dam, Proc. IEEE Symposium on Research Frontiers in Virtual Reality, pp. 5-8, October 1993.
    • “Advanced Virtual Reality Applications”, SIGGRAPH-94 Tutorial Note 2.
  • Is VR different from the desktop CG ?
    • Yes, remember Model P (perception, interaction, world model)
  • Does VR require different technology ?
    • It can be done with the existing technology.
    • But, hard to implement,

hard to extend,

not very good quality.

slide70
Software is inadequate.
    • hard to implement  low-level tools
    • hard to extend  bigger worlds,

increased realism,

more participants,

maintaining realtime.

points to consider
Points to consider

(1) interaction

(2) operation of the virtual world

(3) object systems

(4) application frameworks

1 interaction
(1) interaction
  • Task-level interaction is not well-defined.

(as opposed to the driver-level)

  • Extension of 2D interaction? (e.g., 3D widget)
  • complicated matters
    • continuous interaction
    • no distinction between interface objects and app objects
    • active objects
    • multi-modal interactions
    • multi-participants
    • little formalism: operational semantics
2 operations in the ve
(2) operations in the VE
  • “main loop” tasks
    • sensing, virtual perceiving, interaction, simulation, perceptualization, rendering
  • Problems
    • latency
    • uniform performance
  • Methods
    • simulation loop
    • separate process for each task
    • fine-grain objects, rather than heavy-weight processes
  • Object model
    • light-weight threads + heavy-weight proc. + distributed proc.
3 object systems
(3) Object systems
  • OOP
  • Display list
  • Sharing
  • Object collaboration
4 object frameworks
(4) object frameworks
  • Graphics library vs. Framework
  • library
    • procedural API
  • framework
    • protocol to integrate and connect multiple software modules.
    • Requirements
      • multiple, continuous inputs
      • multiple outputs
      • time-varying behavior and simulation
      • time-critical computing
3 computational challenges a new time model
3. Computational challenges:A new time model
  • Has been the subject of intensive studies in distributed computing and parallel computing.
  • For the time-varying behavior and simulation
  • time in the real world, time in the virtual world,

time in another world.

  • Synchronization among processes
    • interaction, simulation, rendering
4 computational challenges time critical computing
4. Computational challenges:Time-critical computing

Philosophy

  • Fast, slightly inaccurate vs. Slow, very accurate
    • What inaccuracies are acceptable?
  • Time budgets are assigned to tasks.
  • Algorithms choose the best compromises in order to meet the time budget.
real time computing vs tcc
Real-time computing vs. TCC
  • Real-time computing
    • provides guaranteed response
      • in a fixed time,
      • In a fixed computational environment.
  • Time-critical computing
    • chooses the appropriate level of degradation
      • in a user-defined time,
      • in a flexible, unpredicted environment.
basic approach
Basic approach
  • User specifies an overall desired frame time.
  • Each task within that frame is assigned a time budget.
    • Issue -- allocating the time budget to each task.
  • Each task determines how to do the best job within that time budget.
    • Issue - meeting the time budget.
    • Approach:
      • how much to be processed?
      • How accurately to be processed?
1 time critical rendering
(1) Time-critical rendering

Determine the following:

  • How much to be processed?
    • # of polygons
  • How accurately to be processed?
    • Rendering techniques

while maximizing the ratio = benefit / cost.

benefits and costs
Benefits and costs
  • Benefits
    • image quality (or something else…)
    • depends on
      • the size of object on screen
      • the location on screen
      • the focus of interaction
  • Cost
    • number of polys * cost per poly
  • Maximize the benefit / cost ratio
2 time critical simulation
(2) Time-critical simulation

Determine the following:

  • How much to be processed?
    • in terms of the objects under computation
  • How accurately to be processed?
    • in terms of the algorithm complexity (e.g., # of iterations)

while maximizing the ratio = benefit / cost.

benefits and costs1
Benefits and costs
  • Benefits
    • accuracy of computation
    • number of objects being computed
  • Costs
    • time complexity
  • Maximize the benefit / cost ratio
    • Benefits and costs are hard to define.
    • No universally accepted approach yet.
5 software architecture
5. Software Architecture

Conceptual model --> functional model --> VR system architecture

  • Horizontal view
  • Vertical view
functional diagram1
Functional diagram

rendering

(Chap. 7)

displaying

(Sec. 3-4,5,6,7)

simulation

(Sec. 6-3,4)

VW

DB

(Sec. 6-1)

Virtual

perception

(Chap. 4)

interaction

(Chap. 5)

Sensing

(Sec. 3-3)

VW

Authoring

(Sec. 6-2)

(Sec. 3-8)

slide86
nVR

rendering

(Chap. 7)

displaying

(Sec. 3-4,5,6,7)

simulation

(Sec. 6-3,4)

VW

DB

(Sec. 6-1)

Virtual

perception

(Chap. 4)

interaction

(Chap. 5)

Sensing

(Sec. 3-3)

VW

Authoring

(Sec. 6-2)

slide87
nVR

rendering

(Chap. 7)

displaying

(Sec. 3-4,5,6,7)

simulation

(Sec. 6-3,4)

VW

DB

(Sec. 6-1)

Virtual

perception

(Chap. 4)

interaction

(Chap. 5)

Sensing

(Sec. 3-3)

VW

Authoring

(Sec. 6-2)

slide88
Initialization
  • Virtual perception
    • Read and interpret input devices
    • Read and interpret the network input
  • Interaction
    • Compute state changes from inputs
    • Compute state changes from net reads
  • Simulation
    • Computational modeling
    • Post state changes to network
  • Render
slide89
Single-thread system
    • process sequentially
    • low frame rate
    • display jump/freeze
  • Multi-thread system
    • A single bottleneck does not starve the entire cycle.
    • Exploit a multi-processor machine.
multi thread system
multi-thread system

cache

Interaction

module

Virtual World

DB

Coordination

module

Simulation

module

cache

Rendering

module

cache

slide92
Each module / sub-system may have multiple threads.
  • simulation module
    • dead reckoning thread
    • collision detection thread
    • other computation threads
  • rendering module
    • multiple pipeline threads

(ex. SGI Performer – 4 threads (app, cull, draw, timer)

vertical view
Vertical view

AuthoringTool

Application

Application

Virtual World Operation

Scene

Description

Effectors

&

Sensors

Rendering

Task &

Simulation

Network

SUN

PIXRECT

Windows

DirectX

OpenGL

SGI GL

X-Window

Virtual

World DB

design considerations
Design considerations
  • Time-critical computing/rendering 개념
  • High-level interaction 정의 용이
  • Scene layout/edit 용이
  • 다양한 어플리케이션에 사용 가능
  • Platform independence
  • Network 기반 다중사용자 시스템으로의 확장성
  • Basic research 에 사용 가능한 도구