underlying technologies part two software
Download
Skip this Video
Download Presentation
Underlying Technologies Part Two: Software

Loading in 2 Seconds...

play fullscreen
1 / 65

Underlying Technologies Part Two: Software - PowerPoint PPT Presentation


  • 110 Views
  • Uploaded on

Underlying Technologies Part Two: Software. Mark Green School of Creative Media. Introduction. Software not as easy as hardware: wide range of software techniques, hard to classify like hardware several components that need to work together, hard to know where to start

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about ' Underlying Technologies Part Two: Software' - carter-carpenter


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
underlying technologies part two software

Underlying TechnologiesPart Two: Software

Mark Green

School of Creative Media

introduction
Introduction
  • Software not as easy as hardware:
    • wide range of software techniques, hard to classify like hardware
    • several components that need to work together, hard to know where to start
    • wide range of hardware configurations, not as simple as 2D software
hardware configurations
Hardware Configurations
  • In 2D have a standard hardware configuration:
    • input: keyboard and mouse
    • output: single 2D display
  • with 3D can have many configurations:
    • HMD
    • projection
    • single screen
hardware configurations1
Hardware Configurations
  • Want to produce an application once, not once for every possible hardware configuration
  • software needs to be more adaptable, change based on hardware configuration
  • complicates the development of support software
range of software techniques
Range of Software Techniques
  • Want our software to be very efficient: reduce latency, high update rates
  • some applications can be quite large, need to efficiently organize data
  • all of this complicates VR software, too many things to consider, hard to know where to start
components
Components
  • What are the main components of a VR application?
    • 3D Objects: geometry and appearance, but may also want sound and force
    • Behavior: the objects need to be able to do things, move and react
    • Interaction: users want to interact with the application, manipulate the objects
3d objects
3D Objects
  • Need object geometry, object’s shape, basis for everything else, called model
  • polygons used for geometry, sometimes restricted to triangles
  • different from animation, free form surfaces based on sophisticated math
  • need speed, so restricted to polygons
3d objects1
3D Objects
  • Where does geometry come from?
  • Really depends on the application
  • Could use a text editor to enter all the polygon vertices, some people actually do this!
  • Could use a program, for example OpenGL, works for small models
3d objects2
3D Objects
  • Use a 3D modeling or animation program
  • for non-programmers this is the easiest way, but it takes time to develop modeling skills
  • also many different program and file formats
  • want a modeler that does a good job of polygons, not all modelers are good at this
3d objects3
3D Objects
  • Another source of objects is scientific and engineering computations
  • can be easy to convert to polygons, already have position data
  • other types of data can also be converted into geometry, but this can be more difficult
3d objects4
3D Objects
  • Also need to consider appearance:
    • colour of the object
    • how it reflects light
    • transparency
    • texture
  • can be done with modeler, or later in the VR program
behavior
Behavior
  • How should objects behave?
    • What happens when the user hits an object?
    • What happens when an object hits another object?
    • Can objects move around the environment?
  • Each object could have a range of behaviors, react differently to different events in the environment
behavior1
Behavior
  • Behavior is harder than modeling
  • animation programs can be useful, but not always
  • animation is quite different:
    • animator is in complete control, knows what’s happening all of the time
    • in VR the user is in control, can interrupt or mess up any animation
behavior2
Behavior
  • Short animations (less than 5 seconds) can be useful, basic motion units
  • other types of behaviors must be programmed or scripted
  • more flexible, can respond to the events that occur in the environment
  • easier to combine, objects can do two things at same time
interaction
Interaction
  • Users want to interact with the environment
  • pick up objects and move them around
  • very different from 2D interaction
  • much more freedom, more direct interaction
  • still exploring the design space, not stable like 2D interaction
  • still working on standard techniques
application structure
Application Structure
  • look at application structure
  • provides a framework for discussing various software technologies
  • divide an application into various components, and then look at the components individually
application structure1
Application Structure

Application

Processing

Input

Devices

Model

Input

Processing

Model Traversal

Output Devices

application structure2
Application Structure
  • Model: representation of objects in the environment, geometry and behavior
  • Traversal: convert the model into graphical, sound, force, etc output
  • Input Processing: determine user’s intentions, act on other part of application
  • application processing: non-VR parts of the application
interaction loop
Interaction Loop
  • Logically the program consists of a loop that samples the user, performs computations and traverses the model

Input processing

Computation

Model Traversal

model
Model
  • Contains the information required to display the environment:
    • geometry, sound, force
    • behavior
  • the graphical part is the most developed, so concentrate on it
  • try to position sound and force within this model
geometry
Geometry
  • This is what we know the best
  • need to have a graphical representation of objects in the environment:
    • accurate shape representation
    • ease of modeling
    • efficient display
    • integrates with behavior
scene graph
Scene Graph
  • Main technique for structuring the model
  • based on hierarchical structure, divide the object into parts or components
  • simplifies the modeling task, work on one part at a time
  • easy to modify the individual parts
  • add behaviors, sound, force, etc to the model
scene graph1
Scene Graph

car

Wheel

Wheel

Wheel

Wheel

Body

scene graph2
Scene Graph
  • Individual units are called nodes:
    • shapes: polygons, meshes, cubes, etc
    • transformations: position the nodes in space
    • material: colour and texture of objects
    • grouping: collecting nodes together as a single object
    • sounds
    • behavior
scene graph3
Scene Graph
  • Many different scene graph architectures, will look at one in more detail later
  • differences:
    • scene graph for whole VE Vs. one per object
    • types of nodes in the scene graph
    • ease of modification, static Vs dynamic
behavior3
Behavior
  • Harder to deal with than geometry
  • simple motions aren’t too bad, but much harder to get sophisticated behavior
  • the general solution now is to write code, okay for programmers
  • would like to have a higher level approach for non-programmers
behavior4
Behavior
  • Problem: want objects to respond to events in the environment
  • can have some motions that are simple animations, but most of the motions need some knowledge of the environment
  • example: an object moving through the environment must be aware of other objects so it doesn’t walk through them
behavior5
Behavior
  • Some simple motions produced by animating transformation nodes
  • animation variables used to control transformation parameters, example: rotation or translation
  • could import animations, use some form of keyframing package to produce the motion
behavior6
Behavior
  • Simple motions could be triggered by events in the environment
  • example: collision detection, if an object is moving through the environment and a collision detected it changes direction
  • hard to come up with good trigger conditions, a few obvious ones, but not much after that
behavior7
Behavior
  • Another approach is to use a general motion model
  • best example of this is physics, try to simulate real physics in the environment
  • this gives a number of natural motions, and objects respond to the environment
  • works well in some environment, but has some problems
behavior8
Behavior
  • One problem is the complexity of the mathematics, often need to simplify
  • computations can be a problem, particularly for complex objects
  • hard to control, need to know forces and torque\'s that produce the desired motions, can be very hard to determine
behavior9
Behavior
  • Some attempts to produce general motion controllers
  • maybe the eventual solution, but nothing much now
  • use of scripting languages, can add some program control to the scene graph, but not full programming
model traversal
Model Traversal
  • The process of going through the model and generating the information to be displayed
  • this is part software and part hardware, look through the entire process
  • hardware parts have implications for how we build models and the graphics techniques used
a simple model
A Simple Model
  • A simplified model of the display process, explains hardware performance

Model

Screen

traverse

geometry

Pixel

traverse
Traverse
  • Traverse the model, determine objects to be drawn, send to graphics hardware
  • usually combination software/hardware, depends on CPU and bus speed
  • early systems were hardware, didn’t scale well
  • many software techniques for culling models
geometry1
Geometry
  • Geometrical computations on polygons: transformations and lighting
  • floating point intensive
  • divide polygons into fragments, screen aligned trapezoid
  • time proportional to number of polygons and vertices
pixel
Pixel
  • Fill fragments, colour interpolation, texture mapping, transparency, hidden surface
  • all the per pixel computations, time depends on number of pixels, also colour depth on low end displays
  • scalable operations, can add more processors for more speed
design considerations
Design Considerations
  • Any of the stages could block, depend on display mix
  • lots of small polygons cause traversal and geometry stages to block
  • large polygons cause pixel stage to block
  • can use buffers to reduce blocking
  • a careful balancing process
design considerations1
Design Considerations
  • CPU/Image Generator trade-off
  • cheap boards just do pixel stage, use CPU for everything else:
    • scales with CPU speed
    • large polygons and texture mapping
  • moving geometry onto board increases performance, trend in low cost displays
pc hardware evolution
PC Hardware Evolution
  • Start with CPU doing most of the work
  • Graphics board:
    • image memory
    • fill and hidden surface
    • texture mapping
  • graphics speed determined by CPU, limited assistance from graphics card
graphics card memory
Graphics Card Memory
  • Memory used for three things:
    • image store
    • hidden surface (z buffer)
    • texture maps
  • texture can be stored in main memory with AGP, but this isn’t most efficient
  • better to have texture memory on board
image memory
Image Memory
  • Amount depends on image size
  • double buffer, two copies of image memory
    • front buffer: image displayed on screen
    • back buffer: where the next image is constructed
  • can construct next image while the current image is displayed, better image quality and faster display
z buffer
Z Buffer
  • Used for hidden surface removal
  • z buffer: one value for each pixel, distance from eye to object drawn at that pixel
  • when drawing a pixel, compare depth of pixel to z buffer
  • if closer draw pixel and update z buffer
  • otherwise, ignore the pixel
graphics acceleration
Graphics Acceleration
  • Next step: move pixel operations to graphics card
  • fill and z buffer 3D triangles
  • add smooth shading and texture mapping
  • CPU does traversal and geometry processing
graphics acceleration1
Graphics Acceleration
  • Next step: move geometry processing to graphics card
  • CPU traverses model, send graphics primitives to display card
  • all transformations and lighting done on graphics card
  • less dependence on CPU
current trends
Current Trends
  • Pixel processing (Geforce 2): a program that processes each pixel, control lighting and other effects
  • support for multiple textures, etc
  • Vertex processing (Geforce 3): a program processes each vertex, can change geometry at display time
  • real-time deformations and IKA
current trends1
Current Trends
  • Move to programming all aspects of the graphics card (3DLabs VP series)
  • Also making programming more sophisticated, closer to CPU
  • Floating point textures and image memory (ATI and 3DLabs VP series)
  • Higher dynamic range -> better image quality, better for programming
input processing
Input Processing
  • Users need to interact with the environment
  • they have a set of input devices, have position and orientation information
  • need to translate this into their intentions
  • Interaction Technique (IT): basic unit of interaction, converts user input into something the application understands
input processing1
Input Processing
  • Each IT address a particular interaction task, something that the user wants to do
  • look at interaction tasks first, then talk a little bit about ITs for them
  • interaction tasks divide into two groups:
    • application independent: required by many different applications
    • application dependent
interaction tasks
Interaction Tasks
  • Mainly look at application independent interaction tasks
  • the main ones are:
    • navigation
    • selection
    • manipulation
    • combination
navigation
Navigation
  • Need to get from one part of the environment to another
  • two types:
    • local
    • global
  • with local navigation the destination is within view, move on continuous path from current location to destination
navigation1
Navigation
  • In global navigation the destination is remote, can’t move directly to it
  • need some way of locating destination, and then some way of jumping to it
  • variation: browsing / exploration don’t have a destination, exploring the environment or looking for particular objects
selection
Selection
  • The selection tasks involves selecting something
  • there are several variations, depending upon what’s being selected:
    • list or command selection
    • object selection
    • location selection
selection1
Selection
  • List selection: a pre-defined list of things to select from
  • example: the commands on a menu
  • need to present the list, and the user selects one item from the list
  • object selection: number of objects not pre-defined, created by the user, changes in size as the program runs
selection2
Selection
  • For object selection can’t just present a list of objects to be selected from
  • location selection: selecting a point in space, may be used as location of object, or as part of an object’s shape
  • can’t see a point in empty space, so this is harder than the previous two
manipulation
Manipulation
  • Standard set of object manipulations, change position, size and orientation
  • grab the object and move it
  • could also have deformations that change the object’s shape
  • hard to get general techniques beyond the few standard ones
combination
Combination
  • Take two or more objects and put them together to form a new object
  • need to match up the shapes exactly, so they join in the right way
  • difficult to do unaided, usually need some form of constraint to simplify the process
application dependent tasks
Application Dependent Tasks
  • Usually involve the application data
  • ways of controlling the view of the data
  • ways of manipulating the data
  • example: a CAD or animation program will have a different set of manipulations than a network visualization program
interaction techniques
Interaction Techniques
  • Not a well established set of techniques, yet
  • depend on input devices and style
  • example: a fixed range device (tracker) sometimes works best with pointing at objects, while a puck or joystick might work better with grabbing
  • need to try different combinations
interaction techniques1
Interaction Techniques
  • Some problems encountered:
    • distance: objects too far away to grab
    • feedback: how do you highlight the object that has been selected?
    • Object to be selected may be hidden by other objects
    • object density may make selection and manipulation difficult
application processing
Application Processing
  • Not much to say here
  • some applications have a considerable amount of processing, computation based on user input
  • don’t want this to effect application latency
  • need to control resources devoted to computation, use other processors
making it run right
Making it run right
  • Now that we have an idea of what’s involved, how do we put it all together
  • want to have low system latency, get images on the screen as fast as possible
  • don’t want to wait for anything
  • divide the application into components that execute separately
decoupled simulation model
Decoupled Simulation Model
  • Separate process for application computations, this is easy
  • separate process for expensive input devices, trackers that need lots of computation or have latency
  • a separate process for input processing and display
  • maybe a separate process for model
application structure3
Application Structure

Application

Processing

Input

Devices

Model

Input

Processing

Model Traversal

Output Devices

decoupled simulation model1
Decoupled Simulation Model
  • Each process can run at its own rate
  • display process runs as fast as possible, doesn’t wait for other processes
  • uses most recent value from input devices and application computation
  • reduces system latency
ad