from mobile augmented reality to ubiquitous augmented reality n.
Skip this Video
Loading SlideShow in 5 Seconds..
From Mobile Augmented Reality to Ubiquitous Augmented Reality PowerPoint Presentation
Download Presentation
From Mobile Augmented Reality to Ubiquitous Augmented Reality

Loading in 2 Seconds...

play fullscreen
1 / 42

From Mobile Augmented Reality to Ubiquitous Augmented Reality - PowerPoint PPT Presentation

  • Uploaded on

From Mobile Augmented Reality to Ubiquitous Augmented Reality. Prof. Dr. Dieter Schmalstieg Institut für Maschinelles Sehen und Darstellen Technische Universität Graz. What is Augmented Reality?. Definition of Augmented Reality. 1. Blends real & virtual, in real environment +

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation

PowerPoint Slideshow about 'From Mobile Augmented Reality to Ubiquitous Augmented Reality' - ora-hudson

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
from mobile augmented reality to ubiquitous augmented reality

From Mobile Augmented Reality to Ubiquitous Augmented Reality

Prof. Dr. Dieter Schmalstieg

Institut für Maschinelles Sehen und Darstellen

Technische Universität Graz

definition of augmented reality
Definition of Augmented Reality

1. Blends real & virtual, in real environment


2. Real-time interactive


3. Registered in 3D

[Azuma PRESENCE 1997]

the promise of mobile ar
The Promise of Mobile AR
  • Bring computer support to task location
  • Examples
    • Navigation
    • Tourism
    • Repair & maintainence
    • In-situ inspection
    • As-built documentation
    • Situation awareness
milgram continuum
Milgram Continuum

Mixed Reality



Reality (AR)


Virtuality (AV)



milgram weiser continuum
Milgram-Weiser Continuum


This is what we are interested in!

[Newman VR 2007]


how to get to ubiquitous ar
How to get to Ubiquitous AR
  • Address scalability problems
  • Only partially considered in typical research
  • (Will discuss technical, not commercial aspects)
ar information processing pipeline



AR System




AR Information Processing Pipeline

* Creation by hand

* Conversion of legacy data

* Creation by inference

* Federation

* Metadata

* HW and SW

* Ergonomic

* Inexpensive

* Robust

* Filtering

* Context aware

[Schmalstieg CG&A 2007]

topics for today
Topics for Today
  • Handheld Augmented Reality
    • Ergonomic
    • inexpensive system platform
  • AR Museum Scavenger Hunt
    • Content creation and delivery
  • Visualization of subsurface infrastructure
    • Conversion of legacy data, metadata
    • Context-aware visualization
ergonomic considerations
Ergonomic Considerations

How long until wearable AR becomes ergonomically andsocially acceptable?

augmented reality on pdas
Augmented Reality on PDAs
  • Stand-alone handheld AR on Smartphones
    • No thin client
    • High scalability
  • PDAs are
    • Inexpensive
    • Socially acceptable
    • Familiar
    • Widespread
  • All in one solution
    • Display
    • Touch screen
    • Camera
the invisible train
The Invisible Train

[Wagner SIGGRAPH EMT 2004]

real time computer vision on phones
Real-Time Computer Vision on Phones

Just artificial fiducial tracking, but it must work…

Adaptive tresholding/vignetting

…all with single fixed point

unit at ~200MHz

BCH or DataMatrix markers

[Wagner ISCW 2003]

tracking performance
Tracking Performance

Performance scales linearly with clock

About 10-20 times slower than PC

overall performance
Overall Performance

12 triangles 2625 triangles 27219 triangles

content creation and delivery
Content Creation and Delivery

„Expedition Schatzsuche“

  • Multiuser Game in Landesmuseum Kärnten
  • Make museum visit more attractive to kids 8-12y
  • Runs on Gizmondo (Windows CE game console)

[Schmalstieg ISMAR 2007]

software architecture



Software architecture
  • Studierstube ES: software stack for client
  • Sphinx: Game server, XML database + finite state machine
  • Game state on server (prefered) or client
  • Interaction on client in 3D (AR) or 2D
  • Bootstrapping and content streaming
game elements
Game Elements
  • Diamonds/question marks tell state of a station
    • green: free, yellow: in use, red: already solved
  • Solve task by taking a picture of exhibit or solve a problem
  • Map of the museum showing station availability
game brauchtum
Game „Brauchtum“
  • Retrieve flat iron for finding out initials of wedding couple
  • Puts the iron into the tailors’ oven
  • Learn about the wedding rider and finish the sequence
game musical tradition
Game „Musical Tradition“
  • Playing the silent piano
  • Listening to the Mandora;
  • Applying the “Stimmbogen” to horn
  • Keeping the organ playing by pumping up the bellows
content creation
Content Creation
  • Import from standard tools when possible
    • Maya, Flash…
    • XMI for visual programming of state charts
  • Online tweaking of the database
  • Lesson: content creation consumed several person month, for a small game!
application scenario

Do dig or not to dig?

  • Use AR „X-Ray-Vision“




STEG 941


STEG 902

Application Scenario
  • Utility companies maintain a GIS of their subsurface infrastructure
  • Grintec GesmH (Graz), GE SmallworldTM
how superman does it
How Superman does it

Lois Lane is going to be alright!

ar information filtering
AR Information Filtering
  • Assume large DB incl. metadata
  • Avoid display clutter?
  • Display only relevant information
    • Important for user
    • Within field of view
  • Procedurally determined, per object
  • Better: (Meta-)data driven,per pixel

[Höllerer ISAR 2001]

scene graph with context markup
Scene Graph with Context Markup
  • Problem
    • Context (metadata) scattered throughout scene graph
    • Finding information requires full search
    • Manipulation may require changing SG in many places
    • Acerbated by mix-and-match scene graph composition, even at runtime!
  • Need to delegate behavior modification to local nodes and their interaction with near nodes

Where is a “blue” nodehidden in the SG?

subsurface data example
Subsurface Data Example


Welding in the town hallmust not happen near gas

  • Accumulate context (= user defined state) during traversal
    • Aggregate markup „set“
    • Generic parameter passing technique
    • Parameters can be sub-scenegraphs
    • Scene graph becomes a template structure
    • Binding in the last possible moment

[Reitmayr VR 2005]

example 3d model stylesheets
Example: 3D Model Stylesheets

world in miniature

Registered overlay

generic model

example 3d model stylesheets1
Example: 3D Model Stylesheets

Registered overlay

World in miniature







Wall style

generic model





dynamic binding of styles
Dynamic Binding of Styles

Danger (red) defined by rules (telemetry, user)

[Mendez ISMAR 2007]

magic lenses for filtering
Magic Lenses for Filtering
  • Spatial + semantic context

[Schall Ubicomp 2007]

visual focus context
Visual Focus+Context

[Kalkofen ISMAR 2007]

Dense scene

Depth perception in AR overlay?

focus context approach
Focus+Context Approach

G-Buffer Technique similar to NPR

Markup „families“ define G-Buffers

Single pass / multiple target rendering

Compositing for dynamic focus+context

Markup defines families

 multiple GBuffers

GBuffer for one family

video processing
Video Processing

Incorporate video image as GBuffer


Explicit depth sorting

Raycasting-like compositing

Scriptable compositing rules



legacy database conversion
Legacy database conversion
  • Procedural modeling to convert GIS to AR
  • Uses Generative Modeling Language[Havemann 2005]
  • Geometry creation on-the-fly
  • Markup provides control parameters for procedural modeling


Progressive revealing


putting it all together
Putting it all together

Submitted to CG&A, 2007

ongoing and future work
Ongoing and Future Work
  • Organize meta-information into ontologies
    • Topological relationships: containment, adjacency, …
    • Pedestrian navigation
    • Indoor infrastructure (e.g. power, network)
    • Use semantic reasoning to infer new data
    • Also infer visualization rules
  • Natural feature tracking for phones
thank you for your attention
Thank you for your attention!


Istvan Barakonyi

Sven Havemann

Denis Kalkofen

Joe Newman

Thomas Pintaric

Gerhard Reitmayr

Gerhard Schall

Daniel Wagner