parallel tracking and mapping for small ar workspaces vision seminar l.
Download
Skip this Video
Download Presentation
Parallel Tracking and Mapping for Small AR Workspaces Vision Seminar

Loading in 2 Seconds...

play fullscreen
1 / 35

Parallel Tracking and Mapping for Small AR Workspaces Vision Seminar - PowerPoint PPT Presentation


  • 238 Views
  • Uploaded on

Parallel Tracking and Mapping for Small AR Workspaces Vision Seminar. 2008. 9. 4 (Thu) Young Ki Baik Computer Vision Lab. References. Parallel Tracking and Mapping for Small AR Workspaces Georg Klein, David Murray, ISMAR 2007 Visual Odometry David Nister et. al. , CVPR 2004. Outline.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Parallel Tracking and Mapping for Small AR Workspaces Vision Seminar' - marlow


Download Now An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
parallel tracking and mapping for small ar workspaces vision seminar

Parallel Tracking and Mapping for Small AR WorkspacesVision Seminar

2008. 9. 4 (Thu)

Young Ki Baik

Computer Vision Lab.

references
References
  • Parallel Tracking and Mapping for Small AR Workspaces
    • Georg Klein, David Murray, ISMAR 2007
  • Visual Odometry
    • David Nister et. al. , CVPR 2004
outline
Outline
  • What is AR?
  • Previous works
  • Proposed methods
  • Demo
  • Conclusion
what is ar
What is AR?
  • AR: Augment Reality

ugumentreality(AR) uses the real scene as the background, and

makes applications with putting 3D objects to the background.

Since the cost of augment reality is not as expensive asfull 3D virtual reality(VR), AR has been very popular research topic.

A

Full 3D (X)

Expensive? (X)

Full 3D? (O) Expensive? (O)

the aim
The Aim
  • AR with a hand-held camera
  • Visual Tracking provides registration
the aim7
The Aim
  • AR with a hand-held camera
  • Visual Tracking provides registration
  • Track without prior model of world
the aim8
The Aim
  • AR with a hand-held camera
  • Visual Tracking provides registration
  • Track without prior model of world
  • Challenges
    • Speed
    • Accuracy
    • Robustness
    • Interaction with real world
existing attempts slam
Existing attempts : SLAM
  • SLAM : Simultaneous Localization and Mapping

can use many different types of sensor to acquire observation data used in building the map such as laser rangefinders, sonar sensors and cameras.

    • Well-established in robotics (using a rich array of sensors)
    • Demonstrated with a single hand-held camera by Davison at 2003 (Mono-SLAM).
    • Mono-SLAM was applied to

AR system at 2004.

existing attempts model based tracking
Existing attempts : Model based tracking
  • Model-based tracking is
    • More robust
    • More accurate
    • Proposed by Lepetit et. al.

at ISMAR 2003

frame by frame slam
Frame by Frame SLAM
  • Why?

is SLAM fundamentally harder?

Time

One frame

Find features

Many DOF

Update camera pose and entire map

Draw graphics

frame by frame slam12
Frame by Frame SLAM
  • SLAM
    • Updating entire map every frame is so expensive!!!
    • Needs “sparse map of high-quality features”

- A. Davison

  • Proposed approach
    • Use dense map(of low quality features)
    • Don’t update the map every frame : Keyframes
    • Split the tracking and mapping into two threads
parallel tracking and mapping
Parallel Tracking and Mapping
  • Proposed method

- Split the tracking and mapping into two threads

Time

Thread #2

Mapping

Update map

One frame

Thread #1

Tracking

Find features

Simple & easy

Update camera pose only

Draw graphics

parallel tracking and mapping14
Parallel Tracking and Mapping

Tracking thread:

  • Responsible estimation of camera pose and rendering augmented graphics
  • Must run at 30 Hz
  • Make as robust and accurate as possible

Mapping thread:

  • Responsible for providing the map
  • Can take lots of time per key frame
  • Make as rich and accurate as possible
tracking thread
Tracking thread
  • Overall flow

Map

Pre-process frame

Project points

Project points

Measure points

Measure points

Update Camera Pose

Update Camera Pose

Coarse stage

Fine stage

Draw Graphics

pre process frame
Pre-process frame
  • Make for pyramid levels

80x60

160x120

640x480

320x240

pre process frame17
Pre-process frame
  • Make for pyramid levels
  • Detect Fast corners
    • E. Rosten et al (ECCV 2006)

80x60

160x120

640x480

320x240

project points
Project Points
  • Use motion model to update camera pose
    • Constant velocity model

Estimatedcurrent Pt+1

Previous posPt

∇t’

Previous pos Pt-1

∇t

Vt =(Pt – Pt-1)/∇t

Pt+1=Pt+∇t’(Vt)

project points19
Project Points
  • Choose subset to measure
    • ~ 50 biggest features for coarse stage
    • 1000 randomly selected for fine stage

~50

1000

80x60

160x120

640x480

320x240

measure points
Measure Points
  • Generate 8x8 matching template (warped from source key-frame:map)
  • Search a fixed radius around projected position
    • Use Zero-mean SSD
    • Only search at Fast corner points
update caemra pose
Update caemra pose
  • 6-DOF problem
    • Obtain by SFM (Three-point algorithm)

?

dray graphics
Dray graphics
  • What can we draw in an unknown scene?
    • Assume single plane visible at start
    • Run VR simulation on the plane
mapping thread
Mapping thread
  • Overall flow

Stereo Initialization

Tracker

Wait for new key frame

Add new map points

Optimize map

Map maintenance

stereo initialization
Stereo Initialization
  • Use five-point-pose algorithm
    • D. Nister et. al. 2006
  • Requires a pair of frames and feature correspondences
  • Provides initial map
  • User input required:
    • Two clicks for two key-frames
    • Smooth motion for feature correspondence
wait for new key frame
Wait for new key frame
  • Key frames are only added if :
    • There is a sufficient baseline to the other key frame
    • Tracking quality is good
    • Key frame (4 level pyramid images and its corners)
  • When a key frame is added :
    • The mapping thread stops whatever it is doing
    • All points in the map are measured in the key frame
    • New map points are found and added to the map
add new map points
Add new map points
  • Want as many map points as possible
  • Check all maximal FAST corners in the key frame :
    • Check score
    • Check if already in map
  • Epipolar search in a neighboring key frame
  • Triangulate matches and add to map
  • Repeat in four image pyramid levels
optimize map
Optimize map
  • Use batch SFM method: Bundle Adjustment
  • Adjusts map point positions and key frame poses
  • Minimize reprojection error of all points in all keyframes (or use only last N key frames)
map maintenance
Map maintenance
  • When camera is not exploring, mapping thread has idle time
  • Data association in bundle adjustment is reversible
  • Re-attempt outlier measurements
  • Try measure new map features in all old key frames
comparison to ekf slam
Comparison to EKF-SLAM
  • More Accurate
  • More robust
  • Faster tracking

<

SLAM based AR

Proposed AR

system and results
System and Results
  • Environment
    • Desktop PC (Intel Core 2 Duo 2.66 GHz)
    • OS : Linux
    • Language : C++
  • Tracking speed
system and results31
System and Results
  • Mapping scalability and speed
    • Practical limit
      • 150 key frames
      • 6000 points
    • Bundle adjustment timing
remaining problem
Remaining problem
  • Outlier management
  • Still brittle in some scenario
    • Repeated texture
    • Passive stereo initialization
  • Occlusion problem
  • Relocation problem
conclusion
Conclusion
  • Conclusion
    • Parallel tracking and mapping process are presented using multi-thread.
  • Contribution
    • Visual odometry system was well presented.
    • Overcome computation by multi-thread
  • Opinion
    • The proposed algorithm can be applied to our research
      • Navigation system
      • 3D tracking system