dynamic 3d scene analysis from a moving vehicle
Download
Skip this Video
Download Presentation
Dynamic 3D Scene Analysis from a Moving Vehicle

Loading in 2 Seconds...

play fullscreen
1 / 28

Dynamic 3D Scene Analysis from a Moving Vehicle - PowerPoint PPT Presentation


  • 139 Views
  • Uploaded on

Dynamic 3D Scene Analysis from a Moving Vehicle. Young Ki Baik (CV Lab.) 2007. 7. 11 (Wed). Dynamic Scene Analysis from a Moving Vehicle. References. Dynamic 3D Scene Analysis from a Moving Vehicle Bastian Leibe, Nico Cornelis, Kurt cornelis, Luc Van Gool

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Dynamic 3D Scene Analysis from a Moving Vehicle' - gavivi


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
dynamic 3d scene analysis from a moving vehicle

Dynamic 3D Scene Analysis from a Moving Vehicle

Young Ki Baik (CV Lab.)

2007. 7. 11 (Wed)

dynamic scene analysis from a moving vehicle
Dynamic Scene Analysis from a Moving Vehicle
  • References
  • Dynamic 3D Scene Analysis from a Moving Vehicle
    • Bastian Leibe, Nico Cornelis, Kurt cornelis, Luc Van Gool
    • Awarded the best paper prize (CVPR 2007)
  • Fast Compact City Modeling for Navigation Pre-Visualization
    • Nico Cornelis, Kurt cornelis, Luc Van Gool (CVPR 2006)
  • Pedestrian detection in crowded scene
    • Bastian Leibe et. al. (CVPR 2005)
  • Putting Objects in Perspective
    • Derek Hoiem et. al. (CVPR 2006)
dynamic scene analysis from a moving vehicle1
Dynamic Scene Analysis from a Moving Vehicle
  • Why?
  • … were they received the best paper prize?
  • They completed the impressive real application with only toy computer vision algorithm.
  • They showed that the field of vision will be a key of future technique to the public.
dynamic scene analysis from a moving vehicle3
Dynamic Scene Analysis from a Moving Vehicle
  • What?
  • …is the purpose of this paper?
  • Detect object in real environment (city road)
  • Localize them in 3D
  • Predict their future motion
  • … is the challenges of this paper?
  • We are moving
  • Objects can be moving
  • Ground may not be planar
dynamic scene analysis from a moving vehicle4
Dynamic Scene Analysis from a Moving Vehicle
  • What methods?
    • … are used to accomplish their purpose?
      • Structure from motion
      • 2D object detection
      • 3D trajectory estimation
dynamic scene analysis from a moving vehicle5
Stereo camera

Aligned stereo image

3D structure info.

Ground plane

2D and 3D Object

3D trajectory

Orientation

Dynamic Scene Analysis from a Moving Vehicle
  • Overall flow

1. SfM

3. Tracking

2. Object detection

dynamic scene analysis from a moving vehicle6
Dynamic Scene Analysis from a Moving Vehicle
  • 3D structure and ground plane
    • 3D Structure from Motion
      • Visual odometry (David Nister)
        • Use pre-calibrated stereo camera
        • Use rectified stereo images
        • Parallel processing

→ Extrinsic camera parameters

→ 3D camera trajectory (in real time)

Nico Cornelis et. al. (CVPR 2006)

dynamic scene analysis from a moving vehicle7
Dynamic Scene Analysis from a Moving Vehicle
  • 3D structure and ground plane
    • Ground plane estimation
      • Known ground positions of wheel base points

Nico Cornelis et. al. (CVPR 2006)

dynamic scene analysis from a moving vehicle8
Dynamic Scene Analysis from a Moving Vehicle
  • 3D structure and ground plane
    • Ground plane estimation
      • Compute normal locally
        • Average over spatial window

Nico Cornelis et. al. (CVPR 2006)

dynamic scene analysis from a moving vehicle9
Dynamic Scene Analysis from a Moving Vehicle
  • SfM Demo

Nico Cornelis et. al. (CVPR 2006)

dynamic scene analysis from a moving vehicle10
Dynamic Scene Analysis from a Moving Vehicle
  • Object detection
    • 2D/3D Interaction method
      • Likelihood of 3D object hypothesis H

→ Given image I and a set of 2D detections h:

dynamic scene analysis from a moving vehicle11
Dynamic Scene Analysis from a Moving Vehicle
  • Object detection
    • 2D object detection

2D recognition

  • ISM detectors

Leibe et. al. (CVPR 2005)

dynamic scene analysis from a moving vehicle12
Dynamic Scene Analysis from a Moving Vehicle
  • Object detection
    • ISM detectors (Leibe et al., CVPR’05, BMVC’06)
      • Battery of 5(car)+1(human) single view detectors
      • Each detectors based on 3 local cues
        • Harris-Laplace, Hessian-Laplace, DoG interest regions
        • Local Shape Context descriptors
      • Result: detections + segmentations

Leibe et. al. (CVPR 2005)

dynamic scene analysis from a moving vehicle13
Dynamic Scene Analysis from a Moving Vehicle
  • Object detection
    • 2D/3D transfer

2D/3D transfer

  • Two image-plane detections are consistent if they correspond to the same 3D object.

→ Cluster 3D detections

→ Multi-viewpoint integration

dynamic scene analysis from a moving vehicle14
Dynamic Scene Analysis from a Moving Vehicle
  • Object detection
    • 3D prior

3D prior

  • By Using 3D structure and ground plane constraint…

→ Distance prior (Distance from the ground plane)

→ Size prior (Gaussian)

Significantly reduced search space and outlier

Hoiem et. al. (CVPR 2006)

dynamic scene analysis from a moving vehicle15
Dynamic Scene Analysis from a Moving Vehicle
  • Quantitative results of detection
    • Detection performance on 2 test sequences
      • Stereo and Ground plane constraints significantly improves precision
dynamic scene analysis from a moving vehicle17
Dynamic Scene Analysis from a Moving Vehicle
  • Object tracking
    • Localization and Trajectory estimation
      • By using detection results
      • Obtain orientation of objects
    • Space-time trajectory analysis
      • By using the concept of a bidirectional Extended Kalman Filter
dynamic scene analysis from a moving vehicle18
Dynamic Scene Analysis from a Moving Vehicle
  • Object tracking
    • 3D Localization for static objects (car)
      • Location
        • Mean-shift search to find set of 3D detection hypotheses
      • Orientation
        • Cluster shape and detector output
dynamic scene analysis from a moving vehicle19
Dynamic Scene Analysis from a Moving Vehicle
  • Object tracking
    • Dynamic model
      • Holonomic motion (Pedestrian)
        • Without external constraints linking its speed and turn rate
      • Nonholonomic motion (Car)
        • Only move along its main axis
        • Only turn while moving
dynamic scene analysis from a moving vehicle20
Dynamic Scene Analysis from a Moving Vehicle
  • Object tracking
    • Trajectory growing
      • Collect detection in time space
dynamic scene analysis from a moving vehicle21
Dynamic Scene Analysis from a Moving Vehicle
  • Object tracking
    • Trajectory growing
      • Collect detection in time space
      • Evaluate under trajectory
        • Bi-directionally
        • Static assumption
dynamic scene analysis from a moving vehicle22
Dynamic Scene Analysis from a Moving Vehicle
  • Object tracking
    • Trajectory growing
      • Collect detection in time space
      • Evaluate under trajectory
        • Bi-directionally
        • Static assumption
      • Adjust trajectory
        • Weighted mean
          • Predicted position
          • Supporting observations
dynamic scene analysis from a moving vehicle23
Dynamic Scene Analysis from a Moving Vehicle
  • Object tracking
    • Trajectory growing
      • Collect detection in time space
      • Evaluate under trajectory
        • Bi-directionally
        • Static assumption
      • Adjust trajectory
        • Weighted mean
          • Predicted position
          • Supporting observations
      • Iteration
dynamic scene analysis from a moving vehicle24
Dynamic Scene Analysis from a Moving Vehicle
  • Object tracking
    • Trajectory growing
      • Collect detection in time space
      • Evaluate under trajectory
        • Bi-directionally
        • Static assumption
      • Adjust trajectory
        • Weighted mean
          • Predicted position
          • Supporting observations
      • Iteration
      • Location and orientation
dynamic scene analysis from a moving vehicle26
Dynamic Scene Analysis from a Moving Vehicle
  • Conclusion
    • Summary
      • Exact value of 3D information
        • help to propose the new concept of detection algorithm
        • raise the performance of detection algorithm.
      • Better detection results
        • Give more reliable tracking results
        • Good orientation estimation
    • Contribution
      • New detection algorithm using 3d information
      • Good integration and visualization of application system
ad