Dynamic 3d scene analysis from a moving vehicle
This presentation is the property of its rightful owner.
Sponsored Links
1 / 28

Dynamic 3D Scene Analysis from a Moving Vehicle PowerPoint PPT Presentation


  • 90 Views
  • Uploaded on
  • Presentation posted in: General

Dynamic 3D Scene Analysis from a Moving Vehicle. Young Ki Baik (CV Lab.) 2007. 7. 11 (Wed). Dynamic Scene Analysis from a Moving Vehicle. References. Dynamic 3D Scene Analysis from a Moving Vehicle Bastian Leibe, Nico Cornelis, Kurt cornelis, Luc Van Gool

Download Presentation

Dynamic 3D Scene Analysis from a Moving Vehicle

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Dynamic 3d scene analysis from a moving vehicle

Dynamic 3D Scene Analysis from a Moving Vehicle

Young Ki Baik (CV Lab.)

2007. 7. 11 (Wed)


Dynamic scene analysis from a moving vehicle

Dynamic Scene Analysis from a Moving Vehicle

  • References

  • Dynamic 3D Scene Analysis from a Moving Vehicle

    • Bastian Leibe, Nico Cornelis, Kurt cornelis, Luc Van Gool

    • Awarded the best paper prize (CVPR 2007)

  • Fast Compact City Modeling for Navigation Pre-Visualization

    • Nico Cornelis, Kurt cornelis, Luc Van Gool (CVPR 2006)

  • Pedestrian detection in crowded scene

    • Bastian Leibe et. al. (CVPR 2005)

  • Putting Objects in Perspective

    • Derek Hoiem et. al. (CVPR 2006)


Dynamic scene analysis from a moving vehicle1

Dynamic Scene Analysis from a Moving Vehicle

  • Why?

  • … were they received the best paper prize?

  • They completed the impressive real application with only toy computer vision algorithm.

  • They showed that the field of vision will be a key of future technique to the public.


Dynamic scene analysis from a moving vehicle2

Dynamic Scene Analysis from a Moving Vehicle

  • Demo (Final result)


Dynamic scene analysis from a moving vehicle3

Dynamic Scene Analysis from a Moving Vehicle

  • What?

  • …is the purpose of this paper?

  • Detect object in real environment (city road)

  • Localize them in 3D

  • Predict their future motion

  • … is the challenges of this paper?

  • We are moving

  • Objects can be moving

  • Ground may not be planar


Dynamic scene analysis from a moving vehicle4

Dynamic Scene Analysis from a Moving Vehicle

  • What methods?

    • … are used to accomplish their purpose?

      • Structure from motion

      • 2D object detection

      • 3D trajectory estimation


Dynamic scene analysis from a moving vehicle5

Stereo camera

Aligned stereo image

3D structure info.

Ground plane

2D and 3D Object

3D trajectory

Orientation

Dynamic Scene Analysis from a Moving Vehicle

  • Overall flow

1. SfM

3. Tracking

2. Object detection


Dynamic scene analysis from a moving vehicle6

Dynamic Scene Analysis from a Moving Vehicle

  • 3D structure and ground plane

    • 3D Structure from Motion

      • Visual odometry (David Nister)

        • Use pre-calibrated stereo camera

        • Use rectified stereo images

        • Parallel processing

          → Extrinsic camera parameters

          → 3D camera trajectory (in real time)

Nico Cornelis et. al. (CVPR 2006)


Dynamic scene analysis from a moving vehicle7

Dynamic Scene Analysis from a Moving Vehicle

  • 3D structure and ground plane

    • Ground plane estimation

      • Known ground positions of wheel base points

Nico Cornelis et. al. (CVPR 2006)


Dynamic scene analysis from a moving vehicle8

Dynamic Scene Analysis from a Moving Vehicle

  • 3D structure and ground plane

    • Ground plane estimation

      • Compute normal locally

        • Average over spatial window

Nico Cornelis et. al. (CVPR 2006)


Dynamic scene analysis from a moving vehicle9

Dynamic Scene Analysis from a Moving Vehicle

  • SfM Demo

Nico Cornelis et. al. (CVPR 2006)


Dynamic scene analysis from a moving vehicle10

Dynamic Scene Analysis from a Moving Vehicle

  • Object detection

    • 2D/3D Interaction method

      • Likelihood of 3D object hypothesis H

        → Given image I and a set of 2D detections h:


Dynamic scene analysis from a moving vehicle11

Dynamic Scene Analysis from a Moving Vehicle

  • Object detection

    • 2D object detection

2D recognition

  • ISM detectors

Leibe et. al. (CVPR 2005)


Dynamic scene analysis from a moving vehicle12

Dynamic Scene Analysis from a Moving Vehicle

  • Object detection

    • ISM detectors (Leibe et al., CVPR’05, BMVC’06)

      • Battery of 5(car)+1(human) single view detectors

      • Each detectors based on 3 local cues

        • Harris-Laplace, Hessian-Laplace, DoG interest regions

        • Local Shape Context descriptors

      • Result: detections + segmentations

Leibe et. al. (CVPR 2005)


Dynamic scene analysis from a moving vehicle13

Dynamic Scene Analysis from a Moving Vehicle

  • Object detection

    • 2D/3D transfer

2D/3D transfer

  • Two image-plane detections are consistent if they correspond to the same 3D object.

    → Cluster 3D detections

    → Multi-viewpoint integration


Dynamic scene analysis from a moving vehicle14

Dynamic Scene Analysis from a Moving Vehicle

  • Object detection

    • 3D prior

3D prior

  • By Using 3D structure and ground plane constraint…

    → Distance prior (Distance from the ground plane)

    → Size prior (Gaussian)

Significantly reduced search space and outlier

Hoiem et. al. (CVPR 2006)


Dynamic scene analysis from a moving vehicle15

Dynamic Scene Analysis from a Moving Vehicle

  • Quantitative results of detection

    • Detection performance on 2 test sequences

      • Stereo and Ground plane constraints significantly improves precision


Dynamic scene analysis from a moving vehicle16

Dynamic Scene Analysis from a Moving Vehicle

  • Detection Demo


Dynamic scene analysis from a moving vehicle17

Dynamic Scene Analysis from a Moving Vehicle

  • Object tracking

    • Localization and Trajectory estimation

      • By using detection results

      • Obtain orientation of objects

    • Space-time trajectory analysis

      • By using the concept of a bidirectional Extended Kalman Filter


Dynamic scene analysis from a moving vehicle18

Dynamic Scene Analysis from a Moving Vehicle

  • Object tracking

    • 3D Localization for static objects (car)

      • Location

        • Mean-shift search to find set of 3D detection hypotheses

      • Orientation

        • Cluster shape and detector output


Dynamic scene analysis from a moving vehicle19

Dynamic Scene Analysis from a Moving Vehicle

  • Object tracking

    • Dynamic model

      • Holonomic motion (Pedestrian)

        • Without external constraints linking its speed and turn rate

      • Nonholonomic motion (Car)

        • Only move along its main axis

        • Only turn while moving


Dynamic scene analysis from a moving vehicle20

Dynamic Scene Analysis from a Moving Vehicle

  • Object tracking

    • Trajectory growing

      • Collect detection in time space


Dynamic scene analysis from a moving vehicle21

Dynamic Scene Analysis from a Moving Vehicle

  • Object tracking

    • Trajectory growing

      • Collect detection in time space

      • Evaluate under trajectory

        • Bi-directionally

        • Static assumption


Dynamic scene analysis from a moving vehicle22

Dynamic Scene Analysis from a Moving Vehicle

  • Object tracking

    • Trajectory growing

      • Collect detection in time space

      • Evaluate under trajectory

        • Bi-directionally

        • Static assumption

      • Adjust trajectory

        • Weighted mean

          • Predicted position

          • Supporting observations


Dynamic scene analysis from a moving vehicle23

Dynamic Scene Analysis from a Moving Vehicle

  • Object tracking

    • Trajectory growing

      • Collect detection in time space

      • Evaluate under trajectory

        • Bi-directionally

        • Static assumption

      • Adjust trajectory

        • Weighted mean

          • Predicted position

          • Supporting observations

      • Iteration


Dynamic scene analysis from a moving vehicle24

Dynamic Scene Analysis from a Moving Vehicle

  • Object tracking

    • Trajectory growing

      • Collect detection in time space

      • Evaluate under trajectory

        • Bi-directionally

        • Static assumption

      • Adjust trajectory

        • Weighted mean

          • Predicted position

          • Supporting observations

      • Iteration

      • Location and orientation


Dynamic scene analysis from a moving vehicle25

Dynamic Scene Analysis from a Moving Vehicle

  • Demo (Final result)


Dynamic scene analysis from a moving vehicle26

Dynamic Scene Analysis from a Moving Vehicle

  • Conclusion

    • Summary

      • Exact value of 3D information

        • help to propose the new concept of detection algorithm

        • raise the performance of detection algorithm.

      • Better detection results

        • Give more reliable tracking results

        • Good orientation estimation

    • Contribution

      • New detection algorithm using 3d information

      • Good integration and visualization of application system


  • Login