Recognizing objects in range data using regional point descriptors
Download
1 / 26

notes - PowerPoint PPT Presentation


  • 326 Views
  • Updated On :

Recognizing Objects in Range Data Using Regional Point Descriptors a.k.a. 3D Shape Contexts A. Frome, D. Huber , R. Kolluri, T. Bulow, and J. Malik . Proceedings of the European Conference on Computer Vision , May, 2004. Talk prepared by Nat Duca, [email protected] Motivation

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'notes' - emily


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Recognizing objects in range data using regional point descriptors l.jpg

Recognizing Objects in Range Data Using Regional Point Descriptors

a.k.a. 3D Shape Contexts

A. Frome, D. Huber, R. Kolluri, T. Bulow, and J. Malik. Proceedings of the European Conference on Computer Vision, May, 2004.

Talk prepared by Nat Duca, [email protected]


Motivation l.jpg
Motivation Descriptors

  • Find instances of known shapes in 2.5D range scans

Image source: Frome04


2d shape contexts l.jpg
2D Shape Contexts Descriptors

  • Take a random point on the shape

Image source: Belongie02


2d shape contexts4 l.jpg
2D Shape Contexts Descriptors

  • Compute the offset vectors to all other samples


2d shape contexts5 l.jpg
2D Shape Contexts Descriptors

  • Histogram the vectors against sectors and shells

  • Perform this for a large sampling of points


Extension to 3d l.jpg
Extension to 3D Descriptors

  • Step 1: pick random points on surface

Image source: Koertgen03


Extension to 3d7 l.jpg
Extension to 3D Descriptors

  • For each point, compute and histogram offsets

Image source: Koertgen03


Extension to 3d8 l.jpg
Extension to 3D Descriptors

  • For each point, compute offsets

Image source: Koertgen03


Extension to 3d9 l.jpg
Extension to 3D Descriptors

  • Now we histogram the offset vectors.

  • The 3D histogram of looks like:

Image source: Frome04


Extension to 3d10 l.jpg
Extension to 3D Descriptors

  • Shells are spaced logarithmically apart

  • Histogram votes are weighted by the volume of the bin

  • Some Ln difference of the histogram vector can be used to compare two contexts

Image source: Frome04, Koertgen03


Challenges l.jpg
Challenges Descriptors

  • How do we orient the histogram “spheres”

  • How do we compute distance between a model and one of its subsets?

  • Speed


Initial histogram orientation l.jpg
Initial histogram orientation Descriptors

  • Align the object’s north-pole to the surface normal

  • Problems:

    • One degree of freedom remains

    • Histogram values depend on the precision of the surface normals

  • The paper solves both problems using:

    • Brute force rotation

    • spherical harmonics


Harmonic shape context l.jpg
Harmonic shape context Descriptors

  • Each shell’s histogram is a spherical function

  • Convert each shell to a harmonic representation and store the amplitude coefficients only

  • Initial histogram placement doesn’t matter,

  • Noise in surface normals doesn’t affect descriptor

Image source: Weisstein04


The big picture partial shape matching l.jpg
the big picture: Partial Shape Matching Descriptors

  • For a query shape Sq and a stored model Si, their nearness is defined as:

A shape context placed randomly on the query surface Sq

A precomputed shape context for model Si query surface


Experiment 1 resilience to noise l.jpg
Experiment 1: resilience to noise Descriptors

  • (a) model with 5cm gaussian noise

  • (b) model with 10cm gaussian noise

  • (c) reference (databased) model

Image source: Frome04


Experiment 2 partial matching l.jpg
Experiment 2: partial matching Descriptors

  • Input:

    or

  • Output:

Image source: Frome04


Evaluating the results l.jpg
Evaluating the results Descriptors

  • Where does the blame lie:

    • Spherical histogram

    • Harmonics representation

    • Point choice

    • Representative descriptor approach

  • Is their presentation fair?


Results for noise l.jpg
Results for noise Descriptors

  • Where does the blame lie:

    • Spherical histogram

    • Harmonics representation

    • Point choice

    • Representative descriptor

  • Is their presentation fair?

    Comments:

  • Recognition rate: across 100 trials, how many times did we get the correct answer back the first time?

  • All three techniques are equivalent in absence of noise

Results for 5cm noise

Image source: Frome04


Results for noise19 l.jpg
Results for noise Descriptors

  • Where does the blame lie:

    • Spherical histogram

    • Harmonics representation

    • Point choice

    • Representative descriptor

  • Is their presentation fair?

    Comments:

  • Why is the harmonic approach doing worse? We expect it to be doing as well or better than the basic approach

10cm noise, 55cm normal window

Image source: Frome04


Results for noise20 l.jpg
Results for noise Descriptors

  • Where does the blame lie:

    • Spherical histogram

    • Harmonics representation

    • Point choice

    • Representative descriptor

  • Is their presentation fair?

    Comments:

  • Notice how, when the normals are better filtered, the harmonics do better! How can this be so?

10cm noise, 105cm normal window

Image source: Frome04


Results for partial matching l.jpg
Results for partial matching Descriptors

  • Where does the blame lie:

    • Spherical histogram

    • Harmonics representation

    • Point choice

    • Representative descriptor

  • Is their presentation fair?

    Comments:

  • Rank depth of R means that the correct answer appeared in the top R results.

  • Clearly, the harmonics are throwing away too much

  • Or is the fact that the shells are rotationally independent to blame?

View 1

Image source: Frome04


Results for partial matching22 l.jpg
Results for partial matching Descriptors

  • Where does the blame lie:

    • Spherical histogram

    • Harmonics representation

    • Point choice

    • Representative descriptor

  • Is their presentation fair?

    Comments:

  • Rank depth of R means that the correct answer appeared in the top R results.

  • The authors claim that the ground is setting off the match

View 2

Image source: Frome04


Speed considerations l.jpg
Speed considerations Descriptors

  • We use a spherical hash with J sectors, and KxL latitudinal and longitudinal divisions

  • The basic vector is (roughly) J x K x L in size

  • The harmonic representation is roughly the same size

  • Without harmonics, they must store L extra rotations in order: J x K x L2

  • They useLocality Sensitive Hashing to reduce the amount of effort required here:


Speed considerations lsh results l.jpg
Speed considerations: LSH results Descriptors

Without hashing

Image source: Frome04


Summary l.jpg
Summary Descriptors

  • What was introduced:

    • 3D histogram extension of 2D shape contexts

    • A poorly-performing spherical harmonic decomposition of the 3D histogram

    • The representative decriptor method works pretty well

  • What would have been nice:

    • Precision of query when the shells are logarithmically or linearly separated

    • Is the representative descriptor approach the limiting factor? We need more data to confirm or deny!


Image sources l.jpg
Image sources Descriptors

Frome04: A. Frome, D. Huber, R. Kolluri, T. Bulow, and J. Malik. Proceedings of the European Conference on Computer Vision, May, 2004

Belongie02: S. Belongie et al. Shape matching and object recognition using shape contexts. IEEE Trans on Pattern Analysis and Machine Intelligence. 24(4):509-522, April 2002.

Koertgen03: M. Körtgen, G.-J. Park, M. Novotni, R. Klein "3D Shape Matching with 3D Shape Contexts", in proceedings of The 7th Central European Seminar on Computer Graphics, April 2003

Weisstein: Eric W. Weisstein. "Spherical Harmonic." From MathWorld--A Wolfram Web Resource. http://mathworld.wolfram.com/SphericalHarmonic.html


ad