1 / 24

Shape-Based Retrieval of Articulated 3D Models Using Spectral Embedding

Shape-Based Retrieval of Articulated 3D Models Using Spectral Embedding Varun Jain and Hao Zhang {vjain,haoz}@cs.sfu.ca GrUVi Lab, School of Computing Science Simon Fraser University, Burnaby, BC Canada Problem Overview … … Problem Overview Database Outline Problem Overview

Faraday
Download Presentation

Shape-Based Retrieval of Articulated 3D Models Using Spectral Embedding

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Shape-Based Retrieval of Articulated3D Models Using Spectral Embedding Varun Jain and Hao Zhang {vjain,haoz}@cs.sfu.ca GrUVi Lab, School of Computing Science Simon Fraser University, Burnaby, BC Canada

  2. Problem Overview

  3. … Problem Overview Database • Outline • Problem Overview • Retrieval Problem • Methods • Spectral Embeddings • Shape Descriptors • Results • Future Work • Acknowledgements Shape Retrieval Applications • Computer aided design • Game design • Shape recognition • Face recognition Query Interface User Output

  4. Shape Retrieval • Outline • Problem Overview • Retrieval Problem • Methods • Spectral Embeddings • Shape Descriptors • Results • Future Work • Acknowledgements How? • Using Correspondence Efficiency?? • Using Global Descriptors For 3D shapes • Fourier Descriptors • Light Field • Spherical Harmonics • Skeletal Graph Matching • Non-rigid transformations?? • Stretching • Articulation (bending)

  5. Shape Retrieval • Outline • Problem Overview • Retrieval Problem • Methods • Spectral Embeddings • Shape Descriptors • Results • Future Work • Acknowledgements Our Method • Normalize non-rigid transformations • Construct affinity matrix • Spectral embedding • Use global shape descriptors • Light Field Descriptor (LFD) • Spherical Harmonics Descriptor (SHD) • Eigenvalues??

  6. Shape Retrieval • Outline • Problem Overview • Retrieval Problem • Methods • Spectral Embeddings • Shape Descriptors • Results • Future Work • Acknowledgements Advantages of Our Method • Handles shape articulation (best performance for articulated shapes). • Flexibility of affinity matrices • Robustness of affinity matrices

  7. Spectral Embeddings

  8. 2 ( ) d i j n g e o ; 2 a e 2 = ¾ i j a a a 1 1 1 2 1 ¢ ¢ ¢ n z } | { a 8 2 3 2 1 ¢ ¢ ¢ > . . . > . . . > > 6 7 . . . A = > n > 6 7 > a a a > 6 7 n n < i i i 1 2 6 7 ¢ ¢ ¢ n £ . . . 6 7 . . . 6 7 . . . > > 6 7 > a a a > 6 7 > > 4 5 1 2 ¢ ¢ ¢ n n n n > > : Spectral Embeddings Affinity matrix • Outline • Problem Overview • Spectral Embeddings • Basics • Problems • Solutions • Shape Descriptors • Results • Future Work • Acknowledgements

  9. ¯ ¯ ¯ T A E ¤ E ¯ ¯ ¯ = ¯ ¯ ¯ ¯ ¯ ¯ ¯ ¯ ¯ ¯ ¯ ¯ ¯ ¯ ¯ ¯ ¯ ¯ ¯ ¯ ¯ ¯ ¯ ¯ k ¯ ¯ ¯ ¯ ¯ ¯ k-dimensional spectral embedding coordinates of ith point of P z } | { ~ ~ ~ ¯ ¯ ¯ u u ¢ ¢ ¢ u k 1 2 ¯ ¯ ¯ 8 2 3 ¯ ¯ ¯ ¯ ¯ ¯ > > > > 6 7 > > 6 7 > . . . > 6 7 > . . . > 6 7 . . . 1 < 6 7 U E ¤ 2 ¢ ¢ ¢ u u u n = = k k k i i i 1 2 6 7 k 6 7 > > 6 . . . 7 > . . . > 6 7 . . . > > 6 7 > > 4 5 > > : Spectral Embeddings Eigenvalue decomposition: Scaled eigenvectors: • Outline • Problem Overview • Spectral Embeddings • Basics • Problems • Solutions • Shape Descriptors • Results • Future Work • Acknowledgements Jain, V., Zhang, H.: Robust 3D Shape Correspondence in the Spectral Domain. Proc. Shape Modeling International 2006.

  10. ~ ~ ~ u u u 2 3 4 8 2 3 ¯ ¯ > ¯ ¯ > > > 6 7 ¯ ¯ > > 6 7 > ¯ ¯ . . . > 6 7 > . . . ¯ ¯ > 6 7 . . . < 6 7 U ¯ ¯ u u u n = i i i 2 3 4 3 6 7 ¯ ¯ 6 7 > > 6 7 . . . ¯ ¯ > . . . > 6 7 . . . ¯ ¯ > > 6 7 > ¯ ¯ > 4 5 > > ¯ ¯ : ¯ ¯ ¯ ¯ ¯ ¯ ¯ ¯ ¯ ¯ Spectral Embeddings • Outline • Problem Overview • Spectral Embeddings • Basics • Problems • Solutions • Shape Descriptors • Results • Future Work • Acknowledgements Examples of 3D embeddings

  11. f g ¾ ¾ ¾ k 1 2 ; ; : : : ; 1 ³ ´ 2 k k 2 u i ¾ = i n p ¸ i = r P ¸ i i Spectral Embeddings EigenValue Descriptor (EVD): • Use deviation in projected data as descriptor: • Our shape descriptor: • Outline • Problem Overview • Spectral Embeddings • Basics • Problems • Solutions • Shape Descriptors • Results • Future Work • Acknowledgements

  12. 2 3 b b b 2 3 ¢ ¢ ¢ 1 1 1 2 ¢ ¢ ¢ 1 a a a 1 1 1 2 1 n n 2 R A b ¢ ¢ ¢ = 2 1 ¢ ¢ ¢ a 6 7 2 1 6 7 6 7 T 6 7 R A A . . . 6 7 = . . . 6 7 . . . . . . 6 7 . . . B 6 7 . . . A = 6 7 = b b b 6 7 ¢ ¢ ¢ i i i 1 2 6 7 ¢ ¢ ¢ a a a n i i i 1 2 6 7 n 6 7 R E E ¤ 6 7 = 6 . . . 7 6 . . . 7 . . . 4 5 . . . . . . 4 5 . . . b b b ¢ ¢ ¢ 1 2 n n n n ¢ ¢ ¢ a a a 1 2 n n n n Spectral Embeddings Why use eigenvalues?? • Outline • Problem Overview • Spectral Embeddings • Basics • Problems • Solutions • Shape Descriptors • Results • Future Work • Acknowledgements

  13. 2 2 ( ) ( ) l k O O + n o g n n Spectral Embeddings • Outline • Problem Overview • Spectral Embeddings • Basics • Problems • Solutions • Shape Descriptors • Results • Future Work • Acknowledgements Problems: • Geodesic distance computation • Efficiency of geodesic distance computation & eigendecomposition:

  14. Spectral Embeddings • Outline • Problem Overview • Spectral Embeddings • Basics • Problems • Solutions • Shape Descriptors • Results • Future Work • Acknowledgements Geodesics using Structural Graph: • Add edges to make mesh connected • Geodesic distance ≈ Shortest graph distance Problem: Unwanted (topology modifying) edges! Solution: Add shortest possible edges. Choice of graph to take edges from: • p-nearest neighbor (may not return connected graph) • p-edge connected [Yang 2004]

  15. 2 3 i i e g e n . . m a x - m n l i t t e x r a p o a o n . . 8 2 3 . . d i i t ¯ ¯ ¯ ¯ b l i e c o m p o s o n 6 7 s u s a m p n g ( ) ~ ~ l n O E u ¢ ¢ ¢ u n > = ¯ ¯ ¯ ¯ l l l 1 £ 6 7 3 ( ) l O > ( ) l l O z } | { > 4 5 n o g n ¯ ¯ ¯ ¯ . . . > 6 7 8 2 3 . . > . . . ¯ ¯ ¯ ¯ ¢ ¢ ¢ a a a . . 1 1 1 2 1 > 6 7 n . . . . . < > ¯ ¯ ¯ ¯ 6 7 > ~ ~ ~ E > ¢ ¢ ¢ a u u u n 2 1 = ¯ ¯ ¯ ¯ 2 3 4 3 £ > 6 7 6 7 n > > 6 7 ¯ ¯ ¯ ¯ 6 7 > > . . . > 6 7 ¯ ¯ ¯ ¯ > 6 . . . 7 < . . . > . . . 6 7 . . . A > 4 5 ¯ ¯ ¯ ¯ n . . . = > £ 6 7 n n > ¯ ¯ ¯ ¯ ¢ ¢ ¢ a a a i i i 1 2 6 : 7 n > ¯ ¯ ¯ ¯ > 6 7 > ¯ ¯ ¯ ¯ > 6 . . . 7 > . . . > 4 5 ¯ ¯ ¯ ¯ . . . > > ¯ ¯ ¯ ¯ : ¢ ¢ ¢ a a a 1 2 n n n n ¯ ¯ ¯ ¯ ¯ ¯ ¯ ¯ n z } | { 8 2 3 ¢ ¢ ¢ a a a 1 1 1 2 1 n > > ¢ ¢ ¢ a a a 2 1 2 2 2 < n 6 7 l A = l l £ 6 7 . . . 4 5 . . . > . . . > : ¢ ¢ ¢ a a a l l l 1 2 n Spectral Embeddings • Outline • Problem Overview • Spectral Embeddings • Basics • Problems • Solutions • Shape Descriptors • Results • Future Work • Acknowledgements Efficiency with Nyström approximation

  16. Shape Descriptor

  17. 2 k £ ¤ p ¸ p 1 ¡ ° i i X ( ) S C P Q i t m o s = ; p 2 ¸ p + ° i i i 1 = Shape Descriptor • Outline • Problem Overview • Spectral Embeddings • Shape Descriptors • Results • Future Work • Acknowledgements • Global Shape Descriptors • Light Field Descriptor (LFD) • Spherical Harmonics Descriptor (SHD) • Our Similarity Measure (EVD):

  18. Results

  19. Experimental Database • Outline • Problem Overview • Spectral Embeddings • Shape Descriptors • Results • Future Work • Acknowledgements McGill 3D Articulated Shapes Database http://www.cim.mcgill.ca/~shape/benchMark/

  20. Precision-Recall plot for McGill database Results • Outline • Problem Overview • Spectral Embeddings • Shape Descriptors • Results • Future Work • Acknowledgements

  21. McGill articulated shape database Results • Outline • Problem Overview • Spectral Embeddings • Shape Descriptors • Results • Future Work • Acknowledgements

  22. Limitations & Future Work • Outline • Problem Overview • Spectral Embeddings • Shape Descriptors • Results • Future Work • Acknowledgements • Non-robustness of geodesic distances • Non-robustness to outliers

  23. Acknowledgements • Outline • Problem Overview • Spectral Embeddings • Shape Descriptors • Results • Future Work • Acknowledgements • McGill 3D Shape Benchmark. • Phil Shilane (LFD & SHD implementations).

  24. Thank You! for you attention

More Related