Michael M. Bronstein

1. Extrinsic and intrinsic similarity of shapes nonrigid Michael M. Bronstein Department of Computer Science Technion – Israel Institute of Technology cs.technion.ac.il/~mbron Technion 1 January 2008

2. Collaborators Alexander Bronstein Ron Kimmel

3. Welcome to nonrigid world!

4. Applications ? CORRESPONDENCE SIMILARITY

5. Rock, scissors, paper Rock Scissors Paper

6. Rock, scissors, paper Hands Rock Scissors Paper

7. Extrinsic vs. intrinsic EXTRINSIC SIMILARITY INTRINSIC SIMILARITY • Are the shapes isometric? • Invariance to inelastic deformations • Are the shapes congruent? • Invariance to rigid motion

8. Metric model Shape = metric space Similarity = isometry EXTRINSIC SIMILARITY INTRINSIC SIMILARITY • Euclidean metric • Isometry = rigid motion • Geodesic metric • Isometry = inelastic deformation

9. Extrinsic similarity – Iterative closest point (ICP) Find the best rigid alignment of two shapes Hausdorff distance In Euclidean space Chen & Medioni, 1991; Besl & McKay, PAMI 1992

10. Extrinsic similarity – limitations EXTRINSICALLY SIMILAR EXTRINSICALLY DISSIMILAR Suitable for nearly rigid shapes Unsuitable for nonrigid shapes

11. Canonical forms Multidimensional scaling (MDS) Isometric embedding A. Elad, R. Kimmel, CVPR 2001

12. Intrinsic similarity – canonical forms ? INTRINSIC SIMILARITY Compute canonical forms EXTRINSIC SIMILARITY OF CANONICAL FORMS = INTRINSIC SIMILARITY OF SHAPES A. Elad, R. Kimmel, CVPR 2001

13. Intrinsically similar Intrinsically dissimilar Intrinsic similarity – limitations Suitable for near-isometric shape deformations Unsuitable for deformations modifying shape topology

14. Extrinsically similar Intrinsically dissimilar Extrinsically dissimilar Intrinsically similar Extrinsically dissimilar Intrinsically dissimilar Desired result: THIS IS THE SAME SHAPE! A. Bronstein, M. Bronstein, R. Kimmel, ICCV 2007

15. Joint extrinsic/intrinsic similarity ? DEFORM X TO MATCH Y EXTRINSICALLY CONSTRAIN THE DEFORMATION TO BE AS ISOMETRIC AS POSSIBLE A. Bronstein, M. Bronstein, R. Kimmel, ICCV 2007

16. Glove fitting example Misfit = Extrinsic dissimilarity Stretching = Intrinsic dissimilarity A. Bronstein, M. Bronstein, R. Kimmel, ICCV 2007

17. If it doesn’t fit, you must acquit! Image: Associated Press

18. ? Extrinsic dissimilarity Intrinsic dissimilarity A. Bronstein, M. Bronstein, R. Kimmel, ICCV 2007

19. Computation of the joint similarity • Optimization variable: the deformed shape vertex coordinates • Assuming has the connectivity of • Split into computation of and • Gradients w.r.t. are required for optimization A. Bronstein, M. Bronstein, R. Kimmel, ICCV 2007

20. Computation of the extrinsic term • Find and fix correspondence between current and • Can be e.g. the closest points • Compute an L2 variant of a one-sided Hausdorff distance and its gradient • Similar in spirit to ICP A. Bronstein, M. Bronstein, R. Kimmel, ICCV 2007

21. Computation of the intrinsic term • Fix trivial correspondence between and • Compute L2 distortion of geodesic distances and gradient • is a fixed matrix of all pair-wise geodesic distances on • Can be precomputed using Dijkstra’s algorithm or fast marching A. Bronstein, M. Bronstein, R. Kimmel, ICCV 2007

22. Computation of the intrinsic term • is function of the optimization variables and needs to be recomputed • First option: modify the Dijkstra’s algorithm or fast marching to compute the gradient in addition to the distance itself • Second option: compute and fix the path of the geodesic • is a matrix of Euclidean distances between adjacent vertices • is a linear operator integrating the path length along fixed path A. Bronstein, M. Bronstein, R. Kimmel, ICCV 2007

23. 1 2 3 4 1 Computation of the joint similarity • Alternating minimization algorithm Compute corresponding points Compute shortest paths and assemble Update to sufficiently decrease If change is small, stop; otherwise, go to Step A. Bronstein, M. Bronstein, R. Kimmel, ICCV 2007

24. Numerical example – dataset = topology change Data: tosca.cs.technion.ac.il

25. Numerical example – intrinsic similarity no topological changes

26. Numerical example – intrinsic similarity Insensitive to strong deformations Sensitive to topological changes = topology-preserving = topology change

27. Numerical example – extrinsic similarity Sensitive to strong deformations Insensitive to topological changes = topology-preserving = topology change

28. Numerical example – joint similarity Insensitive to topological changes... …and to strong deformations = topology-preserving = topology change

29. Numerical example – ROC curves 100 Extrinsic EER=10.3% Joint Intrinsic EER=1.6% 10 EER=7.7% Intrinsic, no topological changes False rejection rate (FRR), % 1 EER=1.1% 0.1 0.1 1 10 100 False acceptance rate (FAR), %

30. Set-valued joint similarity Dissimilar Extrinsic dissimilarity Similar Intrinsic dissimilarity

31. Shape morphing Stronger intrinsic similarity (larger λ) Stronger extrinsic similarity (smaller λ)

32. Conclusion • Extrinsic similarity is insensitive to topology changes, but sensitive to nonrigid deformations • Intrinsic similarity is insensitive to nearly-isometric nonrigid deformations, but sensitive to topology changes • Joint similarity is insensitive to both nonrigid deformations and topology changes • Can be thought of as nonrigid ICP • Can be used to produce as isometric as possiblemorphs

33. Open issues • Efficient minimization (good initialization, multiresolution) • Only topology of one shape can change: topology of Z = topology of X • Mesh validity not enforced: self intersections may occur (may be important in computer graphics applications)

34. Shameless advertisement • Published by Springer • To appear in early 2008 • ~350 pages • Over 50 illustrations • Color figures Additional information tosca.cs.technion.ac.il

35. Workshop on Nonrigid Shape Analysis and Deformable Image Alignment (NORDIA) June 2008, Anchorage, Alaska in conjunction with CVPR’08