Methods for 3D Shape Matching and Retrieval

1. Methods for 3D Shape Matching and Retrieval Marcin Novotni & Reinhard Klein University of Bonn Computer Graphics Group

2. Our Aim #1 • Given an example: • Find the most similar object(s) in a database … , , , Marcin Novotni  Reinhard Klein University of Bonn  Computer Graphics Group

3. Motivation • Lots of 3D archives: • WWW • Proprietary databases • ... • Search engines for data: • Text, 2D images, music (MIDI), … • Emerging since 1998 for 3D Marcin Novotni  Reinhard Klein University of Bonn  Computer Graphics Group

4. Our Aim #2 • Direct matching • Alignment • Establishing correspondences Marcin Novotni  Reinhard Klein University of Bonn  Computer Graphics Group

5. Motivation • Partial matching/retrieval Marcin Novotni  Reinhard Klein University of Bonn  Computer Graphics Group

6. Motivation • Partial matching/retrieval • Statistical shape analysis • Morphing • Texture transfer • Registration Marcin Novotni  Reinhard Klein University of Bonn  Computer Graphics Group

7. General Problem Abstract representation facilitating: • identification of salient features of 3D objects • description of features • comparison (matching) Marcin Novotni  Reinhard Klein University of Bonn  Computer Graphics Group

8. Overview • Matching for 3D Shape Retrieval • Correspondence Matching Marcin Novotni  Reinhard Klein University of Bonn  Computer Graphics Group

9. Matching for 3D Shape Retrieval Marcin Novotni  Reinhard Klein University of Bonn  Computer Graphics Group

10. General Problem • We need a Descriptor →D( ) D : Marcin Novotni  Reinhard Klein University of Bonn  Computer Graphics Group

11. General Problem We need a Distance Measure : = d( , ) d( , ) D( ) D( ) Marcin Novotni  Reinhard Klein University of Bonn  Computer Graphics Group

12. General Problem • We need a Distance Measure : • Close to (application driven) notion of resemblance • Computationally cheap and robust d( , ) d( , ) d( , ) ≤ ≤ Marcin Novotni  Reinhard Klein University of Bonn  Computer Graphics Group

13. x1 D( ) ≡ xn 3D Zernike Descriptors • Feature vectors Xi: 3D Zernike Descriptors [Canterakis ’99, Novotni & Klein ’03, ’04] Distance Measure: Euclidean Distance Marcin Novotni  Reinhard Klein University of Bonn  Computer Graphics Group

14. 3D Zernike Descriptors Retrieval performance [Novotni & Klein ‘03 ’04] • Slightly better than [Funkhouser et al. ’02] • Object class dependent performance! • Class dependent coefficient importance! Marcin Novotni  Reinhard Klein University of Bonn  Computer Graphics Group

15. 3D Zernike Descriptors Faces Airplanes Chairs Importance Coeff No. (Frequency) Marcin Novotni  Reinhard Klein University of Bonn  Computer Graphics Group

16. 3D Zernike Descriptors • Relevance feedback: • User selects relevant / irrelevant items • Distance measure is tuned • Learning Machines: • SVM (Support vector machines) [Vapnik ‘95] • One class SVM [Schölkopf et al. ’99] • (K)BDA ((Kernel) Biased Discriminant Analysis) [Zhou et al. ‘01] Marcin Novotni  Reinhard Klein University of Bonn  Computer Graphics Group

17. Correspondence Matching Marcin Novotni  Reinhard Klein University of Bonn  Computer Graphics Group

18. Geometric Similarity Estimation • Idea [Novotni & Klein 2001]: • Definition of „geometric“ similarity in terms of a geometric distance • Intuitive, simple, robust. Marcin Novotni  Reinhard Klein University of Bonn  Computer Graphics Group

19. Geometric Similarity Estimation Database objects example Normalized volumetric error 0.00 6.78 8.85 30.29 38.09 67.53 Marcin Novotni  Reinhard Klein University of Bonn  Computer Graphics Group

20. Geometric Similarity Estimation • Classification by user set threshold Marcin Novotni  Reinhard Klein University of Bonn  Computer Graphics Group

21. Geometric Similarity Estimation • Measures deformation magnitude Marcin Novotni  Reinhard Klein University of Bonn  Computer Graphics Group

22. Correspondence Matching ? Marcin Novotni  Reinhard Klein University of Bonn  Computer Graphics Group

23. Correspondence Matching ? Marcin Novotni  Reinhard Klein University of Bonn  Computer Graphics Group

24. Correspondence Matching ? Marcin Novotni  Reinhard Klein University of Bonn  Computer Graphics Group

25. Correspondence Matching ? Marcin Novotni  Reinhard Klein University of Bonn  Computer Graphics Group

26. Correspondence Matching • Ideally: dense mapping ? Marcin Novotni  Reinhard Klein University of Bonn  Computer Graphics Group

27. Correspondence Matching • Ideally: dense mapping • Deformation by mapping semantics [D’Arcy Thompson 1917: On Growth and Form] Marcin Novotni  Reinhard Klein University of Bonn  Computer Graphics Group

28. Correspondence Matching • Ideally: dense mapping • Easier: mapping salient points • Curvature extremes • Corners (Harris points in 2D) • Etc… • Scale space extremes Marcin Novotni  Reinhard Klein University of Bonn  Computer Graphics Group

29. Correspondence Matching • Ideally: dense mapping • Easier: mapping salient points • Curvature extremes • Corners (Harris points in 2D) • Etc… • Scale space extremes Marcin Novotni  Reinhard Klein University of Bonn  Computer Graphics Group

30. Correspondence Matching • Scale Space extremes [Lindeberg ‘94] Marcin Novotni  Reinhard Klein University of Bonn  Computer Graphics Group

31. Correspondence Matching We have: • Salient points • Spatial position • Size of local blobs How to match??? Marcin Novotni  Reinhard Klein University of Bonn  Computer Graphics Group

32. Correspondence Matching Criteria for correspondences: Similar • Local geometries • Constellations of points Marcin Novotni  Reinhard Klein University of Bonn  Computer Graphics Group

33. Correspondence Matching Criteria for correspondences: Similar • Local geometries • Constellations of points Marcin Novotni  Reinhard Klein University of Bonn  Computer Graphics Group

34. Correspondence Matching • Local description Marcin Novotni  Reinhard Klein University of Bonn  Computer Graphics Group

35. Correspondence Matching • Local description Marcin Novotni  Reinhard Klein University of Bonn  Computer Graphics Group

36. Correspondence Matching Assumption: Similar local descriptors Similar local geometries Marcin Novotni  Reinhard Klein University of Bonn  Computer Graphics Group

37. Correspondence Matching Criteria for correspondences: Similar • Local geometries • Constellations of points Marcin Novotni  Reinhard Klein University of Bonn  Computer Graphics Group

38. Correspondence Matching Similar constellations of points • Smooth mappings leave constellations consistent • Idea • Constellations are consistent if mapping is smooth Marcin Novotni  Reinhard Klein University of Bonn  Computer Graphics Group

39. Correspondence Matching Marcin Novotni  Reinhard Klein University of Bonn  Computer Graphics Group

40. Correspondence Matching Marcin Novotni  Reinhard Klein University of Bonn  Computer Graphics Group

41. Correspondence Matching Marcin Novotni  Reinhard Klein University of Bonn  Computer Graphics Group

42. Correspondence Matching Marcin Novotni  Reinhard Klein University of Bonn  Computer Graphics Group

43. Correspondence Matching Marcin Novotni  Reinhard Klein University of Bonn  Computer Graphics Group

44. Correspondence Matching Similar constellations of points • Idea: • Constellations are consistent if mapping is smooth • Thin Plate Spline interpolation [Brookstein ’89]  minimize: Total curvature Marcin Novotni  Reinhard Klein University of Bonn  Computer Graphics Group

45. Correspondence Matching  minimize: Minimizer (Thin Plate Spline interpolator): Affine part Nonlinear deformation Marcin Novotni  Reinhard Klein University of Bonn  Computer Graphics Group

46. Correspondence Matching  minimize: Minimizer (Thin Plate Spline interpolator): 2D Thin Plate Spline Marcin Novotni  Reinhard Klein University of Bonn  Computer Graphics Group

47. Correspondence Matching  minimize: Minimizer (Thin Plate Spline interpolator): Can be computed by a (N+4)x(N+4) matrix inversion Marcin Novotni  Reinhard Klein University of Bonn  Computer Graphics Group

48. Correspondence Matching Find (sub)sets of correspondences: • Small local descriptor distances • Small deformation energy Hierarchical pruning and clustering • Using: • Local descriptors • Geometrical constellation consistency Marcin Novotni  Reinhard Klein University of Bonn  Computer Graphics Group

49. Correspondence Matching Marcin Novotni  Reinhard Klein University of Bonn  Computer Graphics Group

50. Correspondence Matching Marcin Novotni  Reinhard Klein University of Bonn  Computer Graphics Group