1 / 45

Object Recognition Based on Shape Similarity

Object Recognition Based on Shape Similarity. Longin Jan Latecki Computer and Information Sciences Dept. Temple Univ., latecki@temple.edu Collaborators: Zygmunt Pizlo, Psychological Sciences Dept., Purdue Univ., Nagesh Adluru, Suzan Köknar-Tezel, Rolf Lakaemper, Thomas Young, Temple Univ.,

rochelp
Download Presentation

Object Recognition Based on Shape Similarity

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Object Recognition Based on Shape Similarity Longin Jan Latecki Computer and Information Sciences Dept. Temple Univ., latecki@temple.edu Collaborators: Zygmunt Pizlo, Psychological Sciences Dept., Purdue Univ., Nagesh Adluru, Suzan Köknar-Tezel,Rolf Lakaemper, Thomas Young, Temple Univ., Xiang Bai, Huazhong Univ. of Sci. & Tech. Wuhan, China

  2. Object Recognition Process: Source: 2D image of a 3D object Object Segmentation Contour Extraction Contour Cleaning, e.g., Evolution Contour Segmentation Matching: Correspondence of Visual Parts

  3. Motivation • Once a significant visual part is recognized the whole recognition process is strongly constrained in possible top-down object models. • (H1)object recognition is preceded by, and based on recognition of visual parts. • (H2): contour extraction is driven by shape similarity to a known shape.

  4. What do you see?

  5. With grouping constraints we can see (i.e., recognize the object).

  6. Object contours • Psychophysical and neurophysiological studies provide an abundance of evidence that contours of objects are extracted in early processing stages of human visual perception. • Contours play a central role in the Gestalt-theory.

  7. Salient visual parts can influence the object recognition (Singh and Hoffman 2001)

  8. Salient visual parts can influence the object recognition (Singh and Hoffman 2001)

  9. Salient visual parts can influence the object recognition (Singh and Hoffman 2001)

  10. Visual parts and shape similarity • (H1)object recognition is preceded by, and based on shape recognition of visual parts. • (H2): contour extraction is driven by shape similarity to a known shape.becomes: • (H2) Contour extraction is based on grouping of contour parts to larger contour parts with grouping assignments driven by shape familiarity.

  11. Contour detection is a difficult inverse problem • A given image could be produced by infinitely many possible 3D scenes. In order to produce a unique, stable and accurate interpretation, the visual system must use a priori constraints (see Pizlo, 2001 for a review). • The solution is obtained by optimizing a cost function which consists of two general terms: 1. how close the solution is to the visual data 2. how well the constraints are satisfied

  12. Partial shape similarity

  13. Partial shape similarity Given only a part (of a shape), find similar shapes • (1) length problem, • (2) scale problem, • (3) distortion problem Query Shape Target Shape Target Shape

  14. Partial Shape Similarity • We reduce partial shape similarity to subsequence matching: • This is done by computing a curvature like value at every contour point. • We do this for complete contours of known objects in our database • and for query contour parts extracted from edge images

  15. Subsequence Matching (shape similarity) Database contours Query contours

  16. Motivation for subsequence matching The top (red) and bottom (blue) sequences represent parts of contours of two different but very similar bone shapes

  17. Motivation(2) Example sequences: a = {1, 2, 8, 6, 8} b = {1, 2, 9, 15, 3, 5, 9}

  18. OSB Algorithm • Goal: given two real-valued sequences a and b, find subsequences a’ of a and b’ of b such that a’ best matches b’ • Possible to skip elements in both a and b • The ability to exclude outliers • Preserve the order of the elements • A one-to-one correspondence

  19. OSB Algorithm (2) • Create a dissimilarity matrix • No restrictions on the distance function d • We used d(ai,bj) = (ai – bj)2 • To find the optimal correspondence, use a shortest path algorithm on a DAG

  20. OSB Algorithm (3) • The nodes of the DAG are all the index pairs of the matrix: (i,j){1,…,m}{1,…,n} • The edge weights w are defined by • C is the jump cost (the penalty for skipping an element)

  21. OSB Algorithm (4) • The edge cost may be extended to impose a warping window • Set a maximal value for k – i – 1 and l – j - 1 • This definition of the edge weights is our main contribution

  22. A Simple Example a = {1, 2, 8, 6, 8} b = {1, 2, 9, 15, 3, 5, 9} The dissimilarity matrix d(ai,bj) = (ai – bj)2

  23. Key (indices) elements distance (1,1) 1 - 1 0 (1,2) 1 - 2 1 (1,3) 1 - 9 64 (1,4) 1 - 15 196 (1,5) 1 - 3 4 (1,6) 1 - 5 16 (1,7) 1 - 9 64 (2,1) 2 - 1 1 (2,2) 2 - 2 0 (2,3) 2 - 9 49 (2,4) 2 - 15 169 (2,5) 2 - 3 1 (2,6) 2 - 5 9 (2,7) 2 - 9 49 . . . ... ... ... (3,1) 8 - 1 49 (3,2) 8 - 2 36 (3,3) 8 - 9 1 (3,4) 8 - 15 49 (3,5) 8 - 3 25 (3,6) 8 - 5 9 (3,7) 8 - 9 1 . . . . . . . . . (4,1) 6 - 1 25 (4,2) 6 - 2 16 (4,3) 6 - 9 9 (4,4) 6 - 15 81 (4,5) 6 - 3 9 (4,6) 6 - 5 1 (4,7) 6 - 9 9 . . . (5,1) 8 - 1 49 (5,2) 8 - 2 36 (5,3) 8 - 9 1 (5,4) 8 - 15 49 (5,5) 8 - 3 25 (5,6) 8 - 5 9 (5,7) 8 - 9 1 The DAG

  24. Experimental results on MPEG 7 dataset, 1400 targets in 70 classes

  25. How to find contour parts in images? • Humans group contours automatically • An adaptive, probabilistic process to perform grouping • All shapes contain local symmetry  exploit local symmetry

  26. Shape model

  27. Play movie

  28. Contour Grouping as Robot Mapping • Rao-Blackwellized particle filter is an adaptive, probabilistic approach • Frequently utilized in SLAM approaches to Robot Mapping • Each particle’s successor is its most likely successor • Particles are resampled to eliminate poorly performing particles

  29. Traversal space generated as discrete “center points”

  30. Center Points • Center points act as center points for maximal radius disk between the two sample points • Full set of center points gives full set of maximal radius disks • Entire set of potential skeletal points • Want to generate a skeletal path that best groups the segments for a given shape model

  31. Center points and particle paths

  32. Shape model • System needs to utilize reference model • Some a priori knowledge to discover the proper shape • Model is a sequence of radii at sample skeleton points • Position in reference model determined by triangulation

  33. Contour Smoothness • Smoothness as a criterion for segment selection • Smoothness is the measure of the amount of turn and the distance between segments • Use least sum of distance to determine both distance and the segment pairing • Smoothness as Gaussian mixture of distance and angle

  34. SLAM framework • Obtain particles by sampling from the maximum posterior probability • x is the path traversal • m is the contour grouping model • z is the observations • u is the reference model

  35. Particle filter • Sampling: The next generation of particles x(i)t is obtained from • the current generation x(i)t-1 by sampling from a proposal distribution for

  36. 2) Importance Weighting: An individual importance weight w(i) • is assigned to each particle, according to

  37. log(M(c2)) – log of pdf that a given pixel is a center point of radius 10

  38. log(M(c2)) – log of pdf that a given pixel is a center point of radius 10

  39. 3) Resampling: Particles with a low importance weight w(i) are typically replaced by samples with a high weight. Residual resampling was used 4) Contour Estimating: For each pose sample x(i)t, the corresponding contour estimate m(i)t is computed based on the trajectory and the history of observations according to

  40. Results • Grouping performed on several pictures • Useful groupings on many images • Little or no noise grouped • Few structural particles missed

  41. Future Work • Integration of the shape similarity of parts and the contour grouping • Learning good contour parts • Further improvements to the contour grouping to make it more robust

More Related