Detecting Curved Symmetric Parts using a Deformable Disc Model

Tom Sie Ho Lee, University of Toronto Sanja Fidler, TTI Chicago Sven Dickinson, University of Toronto

Overview

Motivation

Robustness to taper

Symmetric part detection as sequence finding

- Recovering an object’s generic part structure is a key step in bottom-up object categorization
- Symmetry has formed the basis of many 2D and 3D generic part representations, e.g. skeletons, shock graphs, generalized cylinders, geons
- Our goal is to detect 2D symmetric parts in a cluttered image

- Good symmetry follows a curvilinear axis
- We find high-affinity sequences of disc hypotheses
- Optimal sequences are computed using dynamic programming
- Our sequence formulation avoids branching clusters

- Tapered parts vary in scale along the axis
- We allow parts to be composed along the axis from disc hypotheses of different scales

Symmetric parts of objects detected by our method

Robustness to curvature

Related work

- Symmetric parts are often curved
- We capture curvature explicitly and recover the part in one piece

- Classical skeletons [Blum ‘67; Brady ‘84] are inapplicable to cluttered scenes
- Filter-based approaches require reliable templates
- Contour-based approaches require quadratic grouping complexity
- Region-based grouping [Levinshtein et al. ‘09] offers a good alternative
- We demonstrate an improvement on [Levinshtein et al.] by capturing more shape variability and applying an optimal algorithm

[ours]

[Levinshtein et al.]

[ours]

[Levinshtein et al.]

[Levinshtein et al.]

[ours]

1. Representing symmetric parts

3. Finding sequences

Results

Deformable discs

Weizmann Horse Database (WHD)

Berkeley Segmentation Database (BSDS)

Disc hypothesis graph

- 81 images of horses
- Manually annotated symmetric parts as groundtruth regions [Levinshtein et al.]
- Count a hit when IoU-overlap > 40% between groundtruth and detected regions

- The medial axis transform [Blum] decomposes a shape into the locus of maximal inscribed discs
- We define a symmetric part as a sequence of deformable discs
- Discs deform to the shape’s boundary while remaining compact

- Organize deformable disc hypotheses into graph
- Place edges between adjacent or overlapping discs
- Find disc sequences in the graph with high affinity

- Source of images of diverse objects on cluttered backgrounds
- We manually annotated symmetric parts on 36 selected images

Object part

Maximal discs

Superpixel approximation

Multi-scale composition

Superpixel approximation

- Use compact superpixels as deformable disc hypotheses
- Superpixels from different scales compose a single part

disc hypothesis graph

Cost of a disc sequence

BSDS

WHD

- Score a sequence P = (d0, …, dn) in terms of local affinities σ(di-1,di) and σ(di-1,di,di+1)
- Affinities favor local grouping of adjacent discs
- Ternary affinities favor curvilinear axis (smoothness)
- Convert affinities into binary costs {si-1,i = 1 - σ(di-1,di)} and ternary costs {ti-1,i,i+1 = 1 - σ(di-1,di,di+1)}

2. Deformable Disc Affinity

Deformable ellipse

Overview of affinity

- Define affinity between adjacent disc hypotheses
- High affinity reflects non-accidental symmetry
- Adjacent discs occupy a region r on which to extract features
- Train affinity on region symmetry features

Qualitative results

Ellipse fitted to a region hypothesis

- Cost is normalized by number of discs in sequence
- Growth term A favors longer sequences

Deformation-invariant space

Our ellipse is parameterized by bending and tapering, axis scaling, and rigid transformations

- Evaluate symmetry in a warped space invariant to bending and tapering deformations
- Determine warp by fitting a deformable ellipse to region
- Extract spatial histogram of boundary edgels
- Extract interior color and texture features

Finding the optimal sequence

boundary edgels

- cost(P) can be globally minimized using dynamic programming
- We use the algorithm of [Felzenszwalb & McAllester ‘06] to compute the global minimum P*

Find parameters w that locally minimize non-linear least squares:

- Perform best-first search using priority queue of candidate sequences
- Dequeue candidate sequences and consider possible extensions
- Repeat minimization to find multiple symmetric parts

Affinity training

- Learn to map a region r to its affinity σ(r)
- Generate positive and negative training regions from annotated dataset
- Extract features on each training region
- Fit logistic regressor σ(r) to training examples

Conclusions

Candidate sequence extensions

under consideration

- Symmetric part detector trained on horse images generalizes to diverse objects
- Symmetry is a powerful and ubiquitous shape regularity

warp W

spatial histogram