learning jigsaws for clustering appearance and shape l.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
Learning Jigsaws for clustering appearance and shape PowerPoint Presentation
Download Presentation
Learning Jigsaws for clustering appearance and shape

Loading in 2 Seconds...

play fullscreen
1 / 26

Learning Jigsaws for clustering appearance and shape - PowerPoint PPT Presentation


  • 238 Views
  • Uploaded on

Learning Jigsaws for clustering appearance and shape. Anitha Kannan, John Winn and Carsten Rother. NIPS 2006. Learning jigsaws. Aim: Cluster regions in images with similar appearance and shape . Examples of clusters (jigsaw pieces). Eye. Cheek. Noses. Eyebrows. Road map.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Learning Jigsaws for clustering appearance and shape' - sandra_john


Download Now An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
learning jigsaws for clustering appearance and shape

Learning Jigsawsfor clustering appearance and shape

Anitha Kannan, John Winn and Carsten Rother

NIPS 2006

learning jigsaws
Learning jigsaws

Aim: Cluster regions in images with similar appearance and shape.

Examples of clusters (jigsaw pieces)

Eye

Cheek

Noses

Eyebrows

road map
Road map
  • Clustering image patches
  • The Jigsaw model
  • Results on toy and real images
  • Learning jigsaw pieces
  • Discussion and conclusions
clustering image patches
Clustering image patches

Patches

Clusters

[Leibe & Schiele, BMVC 2003]

clustering image patches5
Clustering image patches

Cluster?

Patch includes background

clustering image patches6
Clustering image patches

Cluster?

Patch wrong shape

clustering image patches7
Clustering image patches

Cluster?

Part is occluded

clustering image patches8
Clustering image patches

Cluster?

Need to adapt the patch shape depending on the image.

road map9
Road map
  • Clustering image patches
  • The Jigsaw model
  • Results on toy and real images
  • Learning jigsaw pieces
  • Discussion and conclusions
aims of jigsaw model
Aims of jigsaw model

Learn clusters (jigsaw pieces) so that:

Clustered patches have similar shape and appearance

Patches are as large as possible

Every image pixel belongs to exactly one patch (i.e. the images are segmented into patches)

the jigsaw model
The Jigsaw model

Jigsaw J

Region of constant offset

...

Image

Offset map

Image

Offset map

Image

Offset map

I

L

I

L

I

L

1

1

2

2

N

N

the jigsaw model12
The Jigsaw model

Jigsaw

Jigsaw J

Mean μ(z) and inverse variance λ(z) for each jigsaw pixel z.

Appearance model

offset at pixel i

Offset map prior (Potts model)

Image

Offset map

I

L

cost of patch boundary

road map13
Road map
  • Clustering image patches
  • The Jigsaw model
  • Results on toy and real images
  • Learning jigsaw pieces
  • Discussion and conclusions
toy example
Toy example

Image

with segmentation

Jigsaw

Mean

Variance

Learned by iteratively maximising joint probability w.r.t. jigsaw and offset maps

(see paper for details)

comparison to epitome model
Comparison to epitome model

[Jojic et al., ICCV 2003]

Jigsaw

Epitome

  • data-driven patch shape
  • non-overlapping patches
  • fixed patch shape
  • overlapping patches
faces example
Faces example

Face images

with segmentations

Jigsaw

128128 mean

Source: Olivetti face database

road map17
Road map
  • Clustering image patches
  • The Jigsaw model
  • Results on toy and real images
  • Learning jigsaw pieces
  • Discussion and conclusions
learning the jigsaw pieces
Learning the jigsaw pieces

Jigsaw J

...

Image

Offset map

Image

Offset map

Image

Offset map

I

L

I

L

I

L

1

1

2

2

N

N

learning the jigsaw pieces19
Learning the jigsaw pieces

Jigsaw J

...

Image

Offset map

Image

Offset map

Image

Offset map

I

L

I

L

I

L

1

1

2

2

N

N

learning the jigsaw pieces20
Learning the jigsaw pieces

Jigsaw J

...

Image

Offset map

Image

Offset map

Image

Offset map

I

L

I

L

I

L

1

1

2

2

N

N

shape clustering on faces
Shape clustering on faces

Commonly used pieces

Jigsaw showing pieces

road map22
Road map
  • Clustering image patches
  • The Jigsaw model
  • Results on toy and real images
  • Learning jigsaw pieces
  • Discussion and conclusions
jigsaw applications
Jigsaw applications
  • Can be used as ‘plug-and-play’ replacement for fixed-shape patch model in existing systems.
  • Applications include:
    • Object recognition/detection
    • Object segmentation
    • Stereo matching
    • Texture synthesis
    • Super-resolution
    • Motion segmentation
    • Image/video compression
jigsaw extensions
Jigsaw extensions
  • Allows rotation/scaling/deformationof the patches.
  • Incorporating shape clustering into the probabilistic model
  • Incorporating additional invariances e.g. to illumination
  • Apply to other domains: audio, biology
conclusions
Conclusions
  • Jigsaw model allows learning the shape and appearance of recurring regions in images.
  • Jigsaw performs unsupervised discovery of object parts.
  • Future work: apply jigsaw to use shape as well as appearance for object recognition and other vision applications.
slide26

Thank you

Jigsaw paper (compressed)