Global and efficient self similarity for object classification and detection
Download
1 / 28

Global and Efficient Self-Similarity for Object Classification and Detection - PowerPoint PPT Presentation


  • 89 Views
  • Uploaded on

Global and Efficient Self-Similarity for Object Classification and Detection. Thomas Deselaers and Vittorio Ferrari. CALVIN group Computer Vision Laboratory ETH Zurich Switzerland. CVPR 2010. Conventional Image Descriptors. Measure direct image properties. gradients. colors.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Global and Efficient Self-Similarity for Object Classification and Detection' - olwen


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Global and efficient self similarity for object classification and detection

Global and Efficient Self-Similarity for Object Classification and Detection

Thomas Deselaers and Vittorio Ferrari

CALVIN group

Computer Vision Laboratory

ETH Zurich

Switzerland

CVPR 2010


Conventional image descriptors
Conventional Image Descriptors

Measure direct image properties

gradients

colors


Self similarity vs conventional descriptors
Self-Similarity vs Conventional Descriptors

[Shechtman, Irani CVPR 07]

Assumption of conventional image descriptors

There is a direct visual property shared by images of objects of the same class (e.g. colors, gradients, …).

This property can be used to compare images.

Self-similarity:

Indirect property: geometric layout of repeating patches within an image

More general property


Local self similarity descriptors
Local Self-Similarity Descriptors

[Shechtman, Irani CVPR 07]


Using local self similarity descriptors
Using Local Self-Similarity Descriptors

Applications: object recognition, image retrieval, action recognition

  • Ensemble matching [Shechtman CVPR 07]

  • Nearest neighbor matching [Boiman CVPR 08]

  • Bag of local self-similarities [Gehler ICCV09, Vedaldi ICCV09, Hörster ACMM08, Lampert CVPR09, Chatfield ICCV09 WS]

Compute LSS descriptors for an image

Assign the LSS descriptors to a codebook

Represent the image as a histogram of LSS descriptors


Self similarity goes global
Self-Similarity goes Global

Capture long-range self-similarities and their spatial arrangement


Self similarity goes global1
Self-Similarity goes Global

Capture long-range self-similarities and their spatial arrangement


Global self similarity tensor
Global Self-Similarity Tensor

compute self-similarity between all pairs of pixels

4D self-similarity tensor

Note: local self-similarities included


Problems with the gss tensor
Problems with the GSS Tensor

11

11

300

500

∼ 20h

∼ 80GB

Aim: Reduce both

Computation time:

Memory requirement:


Outline
Outline

  • Efficient global self-similarity tensor

  • Global self-similarity descriptors

    • Bag of correlation surfaces

    • Self-similarity hypercubes

  • Detection with self-similarity hypercubes

    • Efficient sliding window

    • Efficient subwindow search

  • Experiments

    • Global self-similarity better than local self-similarity

    • Complementary to conventional descriptors

    • Object detection possible


Efficient global self similarity tensor
Efficient Global Self-Similarity Tensor

Find an efficient approximation to

If two patches are assigned to the same prototype, they are similar

Quantize patches according to codebook

Reduces runtime to speedup:

750


Efficient global self similarity
Efficient Global Self-Similarity

Two patches are only similar if they are assigned to the same prototype

Reduces memory to reduction:


Patch prototype codebooks
Patch Prototype Codebooks

Remember: Self-similarity encodes image content indirectly

Image-specific codebooks can be smaller than conventional ones

see paper for more generic codebooks and extensive evaluation


Global self similarity descriptors
Global Self-Similarity Descriptors

  • So far:

  • Compact GSS computed efficiently

  • Now:

  • Descriptors that can be used in machine learning classifiers

    • Fixed dimensionality

    • Compact representation

  • Self-similarity hypercubes: now

  • Bag of correlation surfaces: only in the paper


Self similarity hybercubes
Self-Similarity Hybercubes

SSH of size


Sshs for detection
SSHs for Detection

  • Computing SSH naïvely requires operations

  • Sliding windows has to evaluate many windows

operations


Efficient computation of sshs
Efficient Computation of SSHs

Compute integral self-similarity tensor:

can be obtained using 16 lookups in

 160000

operations to compute SSH

for an image window

 ∼5000x speedup


Efficient subwindow search for ssh
Efficient Subwindow Search for SSH

  • Derive an upper bound on the score of a set of windows

  • Section 5.2 in our paper

  • Similar to [Lampert PAMI09]


Experiments object classification
Experiments: Object classification

PASCAL 07 objects

  • 9608 cropped images of objects from PASCAL 07

  • 20 classes

    Task: Classify each test image into one of 20 classes

    Model: Linear SVM

    Train: train+val Test: test


Classification on the pascal 07 objects set
Classification on the PASCAL 07 objects set

classification accuracy [%]

+ GSS outperform LSS

+ Self-Similarity is truly complementary to conventional descriptors


Experiments object detection
Experiments: Object detection

e.g. [Ferrari CVPR07, Maji CVPR09]

ETHZ Shape Classes

  • 255 images

  • 5 classes (apple logos, bottles, giraffes, mugs, swans)

    Task: Detect objects in images

    Detector: Linear SVM, sliding windows


Detection results
Detection Results

DR at FPPI 0.4

}

bottles

giraffes

SSH

+ SSH outperforms BOLSS

+ it is possible to use GSS for detection with good results

apple logos

swans

mugs

DR at 0.5 PASCAL overlap

}

BoLSS

Comparison results (avg):

[Ferrari CVPR07]: 71.9

[Maji CVPR09]: 93.2

… many more

FPPI 0.4


Runtimes for computing descriptors
Runtimes for Computing Descriptors

  • 200x200 image

  • GSS tensor

    • directly: 5512s (∼1.5 hours)

    • using our method: 81s (∼1.5 minutes)

  • Computing descriptors: few seconds

  • Our method: 70x speedup

  • For Reference:

    • GIST: 0.4s

    • BOLSS: 0.7s


Runtimes for detection
Runtimes for Detection

June 2014

Given the prototype assignment map (80s) (once only)

SSH sliding window: 30s/img (once per class)

For Comparison

  • Computing direct GSS tensor for 25000 windows: 4 years/img

    Speedup: ∼1 million

    ⇒ Using our methods, GSS can be used for object detection

    For Reference:

  • Felzenszwalb PAMI 09: 5s.


Global and efficient self similarity for object classification and detection1

Feasible

Global and Efficient Self-Similarity for Object Classification and Detection

Thomas Deselaers and Vittorio Ferrari

CALVIN group

Computer Vision Laboratory

ETH Zurich

Switzerland

CVPR 2010


Conclusion
Conclusion

  • self-similarity should be considered globally

    • Global self-similarity performs better than local self-similarity

  • truly complementary to conventional descriptors

  • global self-similarity is feasible

    • efficient computation of self-similarity

    • two descriptors based on self-similarity

  • global self-similarity for detection

  • code will be available soon


Thank you for your attention

Thank you for your attention

Thomas Deselaers and Vittorio Ferrari

Global and Efficient Self-Similarity for

Object Classification and Detection

Code will be available

http://www.vision.ee.ethz.ch/~calvin


Thank you for your attention1

Thank you for your attention

Thomas Deselaers and Vittorio Ferrari

Global and Efficient Self-Similarity for

Object Classification and Detection

Code will be available

http://www.vision.ee.ethz.ch/~calvin


ad