visual categorization with bags of keypoints
Download
Skip this Video
Download Presentation
Visual Categorization With Bags of Keypoints

Loading in 2 Seconds...

play fullscreen
1 / 50

Visual Categorization With Bags of Keypoints - PowerPoint PPT Presentation


  • 254 Views
  • Uploaded on

Visual Categorization With Bags of Keypoints. Outline. Introduction Method Experiments Conclusion. Outline. Introduction Visual Categorization Is NOT Expected Goals Bag of Words Analogy Method Experiments Conclusion. Introduction. A method for generic visual categorization.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about ' Visual Categorization With Bags of Keypoints' - enid


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
outline
Outline
  • Introduction
  • Method
  • Experiments
  • Conclusion
outline1
Outline
  • Introduction
    • Visual Categorization Is NOT
    • Expected Goals
    • Bag of WordsAnalogy
  • Method
  • Experiments
  • Conclusion
introduction
Introduction
  • A method for genericvisualcategorization

Face

visual categorization is not
Visual Categorization Is NOT
  • Recognition
    • Concerns the identification of particularobject instances

Prasad

Xinwu

visual categorization is not1
Visual Categorization Is NOT
  • Content based image retrieval
    • Retrieving images on the basis of low-level image features
visual categorization is not2
Visual Categorization Is NOT
  • Detection
    • Decidingwhether or not a member of one visualcategoryispresent in a given image

Face Yes

Cat No

visual categorization is not3
Visual Categorization Is NOT
  • Detection
    • Decidingwhether or not a member of one visualcategoryispresent in a given image
    • « One Visual Category » - soundssimilar
    • Yetmost of the existingdetection techniques require
      • Precisemanualalignment of the training images
      • Segregation of these images intodifferentviews
    • Bagsof Keypointsdon’tneedany of these
expected goals
Expected Goals
  • Shouldbereadilyextendable
  • Shouldhandle the variations in view, imaging, lighting condition, occlusion
  • Shouldhandle intra class variations
bag of words analogy
Bag of WordsAnalogy

Image Credits: Cordelia Schmid

bag of words analogy1
Bag of WordsAnalogy

Image Credits: Li Fei Fei

bag of words analogy2
Bag of WordsAnalogy

Image Credits: Li Fei Fei

bag of words analogy3
Bag of WordsAnalogy
  • Zhu et al – 2002 have usedthismethod for categorizationusingsmall square image windows – calledkeyblocks
  • But keyblocksdon’t posses any invariance propertiesthatBagsof Keypointsposses
outline2
Outline
  • Introduction
  • Method
    • Detection And Description of Image Patches
    • Assignment of Patch Descriptors
    • Contruction of Bag of Keypoints
    • Application of Multi-Class Classifier
  • Experiments
  • Conclusion
method
Method
  • 4 main steps
    • Detection And Description of Image Patches
    • Assignment of Patch Descriptors
    • Contruction of Bag of Keypoints
    • Application of Multi-Class Classifier
      • Categorization by Naive Bayes
      • Categorization by SVM
  • Designed to maximize classification accuracywhileminimizingcomputational effort
detection and description of image patches
Detection And Description of Image Patches
  • Descriptorsshouldbe invariant to variation but have enough information to discriminatedifferentcategories

Image Credits: Li Fei Fei

detection and description of image patches1
Detection And Description of Image Patches
  • Detection – Harris affine detector
    • Last presentation by Guru and Shreyas
  • Description – SIFT descriptor
    • 128 dimensionalvector – 8 * (4*4)
assignment of patch descriptors
Assignment of Patch Descriptors
  • When a new query image isgiven, the deriveddescriptorsshouldbeassigned to onesthat are already in our training dataset
  • Check themwith
    • All the descriptorsavailable in the training dataset – tooexpensive
    • Only a few of them – but not too few
  • The number of descriptorsshouldbecarefullyselected
assignment of patch descriptors1
Assignment of Patch Descriptors
  • Each patch has a descriptor, whichis a point in somehigh-dimensionalspace (128)

Image Credits: K. Grauman, B. Leibe

assignment of patch descriptors2
Assignment of Patch Descriptors
  • Close points in feature space, means similar descriptors, which indicates similar local content

Image Credits: K. Grauman, B. Leibe

assignment of patch descriptors3
Assignment of Patch Descriptors
  • To reduce the hugenumber of descriptorsinvolved (600 000), they are clustered
  • Using K-means

K-meansisrunseveral times usingdifferent K values and initial positions

One with the lowestempiricalriskisused

Image Credits: K. Grauman, B. Leibe

assignment of patch descriptors4
Assignment of Patch Descriptors
  • Now the descriptorspace looks like

Featurespaceisquantized

These cluster centers are the prototype words

Theymake the vocabulary

Image Credits: K. Grauman, B. Leibe

assignment of patch descriptors5
Assignment of Patch Descriptors
  • When a query image comes

Itsdescriptors are attached to the nearest cluster center

That particularwordispresent in the query image

Image Credits: K. Grauman, B. Leibe

assignment of patch descriptors6
Assignment of Patch Descriptors
  • Vocabularyshouldbe
    • Large enough to distinguish relevant changes in the image parts
    • Not so large that noise startsaffecting the categorization
contruction of bags of keypoints
Contruction of Bagsof Keypoints
  • Summarize entire image based on its distribution (histogram) of word occurrences

Image Credits: Li Fei Fei

application of multi class classifier
Application of Multi-Class Classifier
  • Apply a multi-class classifier, treat the bag of keypoints as the featurevector, thusdeterminewhichcategory or categories to assign to the image
    • Categorization by Naive Bayes
    • Categorization by SVM
categorization by naive bayes
Categorization by Naive Bayes
  • Can be viewed as the maximum a posteriori probability classifier for a generative model
  • To avoidzeroprobabilities of , Laplace smoothingisused
categorization by svm
Categorization by SVM
  • Find a hyperplane which separates two-class data with maximal margin
categorization by svm1
Categorization by SVM
  • Classification function:

f(x) = sign(wTx+b) where w, b parameters of the hyperplane

categorization by svm2
Categorization by SVM
  • Data sets not always linearly separable
    • error weighting constant to penalizes misclassification of samples in proportion to their distance from the classification boundary
    • A mapping φis made from the original data space of X to another feature space

This isused in Bag of Keypoints

categorization by svm3
Categorization by SVM
  • What do youmean by mappingfunction?
categorization by svm4
Categorization by SVM
  • Can be formulated in terms of scalar products in the second feature space, by introducing the kernel
  • Then the decision function becomes
outline3
Outline
  • Introduction
  • Method
  • Experiments
  • Conclusion
experiments
Experiments
  • Somesamplesfrom the inhousedataset
experiments1
Experiments
  • Impact of the number of clusters on classifier accuracy and evaluate the performance of
    • Naive Bayes classifier
    • SVM
  • Three performance measures are used
    • Confusion matrix
    • Overallerror rate
    • Meanranks
results1
Results
  • For K = 1000
    • Naive Bayes 28% SVM 15%
experiments2
Experiments
  • Performance of SVM in anotherdataset
results3
Results
  • Multiple objects of the samecategory/ partial view
  • Misclassifications
outline4
Outline
  • Introduction
  • Method
  • Experiments
  • Conclusion
    • Future Work
conclusion
Conclusion
  • Advantages
    • Bag of Keypointsis simple
    • Computationally efficient
    • Invariant to affine transformations, occlusions, lighting, intra-class variations
future work
Future Work
  • Extend to more visualcategories
  • Extend the categorizer to incorporategeometric information
  • Make the methodrobustwhen the object of interestisoccupyingonly a small fraction of the image
  • Investigatemany alternatives for each of the four steps of the basic method
slide44
Q&A

Thank You

ad