visual categorization with bags of keypoints n.
Skip this Video
Loading SlideShow in 5 Seconds..
Visual Categorization With Bags of Keypoints PowerPoint Presentation
Download Presentation
Visual Categorization With Bags of Keypoints

Loading in 2 Seconds...

  share
play fullscreen
1 / 50
Download Presentation

Visual Categorization With Bags of Keypoints - PowerPoint PPT Presentation

enid
282 Views
Download Presentation

Visual Categorization With Bags of Keypoints

- - - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript

  1. Visual CategorizationWithBags of Keypoints

  2. Outline • Introduction • Method • Experiments • Conclusion

  3. Outline • Introduction • Visual Categorization Is NOT • Expected Goals • Bag of WordsAnalogy • Method • Experiments • Conclusion

  4. Introduction • A method for genericvisualcategorization Face

  5. Visual Categorization Is NOT • Recognition • Concerns the identification of particularobject instances Prasad Xinwu

  6. Visual Categorization Is NOT • Content based image retrieval • Retrieving images on the basis of low-level image features

  7. Visual Categorization Is NOT • Detection • Decidingwhether or not a member of one visualcategoryispresent in a given image Face Yes Cat No

  8. Visual Categorization Is NOT • Detection • Decidingwhether or not a member of one visualcategoryispresent in a given image • « One Visual Category » - soundssimilar • Yetmost of the existingdetection techniques require • Precisemanualalignment of the training images • Segregation of these images intodifferentviews • Bagsof Keypointsdon’tneedany of these

  9. Expected Goals • Shouldbereadilyextendable • Shouldhandle the variations in view, imaging, lighting condition, occlusion • Shouldhandle intra class variations

  10. Bag of WordsAnalogy Image Credits: Cordelia Schmid

  11. Bag of WordsAnalogy Image Credits: Li Fei Fei

  12. Bag of WordsAnalogy Image Credits: Li Fei Fei

  13. Bag of WordsAnalogy • Zhu et al – 2002 have usedthismethod for categorizationusingsmall square image windows – calledkeyblocks • But keyblocksdon’t posses any invariance propertiesthatBagsof Keypointsposses

  14. Outline • Introduction • Method • Detection And Description of Image Patches • Assignment of Patch Descriptors • Contruction of Bag of Keypoints • Application of Multi-Class Classifier • Experiments • Conclusion

  15. Method • 4 main steps • Detection And Description of Image Patches • Assignment of Patch Descriptors • Contruction of Bag of Keypoints • Application of Multi-Class Classifier • Categorization by Naive Bayes • Categorization by SVM • Designed to maximize classification accuracywhileminimizingcomputational effort

  16. Detection And Description of Image Patches • Descriptorsshouldbe invariant to variation but have enough information to discriminatedifferentcategories Image Credits: Li Fei Fei

  17. Detection And Description of Image Patches • Detection – Harris affine detector • Last presentation by Guru and Shreyas • Description – SIFT descriptor • 128 dimensionalvector – 8 * (4*4)

  18. Assignment of Patch Descriptors • When a new query image isgiven, the deriveddescriptorsshouldbeassigned to onesthat are already in our training dataset • Check themwith • All the descriptorsavailable in the training dataset – tooexpensive • Only a few of them – but not too few • The number of descriptorsshouldbecarefullyselected

  19. Assignment of Patch Descriptors • Each patch has a descriptor, whichis a point in somehigh-dimensionalspace (128) Image Credits: K. Grauman, B. Leibe

  20. Assignment of Patch Descriptors • Close points in feature space, means similar descriptors, which indicates similar local content Image Credits: K. Grauman, B. Leibe

  21. Assignment of Patch Descriptors • To reduce the hugenumber of descriptorsinvolved (600 000), they are clustered • Using K-means K-meansisrunseveral times usingdifferent K values and initial positions One with the lowestempiricalriskisused Image Credits: K. Grauman, B. Leibe

  22. Assignment of Patch Descriptors • Now the descriptorspace looks like Featurespaceisquantized These cluster centers are the prototype words Theymake the vocabulary Image Credits: K. Grauman, B. Leibe

  23. Assignment of Patch Descriptors • When a query image comes Itsdescriptors are attached to the nearest cluster center That particularwordispresent in the query image Image Credits: K. Grauman, B. Leibe

  24. Assignment of Patch Descriptors • Vocabularyshouldbe • Large enough to distinguish relevant changes in the image parts • Not so large that noise startsaffecting the categorization

  25. Contruction of Bagsof Keypoints • Summarize entire image based on its distribution (histogram) of word occurrences Image Credits: Li Fei Fei

  26. Application of Multi-Class Classifier • Apply a multi-class classifier, treat the bag of keypoints as the featurevector, thusdeterminewhichcategory or categories to assign to the image • Categorization by Naive Bayes • Categorization by SVM

  27. Categorization by Naive Bayes • Can be viewed as the maximum a posteriori probability classifier for a generative model • To avoidzeroprobabilities of , Laplace smoothingisused

  28. Categorization by SVM • Find a hyperplane which separates two-class data with maximal margin

  29. Categorization by SVM • Classification function: f(x) = sign(wTx+b) where w, b parameters of the hyperplane

  30. Categorization by SVM • Data sets not always linearly separable • error weighting constant to penalizes misclassification of samples in proportion to their distance from the classification boundary • A mapping φis made from the original data space of X to another feature space This isused in Bag of Keypoints

  31. Categorization by SVM • What do youmean by mappingfunction?

  32. Categorization by SVM • Can be formulated in terms of scalar products in the second feature space, by introducing the kernel • Then the decision function becomes

  33. Outline • Introduction • Method • Experiments • Conclusion

  34. Experiments • Somesamplesfrom the inhousedataset

  35. Experiments • Impact of the number of clusters on classifier accuracy and evaluate the performance of • Naive Bayes classifier • SVM • Three performance measures are used • Confusion matrix • Overallerror rate • Meanranks

  36. Results

  37. Results • For K = 1000 • Naive Bayes 28% SVM 15%

  38. Experiments • Performance of SVM in anotherdataset

  39. Results

  40. Results • Multiple objects of the samecategory/ partial view • Misclassifications

  41. Outline • Introduction • Method • Experiments • Conclusion • Future Work

  42. Conclusion • Advantages • Bag of Keypointsis simple • Computationally efficient • Invariant to affine transformations, occlusions, lighting, intra-class variations

  43. Future Work • Extend to more visualcategories • Extend the categorizer to incorporategeometric information • Make the methodrobustwhen the object of interestisoccupyingonly a small fraction of the image • Investigatemany alternatives for each of the four steps of the basic method

  44. Q&A Thank You