1 / 50

Visual Categorization With Bags of Keypoints

Visual Categorization With Bags of Keypoints. Outline. Introduction Method Experiments Conclusion. Outline. Introduction Visual Categorization Is NOT Expected Goals Bag of Words Analogy Method Experiments Conclusion. Introduction. A method for generic visual categorization.

enid
Download Presentation

Visual Categorization With Bags of Keypoints

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Visual CategorizationWithBags of Keypoints

  2. Outline • Introduction • Method • Experiments • Conclusion

  3. Outline • Introduction • Visual Categorization Is NOT • Expected Goals • Bag of WordsAnalogy • Method • Experiments • Conclusion

  4. Introduction • A method for genericvisualcategorization Face

  5. Visual Categorization Is NOT • Recognition • Concerns the identification of particularobject instances Prasad Xinwu

  6. Visual Categorization Is NOT • Content based image retrieval • Retrieving images on the basis of low-level image features

  7. Visual Categorization Is NOT • Detection • Decidingwhether or not a member of one visualcategoryispresent in a given image Face Yes Cat No

  8. Visual Categorization Is NOT • Detection • Decidingwhether or not a member of one visualcategoryispresent in a given image • « One Visual Category » - soundssimilar • Yetmost of the existingdetection techniques require • Precisemanualalignment of the training images • Segregation of these images intodifferentviews • Bagsof Keypointsdon’tneedany of these

  9. Expected Goals • Shouldbereadilyextendable • Shouldhandle the variations in view, imaging, lighting condition, occlusion • Shouldhandle intra class variations

  10. Bag of WordsAnalogy Image Credits: Cordelia Schmid

  11. Bag of WordsAnalogy Image Credits: Li Fei Fei

  12. Bag of WordsAnalogy Image Credits: Li Fei Fei

  13. Bag of WordsAnalogy • Zhu et al – 2002 have usedthismethod for categorizationusingsmall square image windows – calledkeyblocks • But keyblocksdon’t posses any invariance propertiesthatBagsof Keypointsposses

  14. Outline • Introduction • Method • Detection And Description of Image Patches • Assignment of Patch Descriptors • Contruction of Bag of Keypoints • Application of Multi-Class Classifier • Experiments • Conclusion

  15. Method • 4 main steps • Detection And Description of Image Patches • Assignment of Patch Descriptors • Contruction of Bag of Keypoints • Application of Multi-Class Classifier • Categorization by Naive Bayes • Categorization by SVM • Designed to maximize classification accuracywhileminimizingcomputational effort

  16. Detection And Description of Image Patches • Descriptorsshouldbe invariant to variation but have enough information to discriminatedifferentcategories Image Credits: Li Fei Fei

  17. Detection And Description of Image Patches • Detection – Harris affine detector • Last presentation by Guru and Shreyas • Description – SIFT descriptor • 128 dimensionalvector – 8 * (4*4)

  18. Assignment of Patch Descriptors • When a new query image isgiven, the deriveddescriptorsshouldbeassigned to onesthat are already in our training dataset • Check themwith • All the descriptorsavailable in the training dataset – tooexpensive • Only a few of them – but not too few • The number of descriptorsshouldbecarefullyselected

  19. Assignment of Patch Descriptors • Each patch has a descriptor, whichis a point in somehigh-dimensionalspace (128) Image Credits: K. Grauman, B. Leibe

  20. Assignment of Patch Descriptors • Close points in feature space, means similar descriptors, which indicates similar local content Image Credits: K. Grauman, B. Leibe

  21. Assignment of Patch Descriptors • To reduce the hugenumber of descriptorsinvolved (600 000), they are clustered • Using K-means K-meansisrunseveral times usingdifferent K values and initial positions One with the lowestempiricalriskisused Image Credits: K. Grauman, B. Leibe

  22. Assignment of Patch Descriptors • Now the descriptorspace looks like Featurespaceisquantized These cluster centers are the prototype words Theymake the vocabulary Image Credits: K. Grauman, B. Leibe

  23. Assignment of Patch Descriptors • When a query image comes Itsdescriptors are attached to the nearest cluster center That particularwordispresent in the query image Image Credits: K. Grauman, B. Leibe

  24. Assignment of Patch Descriptors • Vocabularyshouldbe • Large enough to distinguish relevant changes in the image parts • Not so large that noise startsaffecting the categorization

  25. Contruction of Bagsof Keypoints • Summarize entire image based on its distribution (histogram) of word occurrences Image Credits: Li Fei Fei

  26. Application of Multi-Class Classifier • Apply a multi-class classifier, treat the bag of keypoints as the featurevector, thusdeterminewhichcategory or categories to assign to the image • Categorization by Naive Bayes • Categorization by SVM

  27. Categorization by Naive Bayes • Can be viewed as the maximum a posteriori probability classifier for a generative model • To avoidzeroprobabilities of , Laplace smoothingisused

  28. Categorization by SVM • Find a hyperplane which separates two-class data with maximal margin

  29. Categorization by SVM • Classification function: f(x) = sign(wTx+b) where w, b parameters of the hyperplane

  30. Categorization by SVM • Data sets not always linearly separable • error weighting constant to penalizes misclassification of samples in proportion to their distance from the classification boundary • A mapping φis made from the original data space of X to another feature space This isused in Bag of Keypoints

  31. Categorization by SVM • What do youmean by mappingfunction?

  32. Categorization by SVM • Can be formulated in terms of scalar products in the second feature space, by introducing the kernel • Then the decision function becomes

  33. Outline • Introduction • Method • Experiments • Conclusion

  34. Experiments • Somesamplesfrom the inhousedataset

  35. Experiments • Impact of the number of clusters on classifier accuracy and evaluate the performance of • Naive Bayes classifier • SVM • Three performance measures are used • Confusion matrix • Overallerror rate • Meanranks

  36. Results

  37. Results • For K = 1000 • Naive Bayes 28% SVM 15%

  38. Experiments • Performance of SVM in anotherdataset

  39. Results

  40. Results • Multiple objects of the samecategory/ partial view • Misclassifications

  41. Outline • Introduction • Method • Experiments • Conclusion • Future Work

  42. Conclusion • Advantages • Bag of Keypointsis simple • Computationally efficient • Invariant to affine transformations, occlusions, lighting, intra-class variations

  43. Future Work • Extend to more visualcategories • Extend the categorizer to incorporategeometric information • Make the methodrobustwhen the object of interestisoccupyingonly a small fraction of the image • Investigatemany alternatives for each of the four steps of the basic method

  44. Q&A Thank You

More Related