1 / 20

Visual Categorization with Bags of Keypoints

Visual Categorization with Bags of Keypoints. Gabriella Csurka Christopher R. Dance Lixin Fan Jutta Willamowski Cedric Bray. Presented by Yun-hsueh Liu. What is Generic Visual Categorization?. Categorization: distinguish different classes Generic Visual Categorization:

chakra
Download Presentation

Visual Categorization with Bags of Keypoints

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Visual Categorization with Bags of Keypoints Gabriella Csurka Christopher R. Dance Lixin Fan Jutta Willamowski Cedric Bray Presented by Yun-hsueh Liu EE 148, Spring 2006

  2. What is Generic Visual Categorization? • Categorization: distinguish different classes • Generic Visual Categorization: • Generic  to cope with many object types simultaneously • readily extended to new object types. • Handle the variation in view, imaging, lighting, occlusion, and typical object and scene variations EE 148, Spring 2006

  3. Previous Work in Computational Vision • Single Category Detection Decide if a member of one visual category is present in a given image. (faces, cars, targets) • Content Based Image Retrieval Retrieve images on the basis of low-level image features, such as colors or textures. • Recognition Distinguish between images of structurally distinct objects within one class. (say, different cell phones) EE 148, Spring 2006

  4. Bag-of-Keypoints Approach Interesting Point Detection Key Patch Extraction Feature Descriptors Bag of Keypoints Multi-class Classifier EE 148, Spring 2006

  5. SIFT Descriptors Interesting Point Detection Key Patch Extraction Feature Descriptors Bag of Keypoints Multi-class Classifier EE 148, Spring 2006

  6. Bag of Keypoints (1) • Construction of a vocabulary • Kmeans clustering find “centroids” (on all the descriptors we find from all the training images) • Define a “vocabulary” as a set of “centroids”, where every centroid represents a “word”. Interesting Point Detection Key Patch Extraction Feature Descriptors Bag of Keypoints Multi-class Classifier EE 148, Spring 2006

  7. Bag of Keypoints (2) • Histogram • Counts the number of occurrences of different visual words in each image Interesting Point Detection Key Patch Extraction Feature Descriptors Bag of Keypoints Multi-class Classifier EE 148, Spring 2006

  8. Multi-class Classifier • In this paper, classification is based on conventional machine learning approaches • Naïve Bayes • Support Vector Machine (SVM) Interesting Point Detection Key Patch Extraction Feature Descriptors Bag of Keypoints Multi-class Classifier EE 148, Spring 2006

  9. Multi-class classifier –Naïve Bayes (1) • Let V = {vi}, i = 1,…,N, be a visual vocabulary, in which each vi represents a visual word (cluster centers) from the feature space. • A set of labeled images I = {Ii } . • Denote Cj to represent our Classes, where j = 1,..,M • N(t,i) = number of times vi occurs in image Ii (keypoint histogram) • Score approach: want to determine P(Cj|Ii), where (*) EE 148, Spring 2006

  10. Multi-class Classifier –Naïve Bayes (2) • Goal: Find one specific class Cj so that has maximum value • In order to avoid zero probability, use Laplace smoothing: EE 148, Spring 2006

  11. Multi-class classifier –Support Vector Machine (SVM) • Input: the keypoints histogram for each image • Multi-class  one-against-all approach • Linear SVM gives better performances than quadratic or cubic SVM • Goal: find hyperplanes which separate multi-class data with maximun margin EE 148, Spring 2006

  12. Multi-class classifier –SVM (2) EE 148, Spring 2006

  13. Evaluation of Multi-class Classifiers • Three performance measures: • The confusion matrix • Each column of the matrix represents the instances in a predicted class • Each row represents the instances in an actual class • The overall error rate •  = Pr(output class = true class) • The mean ranks • The mean position of the correct labels when labels output by the multi-class classifier are sorted by the classifier score. EE 148, Spring 2006

  14. n-Fold Cross Validation • What is “fold”? • Randomly break the dataset into n partitions • Example: suppose n = 10 • Training on 2, 3,…,10; testing on 1 = result 1 • Training on 1, 3,…,10; testing on 2 = result 2 • … • Answer = Average of result 1, result 2, …. EE 148, Spring 2006

  15. Experiment on Naïve Bayes –k’s effect • Present the overal error rate as a function of # of clusters k • Result • Error rate decreases as k increases • Selecting point: k = 1000 • After passing the selecting point, the error rate decreases slowly EE 148, Spring 2006

  16. Experiment on Naïve Bayes –Confusion Matrix EE 148, Spring 2006

  17. Experiment on SVM –Confusion Matrix EE 148, Spring 2006

  18. Interpretation of Results • The confusion matrix • In general, SVM has more correct predictions than Naïve Bayes does • The overall error rate • In general, Naïve Bayes > SVM • The Mean Rank • In general, SVM < Naïve Bayes EE 148, Spring 2006

  19. Why do we have errors? • There are objects from more than 2 classes in one image • The data set is not totally clean (noise) • Each image is given only one training label EE 148, Spring 2006

  20. Conclusion • Bag-of-Keypoints is a new and efficient generic visual categorizer. • Evaluated on a seven-category database, this method is proved that it is robust to • Choice of clusters, background clutter, multiple objects • Any Questions? • Thank you for listening to my presentation!! :) EE 148, Spring 2006

More Related