1 / 32

Beyond bags of features: Part-based models

Beyond bags of features: Part-based models. Many slides adapted from Fei-Fei Li, Rob Fergus, and Antonio Torralba. visual codeword with displacement vectors. Implicit shape models. Visual codebook is used to index votes for object position.

rollo
Download Presentation

Beyond bags of features: Part-based models

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Beyond bags of features:Part-based models Many slides adapted from Fei-Fei Li, Rob Fergus, and Antonio Torralba

  2. visual codeword withdisplacement vectors Implicit shape models • Visual codebook is used to index votes for object position training image annotated with object localization info B. Leibe, A. Leonardis, and B. Schiele, Combined Object Categorization and Segmentation with an Implicit Shape Model, ECCV Workshop on Statistical Learning in Computer Vision 2004

  3. Implicit shape models • Visual codebook is used to index votes for object position test image B. Leibe, A. Leonardis, and B. Schiele, Combined Object Categorization and Segmentation with an Implicit Shape Model, ECCV Workshop on Statistical Learning in Computer Vision 2004

  4. Implicit shape models: Training • Build codebook of patches around extracted interest points using clustering

  5. Implicit shape models: Training • Build codebook of patches around extracted interest points using clustering • Map the patch around each interest point to closest codebook entry

  6. Implicit shape models: Training • Build codebook of patches around extracted interest points using clustering • Map the patch around each interest point to closest codebook entry • For each codebook entry, store all positions it was found, relative to object center

  7. Implicit shape models: Testing • Given test image, extract patches, match to codebook entry • Cast votes for possible positions of object center • Search for maxima in voting space • Extract weighted segmentation mask based on stored masks for the codebook occurrences

  8. Original image Example: Results on Cows Source: B. Leibe

  9. Interest points Example: Results on Cows Source: B. Leibe

  10. Example: Results on Cows Matched patches Source: B. Leibe

  11. Example: Results on Cows Probabilistic votes Source: B. Leibe

  12. Example: Results on Cows Hypothesis 1 Source: B. Leibe

  13. Example: Results on Cows Hypothesis 2 Source: B. Leibe

  14. Example: Results on Cows Hypothesis 3 Source: B. Leibe

  15. Additional examples B. Leibe, A. Leonardis, and B. Schiele, Robust Object Detection with Interleaved Categorization and Segmentation, IJCV 77 (1-3), pp. 259-289, 2008.

  16. Generative part-based models R. Fergus, P. Perona and A. Zisserman, Object Class Recognition by Unsupervised Scale-Invariant Learning, CVPR 2003

  17. What is a generative model? • Model the probability of an image given a class vs.

  18. Making a decision • Maximum a posteriori (MAP) decision: assign the image to the class that gets the highest posterior probability P(class | image)

  19. Making a decision • Maximum a posteriori (MAP) decision: assign the image to the class that gets the highest posterior probability P(class | image) • Bayes rule:

  20. posterior likelihood prior Generative vs. discriminative models • Generative methods: model likelihood and prior • Discriminative methods: model posterior

  21. Generative vs. discriminative models Generative Discriminative Posterior probabilities Class densities

  22. The Naïve Bayes model • Generative model for a bag of features • Assume that each feature fi is conditionally independent given the class c:

  23. Generative part-based model Partdescriptors Partlocations Candidate parts

  24. Generative part-based model h: assignment of features to parts Part 1 Part 3 Part 2

  25. Generative part-based model h: assignment of features to parts Part 1 Part 3 Part 2

  26. Generative part-based model Distribution over patchdescriptors High-dimensional appearance space

  27. Generative part-based model Distribution over jointpart positions 2D image space

  28. Results: Faces Faceshapemodel Patchappearancemodel Recognitionresults

  29. Results: Motorbikes and airplanes

  30. Pictorial structures • Set of parts (oriented rectangles) connected by edges • Recognition problem: find the most probable part layout l1, …, ln in the image P. Felzenszwalb and D. Huttenlocher, Pictorial Structures for Object Recognition, IJCV 61(1), 2005

  31. Pictorial structures • MAP formulation: maximize posterior • Energy-based formulation: minimize minus the log of probability: Appearance Geometry Deformation cost Matching cost

  32. Pictorial structures: Complexity • Suppose there are n parts, and each has h possible positions in the image • What is the complexity of finding the optimal layout? • Brute force search: O(hn) • Tree-structured model: O(h2n) • Deformation cost is quadratic: O(hn)

More Related