1 / 38

Visual Object Recognition

Visual Object Recognition. Rob Fergus Courant Institute, New York University. http://cs.nyu.edu/~fergus/icml_tutorial/. Agenda. Introduction Bag-of-words models Visual words with spatial location Part-based models Discriminative methods Segmentation and recognition

lefty
Download Presentation

Visual Object Recognition

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Visual Object Recognition Rob Fergus Courant Institute, New York University http://cs.nyu.edu/~fergus/icml_tutorial/

  2. Agenda • Introduction • Bag-of-words models • Visual words with spatial location • Part-based models • Discriminative methods • Segmentation and recognition • Recognition-based image retrieval • Datasets & Conclusions

  3. Recognizing and Learning Object Categories: Year 2007 Li Fei-Fei, Princeton Rob Fergus, NYU Antonio Torralba, MIT http://people.csail.mit.edu/torralba/shortCourseRLOC

  4. Agenda • Introduction • Bag-of-words models • Visual words with spatial location • Part-based models • Discriminative methods • Segmentation and recognition • Recognition-based image retrieval • Datasets & Conclusions

  5. So what does object recognition involve?

  6. Classification: are there street-lights in the image?

  7. Detection: localize the street-lights in the image

  8. Object categorization mountain tree building banner street lamp vendor people

  9. Scene and context categorization • outdoor • city • …

  10. meters Ped Ped Car meters Application: Assisted driving Pedestrian and car detection Lane detection • Collision warning systems with adaptive cruise control, • Lane departure warning systems, • Rear object detection systems,

  11. Application:Computational photography

  12. Application: Improving online search Query: STREET Organizing photo collections

  13. Challenges 1: view point variation Michelangelo 1475-1564

  14. Challenges 2: scale

  15. Challenges 3: illumination slide credit: S. Ullman

  16. Challenges 4: background clutter Bruegel, 1564

  17. Challenges 5: occlusion http://lh5.ggpht.com/_wJc6t2hDl2M/RrL7Gh6sS7I/AAAAAAAAAYY/n3xaHc2opls/DSC00633.JPG

  18. Challenges 6: deformation http://img.timeinc.net/time/asia/magazine/2007/1112/racehorse_1112.jpg Xu, Beihong 1943

  19. History: single object recognition Object 1 Object 2 Object 3

  20. Single object recognition history: Geometric methods David Lowe [1985] Rothwell et al. [1992]

  21. Single object recognition history: Appearance-based methods • Murase & Nayer 1995 • Schmid & Mohr 1997 • Lowe, et al. 1999, 2003 • Mahamud and Herbert, 2000 • Ferrari et al. 2004 • Rothganger et al. 2004 • Moreels and Perona, 2005 • …

  22. Challenges 7: intra-class variation Shoe class Instance 1 Instance 2 Instance 3

  23. History: early object categorization

  24. Fischler, Elschlager, 1973 • Turk and Pentland, 1991 • Belhumeur, Hespanha, & Kriegman, 1997 • Rowley & Kanade, 1998 • Schneiderman & Kanade 2004 • Viola and Jones, 2000 • Heisele et al., 2001 • Amit and Geman, 1999 • LeCun et al. 1998 • Belongie and Malik, 2002 • DeCoste and Scholkopf, 2002 • Simard et al. 2003 • Poggio et al. 1993 • Argawal and Roth, 2002 • Schneiderman & Kanade, 2004 • …..

  25. ~10,000 to 30,000

  26. Three main issues • Representation • How to represent an object category • Learning • How to form the classifier, given training data • Recognition • How the classifier is to be used on novel data

  27. Representation • Generative / discriminative / hybrid

  28. Representation • Generative / discriminative / hybrid • Appearance only or location and appearance

  29. Representation • Generative / discriminative / hybrid • Appearance only or location and appearance • Invariances • View point • Illumination • Occlusion • Scale • Deformation • Clutter • etc.

  30. Representation • Generative / discriminative / hybrid • Appearance only or location and appearance • Invariances • Part-based or global with sub-window

  31. Representation • Generative / discriminative / hybrid • Appearance only or location and appearance • Invariances • Parts or global w/sub-window • Use set of features or each pixel in image

  32. Learning • Unclear how to model categories, so learn rather than manually specify

  33. Learning • Unclear how to model categories, so learn rather than manually specify • Methods of training: generative vs. discriminative

  34. Learning • Unclear how to model categories, so learn rather than manually specify • Methods of training: generative vs. discriminative • Level of supervision • Manual segmentation; bounding box; image labels; noisy labels Contains a motorbike

  35. Learning • Unclear how to model categories, so learn rather than manually specify • Methods of training: generative vs. discriminative • Level of supervision • Manual segmentation; bounding box; image labels; noisy labels • -- Training images: • Issue of over-fitting (typically limited training data) • Negative images for discriminative methods

  36. Learning • Unclear how to model categories, so learn rather than manually specify • Methods of training: generative vs. discriminative • Level of supervision • Manual segmentation; bounding box; image labels; noisy labels • -- Training images: • Issue of over-fitting (typically limited training data) • Negative images for discriminative methods • -- Priors

  37. Recognition • Scale / orientation range to search over • Speed • Context

  38. Recognition • Context enables pruning of detector output Hoiem, Efros, Herbert, 2006

More Related