1 / 37

Machine learning & category recognition

Learn about the difficulties in recognizing objects despite changes in scale, lighting, and more. Explore applications like image search, product identification, and copy detection. Understand the significance of machine learning in visual recognition and the extraction of invariant image descriptors for efficient search. Discover the importance of instance-level recognition and object localization techniques using geometric verification and statistical modeling. This class offers insights into machine learning approaches for image classification and object localization, providing tools for efficient visual category recognition.

katiem
Download Presentation

Machine learning & category recognition

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Machine learning & category recognition Cordelia Schmid Jakob Verbeek

  2. This class • Part 1: Visual object recognition • Part 2 : Machine learning

  3. Visual recognition - Objectives • Particular objects and scenes, large databases …

  4. Difficulties Finding the object despite possibly large changes in scale, viewpoint, lighting and partial occlusion  requires invariant description Scale Viewpoint Occlusion Lighting

  5. Difficulties • Very large images collection  need for efficient indexing • Flickr has 2 billion photographs, more than 1 million added daily • Facebook has 15 billion images (~27 million added daily) • Large personal collections • Video collections, i.e., YouTube

  6. Applications Search photos on the web for particular places ...in these images and 1M more Find these landmarks

  7. Applications • Take a picture of a product or advertisement  find relevant information on the web [Pixee – Milpix]

  8. Applications • Finding stolen/missing objects in a large collection …

  9. Applications • Copy detection for images and videos Search in 200h of video Query video

  10. Sony Aibo – Robotics Recognize docking station Communicate with visual cards Place recognition Loop closure in SLAM Applications 10 K. Grauman, B. Leibe Slide credit: David Lowe

  11. Instance-level recognition: Approach • Extraction of invariant image descriptors • Matching descriptors between images • Matching of the query images to all images of a database • Speed-up by efficient indexing structures • Geometric verification • Verification of spatial consistency for a short list

  12. This class • Lecture 2: Local invariant features • Student presentation: scale and affine invariant interest point detectors

  13. This class • Lecture 3: Instance-level recognition: efficient search • Student presentation: scalable recognition with a vocabulary tree

  14. Visual recognition - Objectives • Object classes and categories (intra-class variability)

  15. Cow Car Visual recognition - Objectives Visual object recognition Tasks • Image classification: assigning label to the image Car: present Cow: present Bike: not present Horse: not present … • Object localization: define the location and the category Location Category

  16. Difficulties: within object variations Variability: Camera position, Illumination,Internal parameters Within-object variations

  17. Difficulties: within-class variations

  18. Visual category recognition • Robust image description • Appropriate descriptors for objects and categories • Statistical modeling and machine learning for vision • Selection and adaptation of existing techniques

  19. Why machine learning? • Early approaches: simple features + handcrafted models • Can handle only few images, simples tasks L. G. Roberts, Machine Perception of Three Dimensional Solids, Ph.D. thesis, MIT Department of Electrical Engineering, 1963.

  20. Why machine learning? • Early approaches: manual programming of rules • Tedious, limited and does not take into accout the data Y. Ohta, T. Kanade, and T. Sakai, “An Analysis System for Scenes Containing objects with Substructures,” International Joint Conference on Pattern Recognition, 1978.

  21. Internet images, personal photo albums Movies, news, sports Why machine learning? • Today lots of data, complex tasks

  22. Medical and scientific images Surveillance and security Why machine learning? • Today lots of data, complex tasks

  23. Why machine learning? • Today: Lots of data, complex tasks • Instead of trying to encode rules directly, learn them from examples of inputs and desired outputs

  24. Types of learning problems • Supervised • Classification • Regression • Unsupervised • Semi-supervised • Reinforcement learning • Active learning • ….

  25. bikes books building cars people phones trees Image classification : Approach Bag-of-features for image classification • Excellent results in the presence of background clutter

  26. Bag-of-features for image classification SVM Extract regions Compute descriptors Find clusters and frequencies Compute distance matrix Classification

  27. This class Spatial pyramids: perform matching in 2D image space • Lecture 4: Bag-of-features models for image classification • Student presentation: beyond bags of features: spatial pyramids

  28. Object category localization: examples Bicycle Car Horse Sofa

  29. Object category localization • Method with sliding windows (Each window is classified as containing or not the targeted object) • Learn a classifier by providing positive and negative examples

  30. Localization approach Histogram of oriented image gradients as image descriptor SVM as classifier, importance weighted descriptors

  31. Localization of “shape” categories Window descriptor + SVM Horse localization

  32. Localization based on shape

  33. This class • Lecutre 5: Category-level object localization • Student presentation: object detection with discriminatively trained part based models

  34. This class - schedule • Session 1, October 1 2010 • Cordelia Schmid: Introduction • Jakob Verbeek: Introduction Machine Learning • Session 2, December 3 2010 • Jakob Verbeek: Clustering with k-means, mixture of Gaussians • Cordelia Schmid: Local invariant features • Student presentation 1 : Scale and affine invariant interest point detectors, Mikolajczyk and Schmid, IJCV 2004. • Session 3, December 10 2010 • Cordelia Schmid: Instance-level recognition: efficient search • Student presentation 2: Scalable recognition with a vocabulary tree, Nister and Stewenisus, CVPR 2006.

  35. This class - schedule Plan for the course • Session 4, December 17 2010 • Jakob Verbeek: Mixture of Gaussians, EM algo.,Fisher Vector image representation • Cordelia Schmid: Bag-of-features models for category-level classification • Student presentation2: Beyond bags of features: spatial pyramid matching for recognizing natural scene categories, Lazebnik, Schmid and Ponce, CVPR 2006. • Session 5, January 7 2011 • Jakob Verbeek: Classification 1: generative and non-parameteric methods • Student presentation 4: Large-scale image retrieval with compressed Fisher vectors, Perronnin, Liu, Sanchez and Poirier, CVPR 2010. • Cordelia Schmid: Category level localization: Sliding window and shape model • Student presentation 5: Object detection with discriminatively trained part based methods, McAllester and Ramanan, PAMI 2010. .

  36. This class - schedule Plan for the course • Session 6, January 14 2011 • Jakob Verbeek: Classification 2: discriminative models • Student presentation 6:TagProp: discriminative metric learning in nearest neighbor models for image auto-annotation, Guillaumin, Mensink, Verbeek and Schmid, ICCV 2009. • Student presentation 7: IMG2GPS: estimating geographic information from a single image, Hays and Efros, CVPR 2008.

  37. This class • Class web page at • http://lear.inrialpes.fr/people/verbeek/MLCR.10.11 • Slides available after class • Student presentations • 20 minutes oral presentation with slides, 5 minutes questions • Two students present together one paper • Grades • 50% final exam • 25% presentation • 25% short quiz after each presentation

More Related