1 / 28

Building a Dictionary of Image Fragments

Building a Dictionary of Image Fragments. Zicheng Liao Ali Farhadi Yang Wang Ian Endres David Forsyth Department of Computer Science, University of Illinois at Urbana-Champaign. Outline. Introduction Related Work Building a Dictionary of Fragments Applications Conclusion. Introduction.

Download Presentation

Building a Dictionary of Image Fragments

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Building a Dictionary of Image Fragments Zicheng Liao Ali Farhadi Yang Wang Ian Endres David Forsyth Department of Computer Science, University of Illinois at Urbana-Champaign

  2. Outline • Introduction • Related Work • Building a Dictionary of Fragments • Applications • Conclusion

  3. Introduction

  4. Introduction • Image fragment – regions that could represent a single object, an object in context or a piece of a scene – form a natural representation of objects. • Key step - to build a large, rich dictionary of image fragments automatically.

  5. Introduction • Four steps: • 1.Generate a set of fragment proposals from image sets. • 2.Verifythe qualities of the generated fragment proposals with a discriminative method. The selected fragments are grouped and indexed by the labels of their source images. • 3.Use a clean-up procedure to remove anomalies from the dictionary within each category. • 4.Matte the resulting fragments out of the training images to get the best possible boundaries.

  6. Related Work • Automated object segmentation • (a) that it is useful to work with more than one segmentation of a particular image, then choose good fragments • (b) that these multiple segmentations can yield estimates of support • (c) that it is possible to identify segments that seem to form a single object, without knowing what the object is

  7. Related Work • Exemplar-based image classification • There exist methods for image classification using region-based exemplar matching and image-based exemplar matching

  8. Building a Dictionary of Fragments • 1. Generating fragment proposals • 2. Fragment verification • 3. Dictionary clean-up • 4. Matting dictionary fragments

  9. Building a Dictionary of Fragments

  10. Building a Dictionary of Fragments-Generating fragment proposals • Proposing regions: This step aims to generate a large and diverse set of proposals that are likely to be object regions.

  11. Building a Dictionary of Fragments-Generating fragment proposals • Ranking proposals: • The next step of is to rank all the proposals in an image so that object regions are ranked higher than non-object regions. • Use a rank-SVM formulation based on various features computed from proposal regions, such as color, texture, geometric surfaces, boundaries, etc.

  12. Building a Dictionary of Fragments-Fragment Verification

  13. Building a Dictionary of Fragments-Fragment Verification • Flickr: These images are downloaded from Flickr. They are mainly about humans, objects, activities, pets, and familiar scenes (indoor and outdoor). • Caltech256: This dataset is widely used for object recognition in the computer vision literature. • PASCAL VOC2010: This is another widely used object recognition dataset. The images are more complex than those in Caltech256.

  14. Building a Dictionary of Fragments-Fragment Verification

  15. Building a Dictionary of Fragments-Fragment Verification • A good fragment can be a whole object, a meaningful component of a larger object , or a scene that consists of part of an scene. • A good fragment’s effective size should cover neither too little nor too much of the entire image domain. If a fragment covers too little of the image, it may contain little discriminative information and is unlikely to be useful for image compositing. On the other hand, if a fragment covers too much of the image, its distinction from the whole image is small.

  16. Building a Dictionary of Fragments-Dictionary Cleanup • Fragments associated with the same tag (e.g. “cat”) in the source images are grouped as “cat” fragments. • In this step we want to remove such within-class fragment anomalies from the dictionary.

  17. Building a Dictionary of Fragments-Dictionary Cleanup • How to cleanup • For each new incoming fragment, we use an adapted asymmetric region-to-image matching algorithm to measure its distance to the fragment set and count the top k (5 in experiment) best matches.

  18. Building a Dictionary of Fragments-Dictionary Cleanup

  19. Building a Dictionary of Fragments-Matting Dictionary Fragments • The closed-form matting algorithm simplifies the matting equation with a local window color smoothness assumption, and transforms the problem into a quadratic optimization problem, which can be solved efficiently via a sparse linear solver.

  20. Applications • 1. Image Classification • 2. Object Localization • 3. Image Composition

  21. Applications- Image Classification • Fragments may have a small advantage because contextual information creates noise, or because our fragment selection procedure will prefer images with high contrast between fragment and background.

  22. Applications- Image Classification

  23. Applications- Object Localization • The spatial support of fragments can be used to accurately localize objects in a query image.

  24. Applications- Object Localization

  25. Applications-Image Composition

  26. Applications-Image Composition

  27. Conclusions • Use the highly localized information of fragment-based matching to do object detection.

  28. Thanks for your listening.

More Related