1 / 17

Content-Based Image Retrieval

Content-Based Image Retrieval. Jeff Bigham. Who is Steve Seitz?. 7. 1. 4. 10. 2. 8. 11. 5. 3. 9. 6. 12. Goals. Efficient use of Visual Resources Effective Search Intuitive Interfaces Understanding?. Image Search Landscape. Inherits problems of text search Ambiguity Spam

porfirioj
Download Presentation

Content-Based Image Retrieval

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Content-Based Image Retrieval Jeff Bigham

  2. Who is Steve Seitz? 7. 1. 4. 10. 2. 8. 11. 5. 3. 9. 6. 12.

  3. Goals • Efficient use of Visual Resources • Effective Search • Intuitive Interfaces • Understanding?

  4. Image Search Landscape • Inherits problems of text search • Ambiguity • Spam • But vision is hard • uses little or no “vision” • often reduces to text search • Rich UI • Text • Images • Video

  5. Find me an image of a… “Pretty girl doing something active, sporty in a summery setting, beach - not wearing lycra, exercise clothes - more relaxed in tee-shirt. Feature is about deodorant so girl should look active - not sweaty but happy, healthy, carefree - nothing too posed or set up - nice and natural looking.”

  6. images.google.com & flickr.com • Collect Associated Text • alt, title, longdesc attributes of img tag • Information from path (e.g., cow.gif) • nearby text on page • Refinement • remove duplicates • cluster-based improvement • Other • PageRank, etc.

  7. Matching Words and Pictures • Learn text/image segment association • Segment image • Generate feature vector for each segment • Associate text with the segment features • Build generative probabilistic model for text given segmentation features

  8. Matching Words and Pictures Kobus Barnard, Pinar Duygulu, Nando de Freitas, David Forsyth, David Blei, and Michael I. Jordan, "Matching Words and Pictures", Journal of Machine Learning Research, Vol 3, pp 1107-1135, 2003.

  9. Other Ways of Searching • Search by Text • Search by Example • Search by Drawing • Search by Model

  10. Other Search Interfaces? Josef Sivic, Andrew Zisserman, "Video Google: A Text Retrieval Approach to Object Matching in Videos," iccv, p. 1470,  Ninth IEEE International Conference on Computer Vision (ICCV'03) - Volume 2,  2003.

  11. Video Google • Viewpoint Invariant Descriptors

  12. Video Google • Visual Vocabulary

  13. Inverted Index Document 1 Now is the time for all good men to come to the aid of their country. Document 2 Summer has come and passed. The innocent can never last.

  14. Video Google (Demo of Video Google)

  15. Computer Vision is Hard • But, humans do it well Luis von Ahn (ESP Game Demo)

  16. Vision as Security? • True image search is a hard AI problem

  17. The End

More Related