1 / 31

Representation in Vision

Explore the key principles and features essential for effective representation in visual imagery. Discover how to capture all relevant information, minimize features, and choose the right cues for shape, material, color, and motion. Learn about popular feature detection methods and the importance of spatial support in feature computation.

ncopeland
Download Presentation

Representation in Vision

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Representation in Vision Derek Hoiem CS 598, Spring 2009 Jan 22, 2009

  2. Pipeline for Prediction Imagery Representation Classifier Predictions

  3. Representation is Key Imagery Representation Classifier Predictions Example: 4’s and 9’s

  4. General Principles of Representation • Coverage • Ensure that all relevant info is captured • Concision • Minimize number of features without sacrificing coverage • Directness • Ideal features are independently useful for prediction

  5. Right features depend on what you want to know • Shape: scene-scale, object-scale, detail-scale • 2D form, shading, shadows, texture, linear perspective • Material properties: albedo, feel, hardness, … • Color, texture • Motion • Optical flow, tracked points • Distance • Stereo, position, occlusion, scene shape • If known object: size, other objects

  6. Cues for Shape • Shading • Most effective at object and detail level

  7. Cues for Shape • Shading

  8. Cues for Shape • Shading

  9. Cues for Shape • Shading

  10. Cues for Shape • Shading

  11. Cues for Shape • Shading

  12. Cues for Shape • Shading

  13. Cues for Shape • Shading • Perspective

  14. Cues for Shape • Shading • Perspective • Boundaries/Form

  15. Cues for Shape • Shading • Perspective • Boundaries/Form • Shadows

  16. Features for Shape • Shading • Image filter responses, intensity A good face detector LM Filter Bank

  17. Features for Shape • Histograms of gradient SIFT – Lowe IJCV 2004 HOG – Dalal Triggs 2005

  18. Features for Shape • Detected boundaries Canny Edge Detector

  19. Features for Shape • Perspective Hoiem Efros Hebert 2005

  20. Features for Shape Texture Location Color Perspective • Indirect cues Hoiem Efros Hebert 2005

  21. Features for Material • Color • Texture (filter banks or HOG over regions) L*a*b* color space HSV color space

  22. Features: Motion Efros et al 2003 Optical Flow Zitnick et al. 2005

  23. Computing Features Compute Features over Image Quantize (Optional) Choose Spatial Support Compute Statistics of Features within Spatial Support 71% 29% Histogram Bin Features RGB Values Quantized to 10 Levels

  24. Big Issue: Spatial Support “superpixels” Image from Mori 2005 multiple segmentations Image from Hoiem et al. 2007 regions from segmentation Image from Jianbo Shi (ncuts)

  25. Things to remember • Think about the right features for the problem • Coverage • Concision • Directness • Think about what features represent • Spatial support, statistics of features important

  26. Ranking of Feature Importance by European Art Byzantine: “Christ as the Good Shephard” 425AD Giotto: “The Mourning of Christ” 1305AD

  27. Shape

  28. Representing Shape: Shading

  29. Representing Shape: Shading

  30. Representing Shape • Assumption that light comes from the top

More Related