1 / 17

Agenda

Agenda. Introduction Bag-of-words models Visual words with spatial location Part-based models Discriminative methods Segmentation and recognition Recognition-based image retrieval Datasets & Conclusions. Aim. Given an image and object category, to segment the object. Object Category

zeke
Download Presentation

Agenda

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Agenda • Introduction • Bag-of-words models • Visual words with spatial location • Part-based models • Discriminative methods • Segmentation and recognition • Recognition-based image retrieval • Datasets & Conclusions

  2. Aim • Given an image and object category, to segment the object Object Category Model Segmentation Cow Image Segmented Cow • Segmentation should (ideally) be • shaped like the object e.g. cow-like • obtained efficiently in an unsupervised manner • able to handle self-occlusion Slide from Kumar ‘05

  3. Examples of bottom-up segmentation • Example: Normalized Cuts, Shi & Malik, 1997 • Difficult without top-down cues Borenstein and Ullman, ECCV 2002

  4. Random Fields for segmentation I = Image pixels (observed) h = foreground/background labels (hidden) – one label per pixel  = Parameters Posterior Joint Likelihood Prior • Generative approach models joint •  Markov random field (MRF) • 2. Discriminative approach models posterior directly •  Conditional random field (CRF)

  5. Likelihood MRF Prior Pairwise Potential (MRF) ij(hi, hj|ij) hi h(labels) {foreground,background} hj Unary Potential i(I|hi,i) Generative Markov Random Field i Prior has no dependency on I j I(pixels) Image Plane

  6. hi hj i j I(pixels) Image Plane Conditional Random Field Lafferty, McCallum and Pereira 2001 Discriminative approach Unary Pairwise • Dependency on I allows introduction of pairwise terms that make use of image. • For example, neighboring labels should be similar only if pixel colors are similar  Contrast term e.g Kumar and Hebert 2003

  7. hi hj i j I(pixels) Figure from Kumar et al., CVPR 2005 Image Plane OBJCUT Kumar, Torr & Zisserman 2005 Unary Pairwise Color Likelihood Distance from Ω Label smoothness Contrast Ω(shape parameter) • Ω is a shape prior on the labels from a Layered Pictorial Structure (LPS) model • Segmentation by: • - Match LPS model to image (get number of samples, each with a different pose • Marginalize over the samples using a single graph cut • [Boykov & Jolly, 2001]

  8. OBJCUT:Shape prior - Ω - Layered Pictorial Structures (LPS) • Generative model • Composition of parts + spatial layout Layer 2 Spatial Layout (Pairwise Configuration) Layer 1 Parts in Layer 2 can occlude parts in Layer 1 Kumar, et al. 2004, 2005

  9. OBJCUT: Results Using LPS Model for Cow In the absence of a clear boundary between object and background Image Segmentation

  10. Layout Consistent Random Field Layout consistency Part detector Winn and Shotton 2006 • Variant of conditional random field I = Image pixels (observed) h = foreground/background labels (hidden) – one label per pixel  = Parameters

  11. Layout CRF: Part detector • Decision forest classifier • Features are differences of pixel intensities Classifier [Lepetit et al. CVPR 2005] Winn and Shotton 2006

  12. Layout consistency (7,2) (8,2) (9,2) (7,3) (8,3) (9,3) (7,4) (8,4) (9,4) Winn and Shotton 2006 Neighboring pixels (p,q) ? (p,q) (p-1,q+1) (p,q+1) (p+1,q+1) Layoutconsistent

  13. Stability of part labelling Part color key

  14. Other recognition & segmentation papers Figure from Borenstein and Ullman, ECCV 2002 Object-Specific Figure-Ground Segregation Stella X. Yu and Jianbo Shi, 2002 Image parsing: Tu, Zhu and Yuille 2003 Implicit Shape Model - Liebe and Schiele, 2003 LOCUS model: See Jon’s talk tomorrowKannan, Jojic and Frey 2004; Winn and Jojic, 2005 Todorovic and Ahuja, CVPR 2006 3D Layout CRF, Hoiem et al. CVPR 2007 See CVPR 2007 course slides for more details

  15. Summary • Strength • Explains every pixel of the image • Useful for image editing, layering, etc. • Issues • Invariance issues • (especially) scale, view-point variations • Inference difficulties

More Related