1 / 30

Learning to Detect A Salient Object

Learning to Detect A Salient Object. Reporter: 鄭綱 (3/2). Outline. Introduction CRF Formulation of Static Salient Object Strong Contrast Center-Surround Histogram Color Spatial-Distribution Learning & Inference for the Model Formulation of Dynamic Salient Object Results Conclusion

trevelian-roberts
Download Presentation

Learning to Detect A Salient Object

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Learning to Detect A Salient Object Reporter: 鄭綱(3/2)

  2. Outline • Introduction • CRF • Formulation of Static Salient Object • Strong Contrast • Center-Surround Histogram • Color Spatial-Distribution • Learning & Inference for the Model • Formulation of Dynamic Salient Object • Results • Conclusion • References

  3. Introduction • Image Labeling Problem Tree Sky Building Plane Lawn

  4. Introduction • Which kinds of information can be used for labeling? • Features from individual sites • Intensity, color, texture, … Vegetation • Interactions with neighboring sites • Contextual information Sky or Building?

  5. Introduction • Contextual information: 2 types of interactions • Interaction with neighboring labels (Spatial smoothness of labels) • neighboring sites tend to have similar labels(except at the discontinuities) Sky • Interactions with neighboring observed data Sky Building

  6. Introduction • Let be the label of the node of the image set S, and Nibe the neighboring nodes of node i. • Three kinds of information for image labeling • Features from local node • Interaction with neighboring labels • Interaction with neighboring observed data node i S-{i} Ni

  7. Introduction • General formulation: where and are called association potential and interaction potential.

  8. Introduction Labels in Spatial Data are NOT independent! • spatially adjacent labels are often the same (Markov Random Fields and Conditional Random Fields) • spatially adjacent elements that have similar features often receive the same label (Conditional Random Fields) • spatially adjacent elements that have different features may not have correlated labels (Conditional Random Fields)

  9. Salient Object? Salience Map?

  10. Formulation of Static Salient Object Detection • CRF model (static image): Maximize!! Minimize!! Salient object feature Pairwise feature Strong contrast Center-surround histogram Color spatial-distribution

  11. Strong Contrast • Generate contrast map for each level of 6-level Gaussian pyramid. Then do linear combination. Level 4 Input image Level 1

  12. Center-Surround Histogram • Salient object usually has a “huge” difference from local area. ……. More different between 2 rectangles !

  13. Center-Surround Histogram where ?? …...

  14. Color Spatial-Distribution • The wider a color is distributed in the image, the less possible a salient object contains this color. • Each pixel is assigned to a color component with the probability:

  15. Color Spatial-Distribution • Then the feature can be defined as a weighted sum:

  16. Formulation of Static Salient Object Detection • CRF model (static image): Maximize!! Minimize!! Salient object feature Pairwise feature Strong contrast Center-surround histogram Color spatial-distribution

  17. Learning & Inference for the Model • The goal of CRF learning is to estimate the linear weights . Training images Gradient descent

  18. Learning & Inference for the Model A training image for example: An training image Label inferred this iterate where Label per pixel Possible label

  19. Learning & Inference for the Model Gradient descent: How to use ground-truth information? Where is the labeled ground-truth. t: iteration

  20. Learning & Inference for the Model • Situations: Ground-truth mistake!!!

  21. Ground-Truth Mistake • Solution: apply Gaussian function to weight every pixel in the rectangle.

  22. Inference • We should find the most probable labeling to maximize in training & detection. • BP – Max-product Belief Propagation [Pearl ‘86] + Can be applied to any energy function – In vision results are usually worse than that of graph cuts – Does not always converge • TRW - Max-product Tree-reweighted Message Passing [Wainwright, Jaakkola, Willsky ‘02] , [Kolmogorov ‘05] + Can be applied to any energy function + Convergence guarantees for the algorithm in [Kolmogorov ’05]

  23. Formulation of Dynamic Salient Object Detection • Similar as static salient object detection! Maximize!! Penalty term of motion Static Salient feature Contrast of motion Center-surround histogram Spatial-distribution of motion

  24. Results From left to right: input image, multi-scale contrast, center-surround histogram, color spatial distribution, and binary mask by CRF.

  25. Results From left to right: Fuzzy growing based method Salience map Their approach Ground-truth

  26. Results Image set A Image set B 1. multi-scale contrast 2. center-surround histogram 3. color spatial distribution 4. combination all 1. FG 2. SM 3. their approach

  27. Conclusion • They model the salient object detection by CRF, where a group of salient features are combined through CRF learning. • It’s a set of novel local, regional & global salient features to define a generic salient object. • Multi-object & no object cases are left as future work.

  28. References (paper & book) • “Learning to Detect A Salient Object”, CVPR 2007, PAMI 2010. • http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.91.4387&rep=rep1&type=pdf • “A Model of Saliency-Based Visual Attention for Rapid Scene Analysis”, PAMI, 1998. • http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.65.2199&rep=rep1&type=pdf • “Pattern Recognition and Machine Learning”, C. M. Bishop. • http://www.library.wisc.edu/selectedtocs/bg0137.pdf

  29. References (about CRF) • “Discriminative Random Fields”, IJCV, 2006. • http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.116.4695&rep=rep1&type=pdf • “Conditional Random Fields: An Introduction”, H. M. Wallach. • http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.64.436&rep=rep1&type=pdf • “Log-linear models & conditional random fields”, C. Elkan, 2008. • http://cseweb.ucsd.edu/~elkan/250B/cikmtutorial.pdf

  30. References (about TRW) • “MAP estimation via agreement on (hyper) trees: Message-passing and linear programming approaches”, IEEE transaction on Information Theory, 2005. • http://arxiv.org/pdf/cs/0508070 • “Convergent Tree-Reweighted Message Passing for Energy Minimization”, PAMI, 2006. • http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.100.2409&rep=rep1&type=pdf • “A Comparative Study of Energy Minimization Methods for Markov Random Fields with Smoothness-Based Priors”, PAMI, 2008. • http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.142.4997&rep=rep1&type=pdf

More Related