1 / 52

Context-Aware Saliency Detection

Context-Aware Saliency Detection. Introduction Principles of context – aware saliency Detection of context – aware saliency Result Application Conclusion. Introduction. How to describe a figure / picture ? Description: What most people think is Important or Salient . Algorithms.

kamala
Download Presentation

Context-Aware Saliency Detection

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Context-AwareSaliency Detection

  2. Introduction • Principles of context – aware saliency • Detection of context – aware saliency • Result • Application • Conclusion

  3. Introduction • How to describe a figure / picture ? • Description: What most people think is Important or Salient.

  4. Algorithms • First glance: human attention. EX: auto focusing. • Dominant object EX: object recognition/segmentation. • context of the dominant objects: image classification, summarization of a photo collection, thumb nailing, and retargeting.

  5. Context-Aware Saliency Detection • Salient regions are distinctive with respect to both their local and global surroundings. • Prioritize regions close to the foci of attention. -> Maintains the background texture. (Gestalt Law)

  6. Application • Retargeting • Summarization

  7. Contribution • Principles for context-aware saliency • Algorithm • Applicability

  8. Principles of context-aware saliency • 1. Local low-level considerations, including factors such as contrast and color. • 2. Global considerations, which suppress frequently occurring features, while maintaining features that deviate from the norm.

  9. Principles of context-aware saliency • 3. Visual organization rules, which state that visual forms may possess one or several centers of gravity about which the form is organized. • 4. High-level factors, such as human faces.

  10. D.Walther and C. Koch. Modeling attention to salient protoobjects. [2006] • X. Hou and L. Zhang. Saliency detection: A spectral residual approach. [2007] • T. Liu, J. Sun, N. Zheng, X. Tang, and H. Shum. Learning to Detect A Salient Object. [2007]

  11. Detection of context-aware saliency • Areas that have distinctive colors or patterns should obtain high saliency. -- P1 • frequently-occurring features should be suppressed. -- P2 • The salient pixels should be grouped together, and not spread all over the image. -- P3

  12. Local-global single-scale saliency • Consider a single patch of scale r at each pixel. Thus, a pixel i is considered salient if the appearance of the patch pi centered at pixel i is distinctive with respect to all other image patches.

  13. Patch image

  14. Euclidean distance between the vectorized patches and in CIE L*a*b color space.

  15. Euclidean distance between the positions of patches and

  16. Dissimilarity • This dissimilarity measure is proportional to the difference in appearance and inverse proportional to the positional distance.

  17. For every patch pi, we search for the K most similar patches in the image.

  18. Multi-scale saliency enhancement • Background pixels (patches) are likely to have similar patches at multiple scales. • Incorporating multiple scales to further decrease the saliency of background pixels, improving the contrast between salient and non-salient regions.

  19. For a patch pi of scale r, we consider as candidate neighbors all the patches in the image. • We choose the K most similar patchs to compute saliency.

  20. The saliency at pixel i is taken as the mean of its saliency at different scales:

  21. Including the immediate context • Gestalt laws: visual forms may possess one or several centers of gravity about which the form is organized.

  22. Simulate the visual contextual effect • 1. A pixel is considered attended if its saliency value exceeds a certain threshold. • 2. each pixel outside the attended areas is weighted according to its Euclidean distance to the closest attended pixel.

  23. High-level factors • Enhancement factors: 1.Recognized objects 2.Face detection

  24. Results • 3 cases: • Images show a single salient object over an uninteresting background. • Images where the immediate surroundings of the salient object shed light on the story the image tells. • Images of complex scenes. • Compare method: • D.Walther and C. Koch. Modeling attention to salient protoobjects. [2006] • X. Hou and L. Zhang. Saliency detection: A spectral residual Approach. [2007]

  25. Applications • Image retargeting • Summarization through collage creation

  26. Image retargeting • Resizing an image by expanding or shrinking the non-informative regions. • Seam carving M. Rubinstein, A. Shamir, and S. Avidan. Improved seam carving for video retargeting. [2008] • Context-aware saliency

  27. Summarization through collage creation • The salient objects as well as informative pieces of the background should be maintained in summaries.

  28. Automating collage creation S. Goferman, A. Tal, and L. Zelnik-Manor. Puzzle-like collage. [2010] • 3 stages: • Compute the saliency maps for images. • Extracts regions-of-interest by considering both saliency and image edge information. • Assemble non-rectangular ROIs.

  29. Conclusion • Propose a new type of saliency: context-aware saliency • Evaluate in 2 applications: retargeting、summarization

  30. EXTRA ?

More Related