1 / 60

Context-aware saliency

Context-aware saliency. Stas Goferman Lihi Zelnik -Manor Ayellet Tal. What is saliency?. …. Please describe this picture. Picture description. Man in a flower field In the fields Spring blossom. Please describe this picture. Picture description. Olympic weight lifter

anakin
Download Presentation

Context-aware saliency

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Context-aware saliency StasGoferman LihiZelnik-Manor Ayellet Tal

  2. What is saliency? …

  3. Please describe this picture

  4. Picture description Man in a flower field In the fields Spring blossom

  5. Please describe this picture

  6. Picture description Olympic weight lifter Olympic victory Olympic achievement

  7. Segmentation • Man in a flower field • In the fields • Spring blossom • Olympic weight lifter • Olympic victory • Olympic achievement

  8. Context-aware salient regions • Man in a flower field • In the fields • Spring blossom • Olympic weight lifter • Olympic victory • Olympic achievement

  9. Principles of saliency Following perceptual properties

  10. Principle 1 • Local low-level factors • Contrast • Color

  11. Walther & Koch – local filtering [Walther and Koch, Neural Networks 2006]

  12. Principle 1 Walther & Koch, 2006 • Local low-level factors • Contrast • Color

  13. Principle 2 • Global considerations • Maintain unique features

  14. Hou & Zhang “spectral residual” [Hou & Zhang CVPR 2007]

  15. Principle 2 Hou & Zhang, 2007 • Global considerations • Maintain unique features

  16. Principles 1 + 2 Local & global

  17. Liu et al – learning saliency Input Multi-scale contrast Center surround Color Final [Liu et al, CVPR 2007]

  18. Principles 1 + 2 Liu et al, 2007 Local & global

  19. Principle 3 • Visual organization (Gestalt) • Few centers of gravity [Koffka] • Position is important!!

  20. Principle 4 Low-level With face detection [Judd et al, ICCV 2009] • High-level • Faces • Objects • People • …

  21. Incorporating the 4 Principles Our result

  22. Global Hou & Zhang, 2007 Local Walther & Koch, 2006 Local + global Liu et al, 2007 Our result

  23. The “how” The steps of our algorithm

  24. Local-global saliency Not salient salient Principles 1-2: Unique appearance  salient

  25. Computing uniqueness Principles 1-2: Unique appearance  salient

  26. Appearance uniqueness Euclidean distance between colors of patches at pi & pj Principles 1-2: Unique appearance  salient

  27. Appearance uniqueness high salient Principles 1-2: Unique appearance  salient

  28. Positional information Similar patches both near and far Not salient Principle 3: Position is important!

  29. Positional information Similar patches near Salient Principle 3: Position is important!

  30. Positional information Normalized Euclidean distance between positions of pi & pj Principle 3: Position is important!

  31. Single scale uniqueness High salient Distance between a pair of patches:

  32. Single scale uniqueness High for K most similar salient Distance between a pair of patches:

  33. Single scale saliency K most similar patches at scale r

  34. Single scale saliency

  35. Multiple scales Scale 1 Scale 4 • Salient at: • Multiple scales  foreground • Few scales  background

  36. Including immediate context Context • Principle 3: • Few centers of gravity

  37. Visual organization Focus points Distance map Final result X

  38. Algorithm summary X Single-scale saliency Multiple scales Final saliency

  39. Results …

  40. Non-interesting background Our result Walther & Koch, 2006 Hou & Zhang, 2007

  41. Non-interesting background Our result Walther & Koch, 2006 Hou & Zhang, 2007

  42. Object + immediate surrounding Our result Walther & Koch, 2006 Hou & Zhang, 2007

  43. Object + immediate surrounding Our result Walther & Koch, 2006 Hou & Zhang, 2007

  44. Complex scenes Our result Walther & Koch, 2006 Hou & Zhang, 2007

  45. Complex scenes Our result Walther & Koch, 2006 Hou & Zhang, 2007

  46. Quantitative evaluation Database of Hou & Zhang

  47. Judd et al. database Our + center Judd Our

  48. Judd et al. database Our + center Judd Our

  49. Boiman & Irani[IJCV’07]

  50. Boiman & Irani[IJCV’07] Input Boiman & Irani Our result

More Related