1 / 18

A Model of Saliency-Based Visual Attention for Rapid Scene Analysis Laurent Itti, Christof Koch, and Ernst Niebur IEEE P

A Model of Saliency-Based Visual Attention for Rapid Scene Analysis Laurent Itti, Christof Koch, and Ernst Niebur IEEE PAMI, 1998. What is Saliency?. Something is said to be salient if it stands out E.g. road signs should have high saliency. Introduction. Trying to model visual attention

aspen
Download Presentation

A Model of Saliency-Based Visual Attention for Rapid Scene Analysis Laurent Itti, Christof Koch, and Ernst Niebur IEEE P

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. A Model of Saliency-Based Visual Attention for Rapid Scene AnalysisLaurent Itti, Christof Koch, and Ernst NieburIEEE PAMI, 1998

  2. What is Saliency? • Something is said to be salient if it stands out • E.g. road signs should have high saliency

  3. Introduction • Trying to model visual attention • Find locations of Focus of Attention in an image • Use the idea of saliency as a basis for their model • For primates focus of attention directed from: • Bottom-up: rapid, saliency driven, task-independent • Top-down: slower, task dependent

  4. Results of the Model • Only considering “Bottom-up”  task-independent

  5. Model diagram

  6. Model • Input: static images (640x480) • Each image at 8 different scales(640x480, 320x240, 160x120, …) • Use different scales for computing “centre-surround” differences (similar to assignment) + Fine scale - Course scale

  7. Feature Maps • Intensity contrast (6 maps) • Using “centre-surround” • Similar to neurons sensitive to dark centre, bright surround, and opposite • Color (12 maps) • Similar to intensity map, but using different color channels • E.g. high response to centre red, surround green

  8. Feature Maps • Orientation maps (24 maps) • Gabor filters at 0º, 45º, 90º, and 135º • Also at different scales  Total of 42 feature maps are combined into the saliency map

  9. Saliency Map • Purpose: represent saliency at all locations with a scalar quantity • Feature maps combined into three “conspicuity maps” • Intensity (I) • Color (C) • Orientation (O) • Before they are combined they need to be normalized

  10. Normalization Operator

  11. Example of operation Leaky integrate-and-fire neurons “Inhibition of return”

  12. Model diagram

  13. Example of operation • Using 2D “winner-take-all” neural network at scale 4 • FOA shifts every 30-70 ms • Inhibition lasts 500-900 ms

  14. Results Image Saliency Map High saliency Locations (yellow circles)

  15. Results • Tested on both synthetic and natural images • Typically finds objects of interest, e.g. traffic signs, faces, flags, buildings… • Generally robust to noise (less to multicoloured noise)

  16. Uses • Real-time systems • Could be implemented in hardware • Great reduction of data volume • Video compression (Parkhurst & Niebur) • Compress less important parts of images

  17. Summary • Basic idea: • Find multiple saliency measures in parallel • Normalize • Combine them to a single map • Use 2D integrate-and-fire layer of neurons to determine position of FOA • Model appears to work accurately and robustly (but difficult to evaluate) • Can be extended with other feature maps

  18. References • Itti, Koch, and Niebur: “A Model of Saliency-Based Visual Attention for Rapid Scene Analysis” IEEE PAMI Vol. 20, No. 11, November (1998) • Itti, Koch: “Computational Modeling of Visual Attention”, Nature Reviews – Neuroscience Vol. 2 (2001) • Parkhurst, Law, Niebur: “Modeling the role of salience in the allocation of overt visual attention”, Vision Research 42 (2002)

More Related