Assessment of Computational Visual Attention Models on Medical Images

1. Assessment of Computational Visual Attention Models on Medical Images Varun Jampani1, Ujjwal1, Jayanthi Sivaswamy1 and Vivek Vaidya2 1CVIT, IIIT Hyderabad, India 2GE Global Research, Bangalore, India • OBJECTIVE: To investigate the performance of computational saliency models in medical images in the context of abnormality detection. 1 2 5 • Results • Computational Visual Attention Models Visual Search in Medical Images • Feature Integration Theory [Treisman and Gelade 1980] • Visual search paradigm involves identifying targetsfrom surrounding distractors • Visual Attention: Process of selectively attending to an area of visual field while ignoring the surrounding visual areas • Influenced by image-dependent bottom-up features and task-dependent top-down features • Plays important role in finding abnormalities in medical images • [Matsumoto et al. 2011]: Showed that bottom-up mechanisms also play a significant role in guiding the eye movements of neurologists looking for stroke lesions on brain CT images • Global-Focal model of visual search for tumor detection in chest x-rays [Nodine and Kundel 1987] • Overall Pattern recognition • Focal attention • Decision making • In Chest X-ray Study • GBVS model (Mdn AUC = 0.77) outperforms (Z=0.0, p < .001) SR model (Mdn AUC = 0.67) and IK model (Mdn AUC = 0.67 ) in predicting eye fixations of observers • AUC value of 0.77 suggests that GBVS saliency model can be used to a reasonably good accuracy to predict the fixations of the observers • Extended GBVS model by giving more importance to lung regions • AUCs for EGBVS (Mdn = 0.81) are significantly higher (Z = 2.0, p < .001) than those of GBVS (Mdn = 0.77) Mean AUCs corresponding to different saliency models for all the observers in chest x-ray study Disjunctive Target Conjunctive Target • Bottom-up and Top-Down Visual attention models • Produces saliency maps which predict salient regions • Bottom-up models used in present study • Itti-Koch Model (IK) [Itti and Koch 2001] • Spectral residual model (SR) [Hou and Zhang 2007] • Graph based visual saliency model (GBVS) [Harel et al. 2007] • In Retinal Image Study • AUC values found to be: 0.72 for IK, 0.70 for GBVS and 0.73 for SR • All three models perform roughly the same • AUC of SR model rose to 0.74 with optic disk suppression • Extended SR model by computing saliency maps at multiple scales and combining them • ESR model (Mean AUC = 0.94) performed significantly better (28.76% increase) than SR model (Mean AUC = 0.74) 3 • Study on Chest X-rays • We evaluated the role of bottom-up saliency in chest x-rays of pneumoconiosis by comparing the saliency maps against the eye fixations of observers of different expertize levels. • Eye movement recordings were done using a remote head free eye tracker (SR Research - Eyelink 1000) • See [Jampani et al. 2011] for experimental details • ROC analysis is done by using saliency maps as classifiers and considering eye fixations as ground truth 4 Average ROC curves for different saliency models in retinal image study • Study on Retinal Images • We evaluated the bottom-up saliency models by comparing the saliency maps against the ground truth markings by medical experts • Publicly available DIARETDB1 dataset [Kauppi et al. 2007] • ROC analysis is done to compare saliency maps with ground truth markings Input Image Sample X-ray segments showing different stages of Pneumoconiosis (Conjunctive Targets) Itti-Koch GBVS Different steps in attracting EGBVS saliency map from a sample chest x-ray Steps for deriving ESR saliency maps SR 6 Saliency maps extracted from 2 sample retinal images • Conclusion and Future Work Eye fixations of an observer on a sample chest x-ray • Despite the importance of top-down knowledge, bottom-up knowledge plays considerable role • Including top-down knowledge of anatomical regions resulted in marginal improvement in accuracy • Saliency models can be used in the development of CAD tools for medical images • Need to incorporate other types of top-down knowledge A sample retinal image and ground truth markings by four medical experts. Taken from DIARETDB1 dataset [Kauppi et al. 2007] Sample retinal images showing hard exudates (Disjunctive Targets) A sample chest x-ray and the corresponding saliency maps computed using different saliency models Center for Visual Information Technology International Institute of Information Technology, Hyderabad, INDIA Contact Email: varunjampani@research.iiit.ac.in