1 / 28

Segmentation of Salient Regions in Outdoor Scenes Using Imagery and 3-D Data

< WACV2008 >. Segmentation of Salient Regions in Outdoor Scenes Using Imagery and 3-D Data. Gunhee Kim Daniel Huber Martial Hebert Carnegie Mellon University Robotics Institute. Perception for Unmanned Vehicles. Input Data. Sensors. Imagery. 3-D scan. Vrml_file. Problem Statement.

zamir
Download Presentation

Segmentation of Salient Regions in Outdoor Scenes Using Imagery and 3-D Data

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. < WACV2008 > Segmentation of Salient Regions in Outdoor Scenes Using Imagery and 3-D Data Gunhee Kim Daniel Huber Martial Hebert Carnegie Mellon University Robotics Institute

  2. Perception for Unmanned Vehicles Input Data Sensors Imagery 3-D scan Vrml_file

  3. Problem Statement • Our ultimate goal is… • Here, we focus on a crucial pre-requisite step !!

  4. Object Detection • Naïve Scanning

  5. Object Detection • Prioritize the searching regions Salient Regions

  6. Problem Statement • Saliency Detection using Imagery and 3-D Data • A Mid-level vision task • No high-level priors, models, or learning • Only low-level information • Ex) pixel colors, (x,y,z)-coordinates

  7. Problem Statement < Input > Information-theoretic optimal clustering < Output > 1. An image Segmentation of top-k most salient regions 2. 3D scan data

  8. Saliency • Saliency: The quality of standing out relative to neighboring items • Top-down • Driven by high-level concepts • Memories and Experiences • Bottom-up • Driven by low-level features • Intensity, contrast, color, orientation, and motion

  9. Two Bottom-Up Models • Itti-Koch-Niebur Model [1] [1] L.Itti, C.Koch and E.Niebur A Model of Saliency-based Visual Attention for Rapid Scene Analysis, PAMI, 1998.

  10. Two Bottom-Up Models • Kadir-Brady Saliency detector [3] • Find the region which are locally complex, and globally discriminative. Scale, rotation, affine invariance More flatter, higher complex [3] T.Kadir and M. Brady, Scale, Saliency and Image Description. IJCV 45 (2):83-105, 2001.

  11. Previous Work for Saliency in 3-D • BILAS [3]: Itti et al [1]’s model • Cole et al. [4]: • Kadir-Brady [2]’s model [1] S. Frintrop, E. Rome, A. N¨uchter, and H. Surmann. A bimodallaser-based attention system.CVIU, 2005. [2] D. M. Cole, A. R. Harrison, and P.M. Newman. Using naturallysalient regions for slam with 3d laser data, ICRAworkshop on SLAM, 2005.

  12. Saliency of 3-D Data • A point cloud • Gestalt laws of grouping [5] Continuity & Simplicity laws Proximity laws

  13. Saliency of 3-D Data • How to detect salient regions in 3-D data? Uniform Gaussian • (Answer) Find the set of clusters that best fit 3-D pdfs • Gaussian • Uniform

  14. Robust Information-Theoretic Clustering (RIC) [6] • Input: A feature set + Families of pdfs • Output: Clusters according to how well they fit the pdfs • Minimum Description Length (MDL) principle • Goodness of fit = compression costs • Huffman-like coding [6] C.Bohm,C.Faloutsos, J.Y.Pan, and C.Plant, Robust information-theoretic clustering, KDD 2006

  15. Proposed Approach Clustering of 3-D Data using RIC Projection of Clusters on an Image Compute Saliency Values of Clusters Image Segmentation of Top-ranked Saliency Regions

  16. Clustering in 3-D Data • RIC clustering Uniform dist Gaussian dist InformationTheoretic Optimal Clustering

  17. Proposed Approach Clustering on 3-D Data using RIC Projection of Clusters to an Image Projection of Clusters to an Image Compute Saliency Values of Clusters Image Segmentation of Top-ranked Saliency Regions

  18. Saliency Features Local Regional Global 3D-data Image data (RGB color)

  19. Saliency Features 1. Compression Cost- how well the cluster is fit to the reference family of pdfs Local 3D-data

  20. Saliency Features 2. Entropy of RGB histograms [7]- Follow Kadir-Brady’s definition Local RGB-data [7] T. Kadir and M. Brady. Saliency, scale and image description. IJCV, 45(2):83–105, 2001.

  21. Saliency Features 3. Center-surround contrast [8]- Follow Itti et al’s definition Regional RGB-data [8] T. Liu, J. Sun, N.-N. Zheng, X. Tang, and H.-Y. Shum. Learning to detect a salient object, CVPR, 2007.

  22. Saliency Features 4. Color spatial distribution [8]- Rare color: More salient Global RGB-data [8] T. Liu, J. Sun, N.-N. Zheng, X. Tang, and H.-Y. Shum. Learning to detect a salient object, CVPR, 2007.

  23. Proposed Approach Clustering on 3-D Data using RIC Projection of Clusters to an Image Projection of Clusters to an Image Compute Saliency Values of Clusters Image Segmentation of Top-ranked Saliency Regions

  24. Image Segmentation • From Sparse Points to Dense Regions • Using Conventional Markov-Random Field (MRF) Models • Labeling Problem

  25. Qualitative Results

  26. Qualitative Results

  27. Future Work • Integration with the Recognition • Over-segmentation & Under-segmentation • Model-specific pdfs

  28. Conclusion • Bottom-up saliency detection using Imagery and 3-D data as a mid-level task • (x,y,z)-coordinates of 3-D data and Colors at pixels. • Practically useful building block for perception of unmanned vehicles

More Related