660 likes | 828 Views
Segmentation and Boundary Detection Using Multiscale Intensity Measurements. Eitan Sharon, Meirav Galun, Ronen Basri, Achi Brandt. Dept. of Computer Science and Applied Mathematics The Weizmann Institute of Science. Image Segmentation. Local Uncertainty. Global Certainty. Local Uncertainty.
E N D
Segmentation and Boundary DetectionUsing Multiscale Intensity Measurements Eitan Sharon, Meirav Galun, Ronen Basri, Achi Brandt Dept. of Computer Science and Applied Mathematics The Weizmann Institute of Science
Image Segmentation Eitan Sharon - Weizmann Institute
Local Uncertainty Eitan Sharon - Weizmann Institute
Global Certainty Eitan Sharon - Weizmann Institute
Local Uncertainty Eitan Sharon - Weizmann Institute
Global Certainty Eitan Sharon - Weizmann Institute
Coarse Measurements for Texture Eitan Sharon - Weizmann Institute
A Chicken and Egg Problem Problem: Coarse measurements mix neighboring statistics Solution: support of measurements is determined as the segmentation process proceeds Eitan Sharon - Weizmann Institute
Segmentation by Weighted Aggregation • Normalized-cuts measure in graphs • Complete hierarchy in linear time • Use multiscale measures of intensity, texture, shape, and boundary integrity Eitan Sharon - Weizmann Institute
Segmentation by Weighted Aggregation • Normalized-cuts measure in graphs • Complete hierarchy in linear time • Use multiscale measures of intensity, texture, shape, and boundary integrity Eitan Sharon - Weizmann Institute
Segmentation by Weighted Aggregation • Normalized-cuts measure in graphs • Complete hierarchy in linear time • Use multiscale measures of intensity, texture, shape and boundary integrity Eitan Sharon - Weizmann Institute
The Pixel Graph Couplings Reflect intensity similarity Low contrast – strong coupling High contrast – weak coupling Eitan Sharon - Weizmann Institute
Hierarchical Graph Eitan Sharon - Weizmann Institute
Hierarchyin SWA Eitan Sharon - Weizmann Institute
Normalized-Cut Measure Eitan Sharon - Weizmann Institute
Normalized-Cut Measure Eitan Sharon - Weizmann Institute
Normalized-Cut Measure Eitan Sharon - Weizmann Institute
Normalized-Cut Measure Minimize: Eitan Sharon - Weizmann Institute
High-energy cut Normalized-Cut Measure Minimize: Eitan Sharon - Weizmann Institute
Normalized-Cut Measure Low-energy cut Minimize: Eitan Sharon - Weizmann Institute
Recursive Coarsening Eitan Sharon - Weizmann Institute
Recursive Coarsening Representative subset Eitan Sharon - Weizmann Institute
Recursive Coarsening For a salient segment : , sparse interpolation matrix Eitan Sharon - Weizmann Institute
Weighted Aggregation aggregate aggregate Eitan Sharon - Weizmann Institute
Segment Detection Eitan Sharon - Weizmann Institute
SWA Detects the salient segments Hierarchical structure Linear in # of points (a few dozen operations per point) Eitan Sharon - Weizmann Institute
Coarse-Scale Measurements • Average intensities of aggregates • Multiscale intensity-variances of aggregates • Multiscale shape-moments of aggregates • Boundary alignment between aggregates Eitan Sharon - Weizmann Institute
Adaptive vs. Rigid Measurements Original Averaging Geometric Our algorithm - SWA Eitan Sharon - Weizmann Institute
Adaptive vs. Rigid Measurements Original Interpolation Geometric Our algorithm - SWA Eitan Sharon - Weizmann Institute
Recursive Measurements: Intensity intensity of pixel i aggregate average intensity of aggregate Eitan Sharon - Weizmann Institute
Use Averages to Modify the Graph Eitan Sharon - Weizmann Institute
Use Averages to Modify the Graph Eitan Sharon - Weizmann Institute
Texture Examples Eitan Sharon - Weizmann Institute
Isotropic and Oriented Filters A brief tutorial Textons by K-Means Malik et al IJCV2001 Eitan Sharon - Weizmann Institute
Isotropic Texture in SWA Intensity Variance Isotropic Texture of aggregate – average of variances in all scales Eitan Sharon - Weizmann Institute
Isotropic Texture in SWA Intensity Variance Isotropic Texture of aggregate – average of variances in all scales Eitan Sharon - Weizmann Institute
Isotropic Texture in SWA Intensity Variance Isotropic Texture of aggregate – average of variances in all scales Eitan Sharon - Weizmann Institute
Oriented Texture in SWA with Meirav Galun Shape Moments • center of mass • width • length • orientation Oriented Texture of aggregate – orientation, width and length in all scales Eitan Sharon - Weizmann Institute
Gestalt – Perceptual Grouping Shashua and Ullman ICCV 1988: A brief Tutorial • Group curves by: • Proximity • Co-linearity Sharon, Brandt, Basri PAMI 2000: Eitan Sharon - Weizmann Institute
Boundary Integrity in SWA Eitan Sharon - Weizmann Institute
Sharpen the Aggregates • Top-down Sharpening: • Expand core • Sharpen boundaries Eitan Sharon - Weizmann Institute
Hierarchyin SWA Eitan Sharon - Weizmann Institute
images on a pentium III 1000MHz PC: Experiments • Our SWA algorithm (CVPR’00 + CVPR’01) • run-time: 5-10 seconds. • Normalized cuts (Shi and Malik, PAMI’00; Malik et al., IJCV’01) • run-time: about 10-15 minutes. • Software courtesy of Doron Tal, UC Berkeley. Eitan Sharon - Weizmann Institute
Isotropic Texture - Horse I Our Algorithm (SWA) Normalized Cuts Eitan Sharon - Weizmann Institute
Isotropic Texture - Horse II Our Algorithm (SWA) Normalized Cuts Eitan Sharon - Weizmann Institute
Isotropic Texture - Tiger Our Algorithm (SWA) Normalized Cuts Eitan Sharon - Weizmann Institute
Isotropic Texture - Butterfly Our Algorithm (SWA) Normalized Cuts Eitan Sharon - Weizmann Institute
Isotropic Texture - Leopard Our Algorithm (SWA) Eitan Sharon - Weizmann Institute
Isotropic Texture - Dalmatian Dog Our Algorithm (SWA) Eitan Sharon - Weizmann Institute
Isotropic Texture - Squirrel Our Algorithm (SWA) Normalized Cuts Eitan Sharon - Weizmann Institute