Segmentation of Polarimetric SAR Data Using Spectral Graph Partitioning: Utilizing Multiple Cues

1. Segmentation of Polarimetric SAR Data Using Spectral Graph Partitioning: Utilizing Multiple Cues Kaan Ersahin*, Ian Cumming and Rabab K. Ward Dept. of Electrical and Computer Engineering University of British Columbia Vancouver, Canada

2. OUTLINE • Motivation • Using Ideas from HVS • Spectral Graph Partitioning (SGP) • Utilizing patch-based similarity in SGP • Utilizing contour information in SGP • Proposed Scheme • Results • Summary • Future Work 2 Segmentation of Polarimetric SAR Data Using Spectral Graph Partitioning: Utilizing Multiple Cues

3. Motivation • Manual segmentation of SAR data is a common practice • Human experts are often good at visual interpretation • Operational use of polarimetric spaceborne systems means: • Daily acquisitions  more data to analyze • Wider spectrum of users with limited or no expertise in SAR Polarimetry • Automated analysis procedures are needed • To develop better decision making tools that require less analyst (human) interaction • Analysis typically involves: Segmentation • e.g., drawing boundaries between agricultural fields, water - ice separation, etc. • Automated segmentation task is very challenging • Edge detection followed by linking or region merging methods often do not perform well • Human vision system (HVS) can perform this task easily • Identify lines, contours, patterns and regions and make decisions based on global information 3 Segmentation of Polarimetric SAR Data Using Spectral Graph Partitioning: Utilizing Multiple Cues

4. Global view Local view Convair-580, C-band, color composite © CSA 2004 Importance of using global view 4 Segmentation of Polarimetric SAR Data Using Spectral Graph Partitioning: Utilizing Multiple Cues

5. Motivation – Developing a better method • A number of useful analysis techniques have been developed • ML classifier based on Wishart distribution (Lee et. al) • Eigenvalue decomposition  H / A / α-angle (Cloude - Pottier) • Target decomposition based on physical models (Freeman - Durden) • … their combinations and variants • These are based on polarimetric attributes of pixels (or averages in a neighborhood) • Not able to capture the information that human observer can pick up • Visual aspectof image data can be used to enhance automated segmentation results • Study how humans handle this task • Use the ideas that have led to the state-of-the-art technique in computer vision 5 Segmentation of Polarimetric SAR Data Using Spectral Graph Partitioning: Utilizing Multiple Cues

6. Continuity Closure Using Ideas from HVS • In computer vision problems (e.g., segmentation, object detection) • The ultimate goal  To reach the performance level of an human expert • What does an image mean for humans? • More than the collection of pixels, represents a meaningful organization of objects or patterns • In late 1930s, GestaltPsychologists 1 studied this phenomenon: perceptual organization • Several cues (i.e., factors that contribute to this process) were reported: Similarity (e.g., brightness, color, shape) XX X X O X X X X X X X X O X X X XXX X X O X X X X X X X X O Proximity (geometric) X X X X X X X XX X X X X X X X X X • In computer vision, a promising technique that can utilize these ideas has emerged: Spectral Graph Partitioning • Gestalt: a configuration or pattern of elements so unified as a whole that its properties cannot be derived from a simple summation of its parts. 6

7. G W  Spectral Graph Partitioning (SGP) • A pair-wise grouping technique: an alternative to central grouping • No assumption on the statistical distribution of the data (e.g., Gaussian) • Avoids the restriction that all the points must be similar to a prototype (i.e., class mean) • Enables combination of multiple cues (e.g., different types of features and data sets) • Offers flexibility in the definition of affinity functions (i.e., measure of similarity) • G = { V , E } is an undirected graph • V nodes (data points or pixels) • E edges (connections between node pairs) • (i,j) weights (similarity between nodei and node j) • W similarity matrix ; its entries are the weights: (i,j) 7 Segmentation of Polarimetric SAR Data Using Spectral Graph Partitioning: Utilizing Multiple Cues

8. To divide the graph into two partitions, intuitively: • similarity between the resulting partitions or • cost of removing all the connections between the candidate partitions (i.e., cut) should be minimized • A better way: Minimize the Normalized Cut  Spectral Graph Partitioning • Shi and Malik (2000) showed that solving the eigenvalue problem for the Normalized Graph Laplacian: provides a reasonable solution. • Yu and Shi (2003) showed that eigenvectors completely characterize all optimal solutions • Space of global optima can be navigated via orthogonal transforms. • Iteratively solve for a discrete solution that is closest to the continuous global optimum using an alternating optimization procedure • Their method is called Multiclass Spectral Clustering (MSC). 8 Segmentation of Polarimetric SAR Data Using Spectral Graph Partitioning: Utilizing Multiple Cues

9. Utilizing Patch-based Similarity in SGP PolSAR Data • We have used SGP for classification based on patch-based similarity (IGARSS 2006) • Spectral Clustering algorithm is modified to account for the unique properties of SAR data • Instead of pixel intensities, the histograms calculated within an edge-aligned window mask are used as attributes. • Similarity is measured using the 2 – distance • Form an affinity matrix to account for spatial proximity • Patch-based similarity cues from multiple channels and proximity are combined in an overall affinity matrix (W) Multi-looking Speckle Reduction Form affinity matrix (W) Spectral Graph Partitioning 9 Segmentation of Polarimetric SAR Data Using Spectral Graph Partitioning: Utilizing Multiple Cues

10. Utilizing Contour Information in SGP • In IGARSS 2007 we used SGP for segmentation based on contour information. The motivation was: • Region-based techniques perform either: • Sequential merging of segments based on an appropriate measure (e.g., likelihood ratio test) • Optimization of a global objective function • Drawback: contour information – a powerful cue for HVS – is not utilized. • Contour-based techniques often start with edge detection, followed by a linking process. • Drawback: Only local information is used; decisions on segment boundaries are made prematurely • Leung and Malik addressed this issue by collecting contour information locally (i.e., through orientation energy (OE), but making the decision globally. 10 Segmentation of Polarimetric SAR Data Using Spectral Graph Partitioning: Utilizing Multiple Cues

11. Orientation Energy at orientation angle of 0  Rotated copies of filters will pick up edge contrast at different orientations: Orientation energy of a pixel located at (x,y)  Useful properties: and form a quadrature pair. Filters are elongated, information is integrated along the edge  Extended contours will stand out as opposed to short ones Utilizing Contour Information in SGP 11 Segmentation of Polarimetric SAR Data Using Spectral Graph Partitioning: Utilizing Multiple Cues

12. Utilizing Contour Information in SGP Dissimilarity of two pixels • Based on the presence of an extended contour, pixel pairs can be assigned to same or different partitions • OE is strong along l2s1 and s3 are in different partitions. • OE is weak along l1s1 and s2 are in the same partition. • Pairwise affinity matrix is formed using Eq. 10: 12 Segmentation of Polarimetric SAR Data Using Spectral Graph Partitioning: Utilizing Multiple Cues

13. Utilizing Contour Information in SGP PolSAR Data Multi-looking • Perform multi-looking on SLC data set Form affinity matrix (W) • Form affinity matrix for each channel based on OE • To account for proximity in the image plane calculate affinities only within a neighborhood. Spectral Graph Partitioning • Perform the steps of Multiclass Spectral Clustering (MSC) algorithm by Yu and Shi. 13 Segmentation of Polarimetric SAR Data Using Contour Information via Spectral Graph Partitioning

14. PolSAR Data • Perform multi-looking on SLC data set Multi-looking • Form affinity matrix W for each data channel • To account for proximity in the image plane calculate affinities only within a neighborhood. • Contour information is measured using Orientation Energy (OE) • Perform the Spectral Graph Partitioning (SGP) using the Multiclass Spectral Clustering (MSC) algorithm. Proximity Contour Information SGP Proposed Scheme • Form affinity matrix W and perform SGP • Similarity is defined between segments obtained from the previous step. ( 2 – distance between the histograms is used) • Only consider adjacent segments Patch-based similarity SGP 14 Segmentation of Polarimetric SAR Data Using Spectral Graph Partitioning: Utilizing Multiple Cues

15. Data Set: Westham Island, B.C. • Data Acquisition: • Convair-580, C - band, Sept. 2004 • For the regions # 1 and # 2 the reference segmentation was formed by: • Inspection of the field boundaries and crops on the day of the acquisition • Visual interpretation of the image data • Manual Segmentation © CSA 2004 15 Segmentation of Polarimetric SAR Data Using Spectral Graph Partitioning: Utilizing Multiple Cues

16. Data Set: Westham Island, B.C. • For region #3 a classification map was formed using: • GPS measurements at the field boundaries • Inspection of the crops in each field on the day of the acquisition • Visual interpretation of the image data © CSA 2004 16 Segmentation of Polarimetric SAR Data Using Spectral Graph Partitioning: Utilizing Multiple Cues

17. Potatoes Hay Barley -1 Pumpkin Bare Soil Barley – 2 Strawberry Turnip Data Set: Westham Island, B.C. 17 Segmentation of Polarimetric SAR Data Using Contour Information via Spectral Graph Partitioning

18. Results – Region # 1 Wishart • 6 fields • Wishart result contains isolated pixels • Proposed Method: • More homogenous • Visually agrees with reference segmentation 18 Segmentation of Polarimetric SAR Data Using Spectral Graph Partitioning: Utilizing Multiple Cues

19. Results – Region # 2 Wishart • 8 fields • Wishart result contains isolated pixels • Proposed Method: • More homogenous • Visually agrees with reference segmentation 19 Segmentation of Polarimetric SAR Data Using Spectral Graph Partitioning: Utilizing Multiple Cues

20. Grass Pumpkin • Problems: • Adjacent fields with same crop type • Concave regions (Similarity calculation using OE suggests there should be two partitions • Non-adjacent fields with same crop type. (To be solved at the level of classification) Results – Region # 3 • 13 different fields 20 Segmentation of Polarimetric SAR Data Using Spectral Graph Partitioning: Utilizing Multiple Cues

21. Summary • A new technique for segmentation of polarimetric SAR data is proposed • Motivated by the visual information content that humans utilize • Is based on SGP which was shown to perform well on computer vision problems • A pair-wise grouping technique instead of central grouping. • Contour cue and Proximity is used for initial partitioning • Patch-based similarity is used later to merge adjacent partitions • Preliminary results are given on image subsets of Convair-580 data (C-band) • Perceptually plausible results: more homogenous, agree with the reference (i.e., manual) segmentation • Resulting classification is better than Wishart • This scheme is flexible to allow further improvement using additional information 21 Segmentation of Polarimetric SAR Data Using Spectral Graph Partitioning: Utilizing Multiple Cues

22. Future Work • Utilize the complete polarimetric information using pairwise similarity of the coherency matrices. • Include additional information (e.g., scattering mechanisms) • Optimize the cue combination scheme • Compare with techniques other than Wishart • Validate methodology for • Different datasets (CV-580) • RADARSAT-2 22 Segmentation of Polarimetric SAR Data Using Spectral Graph Partitioning: Utilizing Multiple Cues

23. Segmentation of Polarimetric SAR Data Using Spectral Graph Partitioning: Utilizing Multiple Cues Kaan Ersahin*, Ian Cumming and Rabab K. Ward Dept. of Electrical and Computer Engineering University of British Columbia Vancouver, Canada