Ensemble Segmentation Using Efficient Integer Linear Programming

Ensemble Segmentation Using EfficientInteger Linear Programming Amir Alush and Jacob Goldberger , “ Ensemble Segmentation Using Efficient Integer Linear Programming ” , IEEE Transactions on PAMI , 2012. Ju-Hsin Hsieh Advisor : Sheng-Jyh Wang 2013/07/22

Outline • Introduction • Method • Experiment result • Conclusion • Reference

Outline • Introduction • What is segmentation? • Challenge • Main idea • Method • Experiment result • Conclusion • Reference

What is segmentation? • Partitioning of an image into several constituent components. • Assign each pixel in the image to one of the image components.

Challenge • Segmentation is not a well-defined task.

Challenge • Segmentations have different numbers of segments and are inconsistent. • How to estimate the quality of each segmentation algorithm in an unsupervised manner? 34 segments 77 segments

Main idea • Combine segmentations of the same image obtained by different algorithms. • Average of all the segmentations. • The quality of segmentation is based on the consistency of the segmentation compared to the other algorithms.

Main idea Input image 0.93 0.93 0.69 0.74 0.65 0.70 Average segmentation

Outline • Introduction • Method • Probabilistic framework - Definition - EM algorithm • Integer Linear Programming • ProcessingProcedure • Additional information • Experiment result • Conclusion • Reference

Probabilistic framework • Formalizing a clustering as a binary classification task. • Origin : A clustering of a set S = { 1 , … , n } into nc clusters • Transform : A set of n-over-2 binary decisions such that xij= 1 if i and j are in the same cluster and xij = 0 otherwise. • Transitive relation : i , j and j , k are in the same cluster.  i , k should be in the same cluster.

Probabilistic framework • An expert l (l =1,…,m)is associated with an unknown probability pl(denote by )of making the correct binary decision xij for each object pair i , j. be the judgment of the lth expert whether objects i and j are in the same cluster or not. • In order to find the unknown parameter p1,…,pm and the unobserved clustering x , we try to use EM algorithm.

Probabilistic framework • E-step : Compute marginal posterior probabilities  approximate it by computing the most likely clustering • correct object label • expert judgment

Probabilistic framework • M-step : (approximated) • correct object label • expert judgment • plreliability parameters

Probabilistic framework • correct object label • expert judgment

Integer Linear Programming • Optimization problem : • No informative prior ( maximum likelihood )  Integer Linear Programming

Integer Linear Programming • , Transitive relation If xij = xjk = 1 then xik = 1 The complexity of ILP is high.

ProcessingProcedure Negative weight Positive weight G=(V,E)with{wij} 1. Divided into “positively connected components” 2. Transform to “Single Edge Partition Tree” 3. Divided into subgraphs

ProcessingProcedure G=(V,E)with{wij} 1. Divided into “positively connected components” 2. Transform to “Single Edge Partition Tree” 3. Divided into subgraphs

ProcessingProcedure 1. Divided into “positively connected components” G( V , E ) G( V , E ) c (V1,E1) Crossing edge E12 Negative edge c (V2,E2)

ProcessingProcedure 1. Divided into “positively connected components” • Approach

ProcessingProcedure 2. Transform to “Single Edge Partition Tree” • Approach • Case 1 Cycle-free graph(tree) V1 V1 V4 V4 V5 V V2 V2 V3 V3

ProcessingProcedure 2. Transform to “Single Edge Partition Tree” • Approach • Case 2 V1 V1 V4 V4 V V2 V2 V3 V3

ProcessingProcedure 2. Transform to “Single Edge Partition Tree” • Approach • Case 3 V1 V1 V4 V4 V V2 V3 V3

ProcessingProcedure 3. Divided into subgraphs V1 V1 V4 V4 V5 V5 V2 V2 V3 V3

Additionalinformation • Image spatial consistency  Neighboring pixels are more likely to be in the same cluster than pixels that are far apart . • Approach  Use mean-shift algorithm to oversegment the image into small, homogeneous regions, known as superpixels.  Merging the MS superpixels, based on consensus among the experts.

Averaging Multiple Unreliable Segmentations ( AMUS ) AMUS 0.93 0.74 Averaging Segmentation 0.69 0.70 0.65 0.93

Averaging Multiple Unreliable Segmentations ( AMUS ) G=(V,E)with{wij} Use MS to get superpixels Divided into “positively connected components” Transform to “Single Edge Partition Tree” Divided into subgraphs Apply ILP to each subgraphs

Outline • Introduction • Method • Experiment result • AMUS algorithm • Compare with other algorithms • Conclusion • Reference

AMUS algorithm 0.62 0.74 0.73 0.87 0.95 0.89 Result Averaging segmentation

Compare with other algorithms AMUS Image CTM TBES MNC UCM • PRI(probabilistic Rand index) • VOI(Variation of information ) • GCE(Global Consistency Error) • Boundary-based F-measure

Conclusion • Segmentation is not a well-defined task. • This paper present a method for combining several segmentations of an image into a single one ( the averaging segmentation ) in order to achieve a more reliable and accurate segmentation result. • This paper also reports the reliability of each segmentation.

Reference • Amir Alush and Jacob Goldberger , “ Ensemble Segmentation Using Efficient Integer Linear Programming ” , IEEE Transactions on PAMI , 2012.

Ensemble Segmentation Using Efficient Integer Linear Programming