Fast Approximate Energy Minimization via Graph Cuts

Fast Approximate EnergyMinimization via Graph Cuts Presenter：Kuang-Jui Hsu Date ：2011/3/24(Thur.)

Outline 1.Important Journal & Conference 2.Paper Outline

Important Journal & Conference Journal Rankings: 1. TPAMI:IEEE Transactions on Pattern Analysis and Machine Intelligence 2. IJCV: International Journal of Computer Vision Conference Rankings : 1. CVPR: IEEE Conference on Computer Vision and Pattern Recognition 2. ICCV: International Conference on Computer Vision 3. ECCV: European Conference On Computer Vision

Paper Outline 1. Energy minimization in early vision 2. Energy minimization via graph cuts 3. Finding the optimal swap move 4. Finding the optimal expansion move 5. Experimental results

Energy minimization in early vision • Goal: find a labeling f that assigns each pixel wheref is both piecewise smooth and consistent with the observed data. • Energy function: : the extent to which is not piecewise smooth. : the disagreement between f and the observed data. • Date energy: : how appropriate a label is for the pixel p given the observed data Ɲ: the set of pairs of adjacent pixels • In this paper, authors allow to be arbitrary

Energy minimization in early vision • Smooth energy: a critical issue, and many different function have been proposed • Major difficult: the enormous computation cost, but typically these energy functions have many local minima. • Simulated annealing was popularized in computer vision and is widely used since it can optimize an arbitrary energy function. • Minimizing an arbitrary requires exponential time, and as consequence simulated annealing is very slow. • Annealing is inefficient partly because at each step it changes the value of a single pixel. • In special cases such energies can be minimized exactly, but the problem is NP-hard in general.

Energy minimization in early vision • Authors develop algorithms that approximately minimize energy E(f) for an arbitrary finite set of labels under two fairly general classes of interaction potentials V : a) Semi-metric: on the space , for any pair of labels α, β , it satisfies two properties. 1. 2. b) Metric: the condition of the semi-metric plus the triangle inequality for any α, β , γ in • Compute a labeling which is a local minimum even when very large moves are allowed. • Move : gives an arbitrary label αto a single pixel

Energy minimization in early vision • The authors’ algorithms are based on graph cuts. • The authors’ two algorithms : a) First algorithm: based on α-β-swap moves and works for any semi-metric b) Second algorithm: based on α-expansion moves but works only for metric • Note that α-expansion moves produce a solution within a known factor of the global minimum of E

Energy minimization via graph cuts • Partitions and moves • Algorithms and properties • Graph cuts

Partitions and moves • Any labeling f can be uniquely represented by a partition of image pixels P= , where is a subset of pixels assigned label l. • One to one correspondence between labelingsf and partitions P • α-β swap: given a pairs of labels α,β , a move from a partition P (labeling f ) to a new partition (labeling ) if • Difference between P and : some pixels that were labeled α in P are now labeled β in , and some pixels that were labeled β in P are now labeled α in . • α-expansion: given a label α , a move from a partition P (labeling f ) to a new partition (labeling ) • An α-expansion move allows any set of image pixels to change their labels to α.

Algorithms and properties α-expansion move algorithm α-β swapmove algorithm

Algorithms and properties • Three important properties: • Guarantee to terminate in a finite number of cycles; Under the fairly general assumptions, terminate in cycles. • Once the algorithm has terminated, the energy of the resulting labeling is a local minimum with respect to a swap or an expansion move. • Expansion move algorithm produces a labeling f such that where is the global minimum and

Graph cuts • Key part: step 3.1, where graph cuts are used to efficiently find • Let be a weighted graph with two distinguished vertices called terminals. • A cut is a set of edges such that the terminals are separated in the induced graph • The cost of cut C, denoted |C|, equals the sum of its edge weights. • Step 3.1 uses a single minimum cut on a graph whose size is . α, β are terminals. , and are weighted edges

Finding the optimal swap move • Critical step: Given an input labeling f (partition P) and a pair of labels α, β, we wish to find a labeling that minimizes E over all labelings within one α- β swap of f . • The technique is based on computing a labeling corresponding to a minimum cut on a graph

Finding the optimal swap move 1.include two terminals α, β 2. 3.The set consists of α, β 4. 5.Each pixel is connected to the terminals α, β by edge 6.t-links: terminal links( ) 7.n-links: neighbor links, each pair of pixels which are neighbor connected by an edge 8.The set of edges thus consists of (the t-links) and (the n-links)

Finding the optimal swap move • Weights assigned to the edges are

Finding the optimal swap move • Any cut C on must sever (include) exactly one t-link for any pixel • Define a natural labeling corresponding to a cut C on • Lemma 1: A labeling corresponding to a cut on is one α-β swap away from the initial labeling f.

Finding the optimal swap move • Property 1: For any cut C and for any n-links

Finding the optimal swap move • Lemma 2 : For any cut C and for any n-link

Finding the optimal swap move • Theorem1: There is a one to one correspondence between cuts C on and labelings that are one α-β swap from f . Moreover, the cost of a cut C on is |C| = plus a constant. Proof : The severed t-linksuniquely determine the labels assigned to pixels p and n-linksthat must to be cut. Compute the cost of a cut C |C|=

Finding the optimal swap move • Corollary 1 : The optimal α-β swap from f is where C is the minimum cut on a graph

Finding the optimal expansion move • Given an input labeling f (partition P) and a label α , we wish to find a lebeling that minimizes E over all labelings within one α-expansion of f . • Solve the problem assuming each is metric, and thus satisfies the triangle inequality. • Based on computing a labeling corresponding to a minimum cut on a graph

Finding the optimal expansion move • The structure of graph • The set of vertices includes the two terminals α and • as well as all image pixels • For each pair of neighboring pixels separated • in the current partition(i.e. ), create an • auxiliary vertex • Auxiliary nodes are introduced at the boundaries between • partition sets . • The set of vertices : • Each pixel is connected to the terminals α and • by t-links , correspondingly. • Each pair of neighboring pixels which are • not separated by the partition P (i.e. ) is • connected by an n-link . • For each pair of neighboring pixels {p , q} such • that ,create a triplet of edges • Where a = is the corresponding auxiliary node. • The edges and connected pixels p and q to • and the t-link connects the auxiliary node • to the terminal • The set of all edges

Finding the optimal expansion move This defines a natural labeling corresponding to a cut C on The weights:

Finding the optimal expansion move • Any cut C on must sever(include) exactly one t-link for any pixel • Lemma 3A labeling corresponding to a cut C on is one α-expansion away from the initial labeling f.

Finding the optimal expansion move Assume that is an auxiliary node between the corresponding pair of neighboring pixels. Property 2 A minimum cut C on satisfies:

Finding the optimal expansion move • Lemma 4 If and then the minimum cut C on satisfies ( ) • Property 1 holds when we substitute for . We will refer to this as property 1( ). • property 1( ) : • Property 1( ) holds for any cut, and Property 2 holds for a minimum cut. • Elementary cut on : A cut that satisfies Properties 1( ) and 2.

Finding the optimal expansion move • Theorem 2 Let be constructed as above given f and α. Then there is a one to one correspondence between elementary cuts on and labelings within one α-expansion of f. Moreover, for any elementary cut C we have Proof : The label at the pixel p determines which of the t-links to p is in C . Property 1( ) shows which n-links between pairs of neighboring pixels {p , q} such that should be severed. Similarly , Property 2 determines which of the links in corresponding to {p , q} such that should be cut. The cost of an elementary cut C : For any pixel =>

Finding the optimal expansion move Lemmas 2 and 4 hold for elementary cuts, since they were based on properties 1 and 2 => => The minimum cut is an elementary cut. • Corollary 2 The optimal α-expansion from f is where C is the minimum cut on .

Experimental results • Use three energy function, each with a quadratic • The First energy function , called E1, uses the truncated quadratic as its smoothness term. • The second(E2) and third (E3) energy function use the Potts model and the truncated L2 distance, correspondingly, as their smoothness penalty V. Potts model: Truncated distance:

Experimental results Simulated annealing Authors’ algorithm

Experimental results

Fast Approximate Energy Minimization via Graph Cuts