Hierarchical Graph Cuts for Semi-Metric Labeling

Hierarchical Graph Cutsfor Semi-Metric Labeling M. Pawan Kumar Joint work with Daphne Koller

Aim To obtain accurate MAP estimate for Semi-Metric MRFs efficiently V1 V2 … … … … … … … … … … … … … … … … … Vn Random Variables V = { V1, V2, …, Vn}

Aim To obtain accurate MAP estimate for Semi-Metric MRFs efficiently lj ab(i,j) a(i) : arbitrary li ab(i,j) = sab d(i,j) sab ≥ 0 b(j) a(i) Vb Va d( i , i ) = 0 for all i d( i , j ) = d( j , i ) > 0 for all i≠j Semi-metric Distance Function d( i , j ) - d( j , k ) ≤ d( i , k ) Metric Distance Function

Aim To obtain accurate MAP estimate for Semi-Metric MRFs efficiently lj ab(i,j) a(i) : arbitrary li ab(i,j) = sab d(i,j) sab ≥ 0 b(j) a(i) Vb Va f* = arg minf a(f(a)) +  ab(f(a),f(b))

Visualizing Metrics l1 w2 w1 l2 l5 w3 w8 w7 w9 w4 w6 l4 l3 w5 d( i , j ) : shortest path defined by the graph

Overview + f1 f2 f

Outline • Simpler Metrics • Labeling for Simpler Metrics • Approximating General Metrics/Semi-Metrics • Combining Labelings • Results

r-HST Metrics w1 w1 w2 w2 w3 w3 w2 w3 l1 l2 l3 l4 l5 l6 Graph is a Tree. Labels are leaves Edge lengths for all children are the same

r-HST Metrics w1 w1 w2 w2 w3 w3 w2 w3 l1 l2 l3 l4 l5 l6 Edge lengths decrease by factor r ≥ 2 w2 ≤ w1/r w3 ≤ w1/r

r-HST Metric Labeling w1 w1 w2 w2 w3 w3 w2 w3 l1 l2 l3 l4 l5 l6 r-HST Metrics admit Divide-and-Conquer Divide original problem into subproblems

r-HST Metric Labeling w1 w1 w2 w2 w3 w3 w2 w3 l1 l2 l3 l4 l5 l6 Subproblem defined at vertex ‘m’ f* = arg minf a(f(a)) +  ab(f(a),f(b)) such that f(a)  m

r-HST Metric Labeling w1 w1 w2 w2 w3 w3 w2 w3 l1 l2 l3 l4 l5 l6 Trivial problem f* = arg minf a(f(a)) +  ab(f(a),f(b)) such that f(a)  { l4}

r-HST Metric Labeling w1 w1 w2 w2 w3 w3 w2 w3 l1 l2 l3 l4 l5 l6 Original problem f* = arg minf a(f(a)) +  ab(f(a),f(b)) such that f(a)  { l1, …, l6}

r-HST Metric Labeling Problems get tougher as we move up Solve the simple subproblems (starting with trivial subproblems) Use their solutions to solve difficult subproblems

f2 f3 f1 r-HST Metric Labeling w w w Find new labeling using -Expansion

f2 f3 f1 r-HST Metric Labeling w w w Continue till we reach the root

Analysis w w m w Mathematical Induction All variables Va such that f*(a)  m 1 bound on the unary potentials 2r/(r-1) bound on the pairwise potentials

Analysis w w m w Mathematical Induction Initial step of M.I. trivial (for leaf nodes) Given children, prove for parent

Analysis w w w  a(f(a)) + f(a) = fi(a) f(b) = fi(b) i  ab(fi(a),fi(b)) + f(a) = fi(a) f(b) = fj(b) i≠j  ab(fi(a),fj(b))

Analysis w w w  a(f*(a)) + i  ab(fi(a),fi(b)) + i≠j  ab(fi(a),fj(b))

Analysis w w w  a(f*(a)) + 2r i  ab(f*(a),f*(b)) + r-1 i≠j  ab(fi(a),fj(b))

Analysis w w w  a(f*(a)) + 2r i  ab(f*(a),f*(b)) + r-1 dmax i≠j  ab(f*(a),f*(b)) 2 dmin

Analysis w w w dmin = 2w dmax = 2w(1+1/r+1/r2+….)

Analysis w w w  a(f*(a)) + 2r i  ab(f*(a),f*(b)) + r-1 2r i≠j  ab(f*(a),f*(b)) r-1

Analysis Overall approximation bound 2r/(r-1) Previous best bound 2r/(r-2) Not Tight ?

minD,Pr(.) maxi≠j ∑tPr(t) dt(i,j) d(i,j) Approximating Metrics Given distance d(.,.) D = {dt(.,.), t = 1,2,… T}, dt(i,j) ≥ d(i,j) Pr(.) over the elements of D

Approximating Metrics w1 w1 w2 w2 w3 w3 w2 w3 l1 l2 l3 l4 l5 l6 r-HST : hierarchical clustering of labels Use a clustering algorithm

Approximating Metrics Fakcharoenphol, Rao and Talwar, 2003 max d(i,j) = 2M mini≠j d(i,j) > 1 Sample   [1,2] Choose a permutation π of labels Level ‘1’ = { l1,…, lh} Level ‘2’ Clustering at level 2??

Level ‘m-2’ Level ‘m-1’ Approximating Metrics Fakcharoenphol, Rao and Talwar, 2003 max d(i,j) = 2M mini≠j d(i,j) > 1 Sample   [1,2] Choose a permutation π of labels Clustering at level m??

Approximating Metrics Fakcharoenphol, Rao and Talwar, 2003 l1 l2 l3 d(1,4) ≤ 2M-m ? π l4 l1 l3 l2 l6 l5

Approximating Metrics Fakcharoenphol, Rao and Talwar, 2003 l2 l3 d(2,4) ≤ 2M-m ? π l4 l1 l3 l2 l6 l5 l1

Approximating Metrics Fakcharoenphol, Rao and Talwar, 2003 l2 l3 d(2,1) ≤ 2M-m ? π l4 l1 l3 l2 l6 l5 l1

Approximating Metrics Fakcharoenphol, Rao and Talwar, 2003 l3 d(3,4) ≤ 2M-m ? π l4 l1 l3 l2 l6 l5 l1 l2

Approximating Metrics Fakcharoenphol, Rao and Talwar, 2003 Edge length = Diameter of cluster / 2 π l4 l1 l3 l2 l6 l5 l3 l1 l2

Approximating Metrics Fakcharoenphol, Rao and Talwar, 2003 Choose . Choose π Initialize root node as trivial cluster (all labels) Choose a cluster at level m-1 Run procedure to get clusters at level m Repeat for all clusters at level m-1 Stop when all clusters are singletons Repeat to get a set of r-HST metrics

Analysis Fakcharoenphol, Rao and Talwar, 2003 d(i,j) ≤ ∑Pr(t) dt(i,j) ≤ O(log h) d(i,j) How many r-HST metrics ?? O(h log h) Charikar, Chekuri, Goel, Guha and Plotkin, 1998

Approximating Semi-Metrics d(i,j) - d(j,k) ≤  d(i,k) d(i,j) ≤ ∑Pr(t) dt(i,j) ≤ O(( log h)2) d(i,j) How many r-HST metrics ?? O(h log h)

Combining Labelings Use -Expansion !!

Analysis Bound for r-HST Labeling = O(1) Distortion for Metrics = O(log h) Bound for Metric Labeling = O(log h) Distortion for Semi-Metrics = O(( log h)2) Bound for Semi-Metric Labeling = O(( log h)2)

Analysis When h < n, all known LP bounds can be obtained using move making algorithms.

Refining the Labeling Current energy Q(f; d) = Q(f; dt) Q(f’; d) ≤ Q(f’; dt), f’ ≠ f Fakcharoenphol, Rao and Talwar, 2003 Find best ft according to dt(.,.) r-HST Metric Labeling f = ft. Repeat till convergence.

Synthetic Data

Image Denoising

Hierarchical Graph Cuts for Semi-Metric Labeling