Understanding Submodular Set Functions for Optimization

A [ B AÅB Submodular set functions • Set function z on V is called submodular if For all A,B µ V: z(A)+z(B) ¸ z(A[B)+z(AÅB) • Equivalent diminishing returns characterization: + ¸ + B A + S B Large improvement Submodularity: A + S Small improvement For AµB, sB, z(A [ {s}) – z(A) ¸ z(B [ {s}) – z(B)

Submodularity and convexity For V = {1,…,n}, and A µ V, let 1A = (w1A,…,wnA) with wiA = 1 if i 2 A, 0 otherwise Key result [Lovasz ’83]: Every submodular set function z induces a function g on Rn+, such that (i) z(A) = g(1A) for all A µ V (ii) g(w) is convex (iii) minA z(A) = minw g(w) s.t. w 2 [0,1]n

x{b} 2 1 x{a} -1 0 1 -2 The submodular polyhedron Pz Pz = {x 2 Rn: x(A) · z(A) for all A µ V} Example: V = {a,b} x(A) = i2 A xi x({b}) · z({b}) Pz x({a,b}) · z({a,b}) x({a}) · z({a})

xw x{b} 2 1 x{a} -1 0 1 Evaluating g(w) w g(w) = maxx2Pz wTx -2 Theorem [Edmonds ’71, Lovasz ‘83]: For any given w, can get optimal solution xw to the LP using the following greedy algorithm: • Order V={e1,…,en} so that w(e1)¸ …¸ w(en) • Let xw(ei) = z({e1,…,ei}) – z({e1,…,ei-1}) Then wT xw = g(w) = maxx2 Pz wT x Pz = {x 2 Rn: x(A) · z(A) for all A µ V}

Feasible region minc,w c s.t. c ¸ wT x for all x2Pz w2 [0,1]n Ellipsoid algorithm[Grötschel, Lovasz, Schrijver ’81] Separation oracle: Find most violated constraint: maxx wT x – c s.t. x 2 Pz Can solve separation using the greedy algorithm!!  Ellipsoid algorithm minimizes SFs in poly-time!

Minimizing submodular functions Ellipsoid algorithm not very practical Want combinatorial algorithm for minimization! Theorem [Iwata (2001)] There is a fully combinatorial, strongly polynomial algorithm for minimizing SFs, that runs in time O(n8 log2 n) Polynomial-time = Practical ???

x{b} 2 1 x{a} -1 0 1 -2 A more practical alternative?[Fujishige ’91, Fujishige et al ‘06] x({a,b})=z({a,b}) Minimum norm algorithm: • Find x* = argmin ||x||2 s.t. x 2 Bzx*=[-1,1] • Return A* = {i: x*(i) < 0} A*={a} Theorem [Fujishige ’91]: A* is an optimal solution! Note: Can solve 1. using Wolfe’s algorithm Runtime finite but unknown!!  x* Base polytope: Bz = PzÅ {x(V) = z(V)} [-1,1] M

Empirical comparison[Fujishige et al ’06] Cut functions from DIMACS Challenge Minimum norm algorithm orders of magnitude faster! Matlab implementation can solve n = 10k in < 6 minutes! Running time (seconds) Lower is better (log-scale!) Minimumnorm algorithm 64 128 256 512 1024 Problem size (log-scale!)

Understanding Submodular Set Functions for Optimization