Submodular Set Function Maximization A Mini-Survey

Submodular Set Function MaximizationA Mini-Survey Chandra Chekuri Univ. of Illinois, Urbana-Champaign

Submodular Set Functions A function f : 2NR is submodular if f(A) + f(B) ≥ f(AB) + f(AB) for all A,B N Equivalently, f(A+j) – f(A) ≥ f(A+i+j) – f(A+i) for all A N and i, j N\A

Submodular Functions • Non-negative submodular set functions f(A) ≥ 0 for all A • Monotone submodular set functions f(ϕ) = 0 and f(A) ≤ f(B) for all A  B • Symmetric submodular set functions f(A) = f(N\A) for all A

Well-known Examples • Cut functions in undirected graphs and hypergraphs (symmetric non-negative) • Cut functions in directed graphs (non-negative) • Rank functions of matroids (monotone) • Coverage in set systems (monotone) • many others ...

Cut Functions in Graphs • G=(V,E) undirected graph • f :2VR+wheref(S) = |δ(S)| S

Coverage in Set Systems • S1, S2, ..., Snsubsets of set U • f : 2{1,2, ..., n} R+ wheref(A) = | i in A Si | • w : U R+ and f(A) = w (i in A Si) • ...

Maximizing Submodular Set Functions Given f on a ground set N via a value oracle max S  Nf(S) S satisfies some constraints

Maximizing Submodular Set Functions Given f on a ground set N via a value oracle max S  Nf(S) S satisfies some constraints Motivation: • Many non-trivial applications (easy to miss!) • Generalize known results for modular functions

Unconstrained Problem max S  Nf(S) • Uninteresting for monotone f • NP-Hard for non-negative f (Max-Cut is a special case) • Very hard to approximate for arbitrary f (reduction from Set Packing)

Unconstrained Problem [Feige-Mirrokni-Vondrak’07] First O(1) approximation for non-negative f! Easy O(1) algorithms, ½ for symmetric case Non-trivial 2/5 = 0.4 approximation (slight improvement to 0.41[Vondrak])

Unconstrained Problem [Feige-Mirrokni-Vondrak’07] First O(1) approximation for non-negative f! Easy O(1) algorithms, ½ for symmetric case Non-trivial 2/5 = 0.4 approximation (slight improvement to 0.41[Vondrak]) Better than ½ requires exponential # of value queries Open Problem: Close gap between 0.41 and ½

Unconstrained Problem [Feige-Mirrokni-Vondrak’07] Random set algorithm: • pick each i in N with prob½, let R be random set • E[f(R)] ≥ OPT/4 • E [f(R)] ≥ OPT/2 for symmetric f Simple Local Search: • Initialize S to best singleton • S = local optimum for adding or deleting if improvement • Output better of S and N\S • 1/3 approx for non-negative f and ½ for symmetric f

Local Search Analysis Lemma: If S is a local optimum then for any I S or S  I, f(I) ≤ f(S). Proof: Say S  I and f(I) > f(S) then by submodularity there exists i in I\Ss.tf(S+i) > f(S). Corollary: Let S* be an optimum solution and S be a local opt. f(S S*) ≤ f(S) and f(S S*) ≤ f(S)

Local Search Analysis f(S S*) ≤ f(S) and f(S S*) ≤ f(S) f(S S*) + f(N\S) ≥f(S*\S) + f(N) ≥f(S*\S) 2f(S) + f(N\S) ≥ f(S*\S) + f(S*  S) ≥ f(S*) implies max (f(S), f(N\S)) ≥ f(S*)/3 N\S S S*

Maximizing Submodular Set Functions with Constraints max S  Nf(S) S satisfies some constraints Question: what constraints? For maximization probs, packing constraints natural SI I is a downward-closed: AI, B  A implies BI

Matroid and Knapsack Constraints • Combinatorial packing constraints • Iis the intersection of some p matroids on N • Lemma: every downward-close family I on N is the intersection of p matroids on N (for some p) • Knapsack or matrix packing constraints • A is a mxnnon-negative matrix, b is mx 1 vector • I= { x{0,1}n| A x ≤ b } • Combination of matroid and knapsack constraints

Matroid Constraints • Uniform matroid: I = { S : |S| ≤ k } • Partition matroid: I = { S : |S Ni| ≤ ki, 1 ≤ i ≤ h } where N1, ..., Nhpartition N, and ki are integers • Laminar matroid: I = { S : |S  U| ≤ k(U), U in F } for a laminar family of sets F • Graphic matroid Matroid polytope is integral and hence one can hope to capture constraints via relaxation in polytope

Monotone Functions

Cardinality Constraint max f(S) such that |S| ≤ k • Max k-Cover problem is special case • Greedy gives (1-1/e) approximation [Nemhauser-Wolsey-Fisher’78] • Unless P=NP no better approximation [Feige’98] • Many applications, routinely used

Greedy Algorithm • S =  • While |S| < kdo • iargmaxjfS(j) • S S+i • Output S

Greedy Analysis • Sj: first j elements picked by Greedy • f(S) = δ1 + δ2... +δk • δj ≥ (OPT – f(Sj-1))/k (monotonicity and submod) • f(S) ≥ (1-1/k)k OPT

A Different Analysis • S = {i1, i2, ..., ik } picked by Greedy in order • S* = {i*1, i*2, ..., i*k } an optimum solution • Form perfect matching between S and S*s.t elements in S  S* are matched to themselves S* S

A Different Analysis • Form perfect matching between S and S* Renumber S*s.tijis matched to i*j • For each j, because Greedy chose i*jinstead of i*j δj= f(Sj-1+ ij) – f(Sj-1) ≥ f(Sj-1+ i*j) – f(Sj-1) ≥ fS(i*j) • Summing up f(S) ≥ ΣjfS(i*j) ≥ fS(S*) = f(SS*) – f(S) hence f(S) ≥ f(SS*)/2 ≥ f(S*)/2 if f is monotone

Why weaker analysis? Let M=(N, I) be any matroid on N Solve max f(S) such that S I [Nemhauser-Wolsey-Fisher’78] Theorem: Greedy is ½ approximation, and analysis is tight even for partition matroids Many applications, unfortunately unknown, till recently, to approximation algorithms community!

Base Exchange Theorem • B and B’ are distinct bases in a matroidM=(N, I) Strong Base Exchange Theorem: There are elements iB\B’ and i’B’\B such that B-i+i’ and B’-i’+iare both bases. Corollary: There is a perfect matching between B\B’ and B’\B such that for each matched pair (i,i’), B-i+i’ is a base .

Proof for Greedy • B is the solution that Greedy outputs • B* is an optimum solution • B and B* are bases of M by monotonicity of f Same argument as before works by using perfect matching between B and B*

Multiple Matroids max f(S) such that S is in intersection of pmatroids [Fisher-Nemhauser-Wolsey’78] Theorem: Greedy is 1/(p+1) approximation Generalize matching argument to match one element of Greedy to p elements of OPT Also works for p-systems

Non-negative functions? max f(S) such that S is in intersection of pmatroids [Lee-Mirrokni-Nagarajan-Sviridenko’08] Theorem: For fixed p, local search based algorithm that achieves Ω(1/p) approximation. [Gupta-Roth-Schoenbeck-Talwar’09] For all p and with simple proof combining Greedy and unconstrained algorithm. Slightly worse constants.

Knapsack Constraints Similar ideas can be used with standard guessing large items etc

Can we do better? • ½ for one matroid, hardness is (1-1/e) • 1/(p+1) for pmatroids, hardness is (p/log p)

Can we do better? [Calinescu-C-Pal-Vondrak’07]+[Vondrak’08]=[CCPV’09] Theorem: There is a randomized (1-1/e) approximation for maximizing a monotone f subject to a matroid constraint.

Can we do better? [Calinescu-C-Pal-Vondrak’07]+[Vondrak’08]=[CCPV’09] Theorem: There is a randomized (1-1/e) approximation for maximizing a monotone f subject to a matroid constraint. [Lee-Sviridenko-Vondrak’09] Theorem: For fixed p≥2, there is a local-search based 1/(p+ε) approximation for intersection of p matroids New useful insight for (two) matroid intersection

Multilinear Extension of f Question: Is there a useful continuous relaxation of f such that it can be optimized? And can we round it effectively?

Multilinear Extension of f [CCPV’07] inspired by [Ageev-Sviridenko] For f: 2N R+define F:[0,1]N R+ as x = (x1, x2, ..., xn) [0,1]|N| F(x) = Expect[ f(x) ] = S Nf(S) px(S) =S Nf(S) iS xiiN\S (1-xi)

Multilinear Extension of f For f: 2N R+define F:[0,1]N R+ as F(x) =S N f(S) iS xiiN\S (1-xi) F is smooth submodular([Vondrak’08]) • F/xi ≥ 0 for all i (monotonicity) • 2F/xixj ≤ 0 for all i,j (submodularity)

Optimizing F(x) [Vondrak’08] Theorem: For any down-monotone polytope P [0,1]nmax F(x)s.tx P can be optimized to within a (1-1/e) approximation if we can do linear optimization over P Algorithm: Continuous-Greedy

Generic Approach Want to solve: max f(S) s.t S I Relaxation:max F(x)s.tx P(I ) • P(I) is a polytope that captures/relaxes S I • Can solve to within (1-1/e) with continuous greedy • How to round? F is a non-linear function

Matroid Polytope Let M=(N,I) be a matroid • P(M) is the independent set polytope of M x ≥ 0 x(S) ≤ rankM(S) for all S N • B(M) is the base polytope of M: add x(N) = rankM(N)

Rounding in Matroids Let M=(N,I) be a matroid • P(M) is the independent set polytope of M • B(M) is the base polytope of M Algorithm: • Run continuous greedy to obtain a point x B(M) such that F(x) ≥ (1-1/e) OPT • Round x to a vertex of B(M) (a base)

Rounding in Matroids [CCPV’07] Theorem: Given any point x in B(M), there is a polynomial time algorithm to round x to a vertex x* (hence a base of M) such that F(x*) ≥ F(x). “Pipage” rounding technique of [Ageev-Sviridenko] adapted to matroids

New Rounding Method [C-Vondrak-Zenklusen’09] Randomized Swap-Rounding: • Express x = mj=1βi Bi (convex comb. of bases) • B = B1 , β = β1 • For k = 2 to mdo • Randomly Mergeβ BandβkBkinto(β + βk) B’ • SetB = B’, β = (β + βk) • Output B

Merging two Bases MergeB’ and B’’ into a random B that looks “half” like B’ and “half” like B’’

Merging two Bases Base ExchangeTheorem: B’-i+j and B’’-j+i are both bases B’ B’’ i j B’B’’ B’B’’

Merging two Bases B’ B’’ i i prob ½ B’ B’’ B’B’’ B’B’’ i j B’B’’ B’B’’ B’ B’’ prob ½ j j B’B’’ B’B’’

Swap Rounding for Matroids Theorem: Swap-Merging with input x in B(M) outputs a random base B such that • E[f(B)] ≥ F(x) and • Pr[ f(B) < (1-δ) F(x)] ≤ exp(- F(x) δ2/8) (concentration for lower tail of submod functions) and • For any vector a  [0,1]n, let μ = ax then • Pr[a1B < (1-δ) μ] ≤ exp(-μδ2/2) • Pr[a1B > (1+δ) μ] ≤ ( eδ / (1+δ)δ) μ(concentration for linear functions) Almost like independent random rounding of x

Applications Can handle matroid constraint plus packing constraints x P(M) and Ax ≤ b • (1-1/e) approximation for submodular functions subject to a matroid plus O(1) knapsack/packing constraints (or many “loose” packing constraints) • Simpler rounding and proof for “thin” spanning trees in ATSP application ([Asadpour etal’10]) • ...

Other Tools On-going work [C-Vondrak-Zenklusen] • Extension of continuous greedy to handle multiple submodular functions simultaneously • Depending rounding via swap method for polyhedra • matroid intersection • matchings and b-matchings in non-bipartite graphs • ... Many applications of dependent randomized rounding[Arora-Frize-Kaplan, Srinivasan’01, ...., Asadpouretal,..]

Illustrative Application Maximum Bipartite Flow in Networks with Adaptive Channel Width [Azar-Madry-Moscibroda-Panigrahy-Srinivasan’09] • Problem motivated by capacity allocation in wireless networks • (1-1/e) approximation via a specialized LP and complicated analysis • An easy O(logn) approximation

Problem Definition Base stations Clients • θ(B,C) : threshold capacity between B and C • α(C): max flow that C desires from base stns • β(B): total capacity of B to serve clients • For each base station B, decide an operating point τ(B) ≤ β(B) • If τ(B) > θ(B,C) thenu(B,C) capacity of link (B,C) is 0, otherwise u(B,C) = τ(B) • Maximize flow from base stations to clients u(B,C) α(C) β(B)

Reduction Base stations Clients B α(C) Copies of B one for each τ(B) t u(τ(B),C) β(B) N : all copies on left f : 2N  R+where f(S) = flow from S to t Partition Matroid constraint: can pick only one copy for each B

Submodular Set Function Maximization A Mini-Survey