Algorithms for Max-min Optimization

1. Algorithms for Max-min Optimization Anupam Gupta Carnegie Mellon University based on works with Viswanath Nagarajan (IBM Watson) and R.Ravi (CMU), Aaron Roth (MSR/Penn), Grant Schoenebeck (Princeton), and Kunal Talwar (MSR)

2. maximization problems Given: universe U of elements max s.t. S in I f(S) collection I of “independent” subsets “value” function f:2U non-negative values When can we solve this? Example #1: Universe = vertices of graph “independent” set = no edge between two vertices f(S) = v in Swv max-weight independent set

3. maximization problems Given: universe U of elements max s.t. S in I f(S) collection I of “independent” subsets “value” function f:2U non-negative values When can we solve this? Example #2: Universe = edges of graph “independent” set = no cycles within the edges f(S) = e in Swe max-weight spanning tree

4. general outline of this talk A quick survey on linear and submodular maximization I.e., when f(S) is either linear or submodular. What is max-min optimization? I.e., cases where f(S) is defined by a minimization (covering) problem. A couple of general theorems about these topics, the ideas behind some of the algorithms, and a couple of simple proofs. max s.t. S in I f(S)

5. case I: linear fn maximization max s.t. S in I linear case: f(S) = e in Swe f(S) If Iforms a matroid, we can solve this exactly! recall: a family of independent sets I forms a matroid if a) it is closed under taking subsets b) if A and B independent and |A| < |B|, there is an element e in B such that A+e independent. In fact, the greedy algorithm suffices… e.g., max-weight spanning tree

6. case I: linear fn maximization max s.t. S in I linear case: f(S) = e in Swe f(S) If I is the intersection of two matroids, also solvable e.g., max-weight bipartite matching max-weight arborescence The greedy algorithm is not the best any more… … but it is a 2-approximation

7. case I: linear fn maximization max s.t. S in I linear case: f(S) = e in Swe f(S) If I is the intersection of p > 2 matroids, NP hard in general But the greedy algorithm is still a p-approximation! can do slightly better via a local-search algorithm…

8. case II: submodular maximization max s.t. S in I f(S) submodular function f Recall f is submodular if f(subset+ e) – f(subset) ≥ f(superset+ e) – f(superset) f is monotone if f(subset) ≤ f(superset)

9. two quick examples Given a collection of sets A1, A2, …, Am universe = {1, 2, …, m} f({1, 3, 7}) = | A1[A3[A7 | Given an undirected graph G = (V,E) universe = V f(S) = # of edges going from S to V\S in the graph. The “set-coverage” function is monotone submodular. The cut function in graphs is (non-monotone) submodular.

10. case II: submodular maximization max s.t. S in I f(S) monotone submodular function f If I is the intersection of p matroids the greedy algorithm is a (p+1)-approximation! S = { } While e such S + e is independent pick the element e* which maximizes f(S+e*) – f(S)

11. the greedy algorithm: a proof Suppose we want to maximize monotone submodular f(S) such that I = { S ½ U | S contains at most k elements } Theorem: f(Sfinal) ≥ ½ f(Sfinal[ OPT) Proof: Suppose not. Cardinality constraint ≥ ½ f(OPT) f(Sfinal) f(Sfinal) n.b. use monotonicityonly in the last step. k k Then f(Sfinal) < ½ f(Sfinal[ OPT)  f(Sfinal[ OPT) – f(Sfinal) > f(Sfinal)   e in OPT\S s.t. f(Sfinal + e) – f(Sfinal) >  at every step of greedy,f(S + e) – f(S) > So greedy’s improvement must have been at least as much. Contradiction

12. case II: submodular maximization max s.t. S in I f(S) non-monotone submodular function f If I is 2U (there are no constraints at all) already the problem is APX-hard! However, the following algorithm is an 4-approximation: S = a random subset of the universe U. again, can do better (won’t talk about it today)…

13. case II: submodular maximization max s.t. S in I f(S) non-monotone submodular function f If I is the intersection of p matroids the greedy algorithm is not good! However, the following algorithm is an O(p) approximation: S1← Greedy(U) Find the subset T of S1 maximizing f(T) S2← Greedy(U\S1) Return the better of T and S2

14. the greedy algorithm: non-monotone proof for cardinality constraints f(S1) ≥ ½ f(S1[ OPT) (from analysis of monotone case) f(T) ≥ ¼ f(S1\ OPT) (from unconstrained max.) f(S2) ≥ ½ f(S2[ (OPT \ S1)) (from monotone case again) ≥ f(OPT) max{ f(T) , f(S2) } ≥ f(OPT)/12 S1← Greedy(U) Find the subset T of S1 maximizing f(T) S2← Greedy(U\S1) Return the better of T and S2

15. quick recap max s.t. S in I f(S) If f is submodular If I is the intersection of p matroids then we can get an O(p) approximation. what can we do for more general functions f( )? I = { S ½ U, |S| ≤ k } for the rest of the talk, just focus on cardinality constraints (some of our results will extend to more general constraints)

16. max-min covering problems Fix a graph G universe = set of nodes fST(S) = min-cost Steiner tree on S fMC(S) = min-cost cut separating S from root r Fix a collection of sets A1, A2, …, Am fSC(S) = size of min-cost set cover on S using these sets Ai max s.t. |S|≤ k f(S) “Which k nodes is most expensive to connect up”? “Which k nodes are most expensive to disconnect from r”? “Which k elements are most expensive to cover”? Of course, if f(S) is submodular (or close), we can use previous results… Can prove, e.g., fSC is very far from any submodular function

17. max-min covering: results (1) For general covering problems… Theorem: if you can solve some covering problem offline and you can solve the problem online (deterministically)  you can solve the max-min version of the problem. Usually offline × online approximation. max s.t. |S|≤ k f(S) These algos extend to intersections of p matroids and q knapsacks…

18. max-min covering: results (2) For the cardinality max-min: O(log m + log n)-approximation for set cover O(1)-approximation for Steiner tree/forest O(log2 n)-approximation for multicut all using the “same” algorithmic idea… max s.t. |S|≤ k f(S)

19. what’s the algorithm? “cost(S)” Recall: want to find S with |S| ≤ kthat maximizes f(S) Generic Idea: call this value T* For each value T do the following: While some element costs “much more” than T/k to satisfy, add it to a set XT Talg = smallest T such that cost(XT) not “much more” than T For proof, need to show that a) for T ≈ T*, we have cost(XT) ≈ T* b) for T a bit smaller than T*, XT contains S of size k, such that cost(S) > T.  all k-sets cost < O(T*)  exists a k-set with cost ≥ (T*)

20. rest of the talk Generic Algorithm: Show how generic algorithm applies to: • maxmin Steiner tree • (time permitting) maxmin set cover For each value T do the following: While some element costs “much more” than T/k to satisfy, add it to a set XT Talg = smallest T such that cost(XT) not “much more” than T

21. Steiner tree Given a metric space (V, ℓ) and root r and a “scenario” S of terminals find the least cost network connecting S to r Results: MST is a 2-approximation. polytime 1.39-approx. algo[ByrkaGrandoniRothvossSanita ’10] APX-hard [Bern Plassmann ‘89]

22. maxmin Steiner tree Given a metric space (V, ℓ) and root r find S approximately achieving max s.t. |S|≤ k SteinerTree(S)

23. the precise algorithm Algorithm: For each value T do: • add the root r to XT. • while exists a vertex v such that distance(v, XT) > 4T/k add v to XT find smallest T such that MST(XT) < 4T. Fact: cost of connecting any set of k guys to MST(XT) ≤4T. Corollary: cost of connecting any set of k guys ≤4T + MST(XT).

24. the analysis(1) proof thatany k-subset can beconnected at cost < 8T* Theorem 1: For T = T*, cost of MST(XT) < 4T. Fact: cost of connecting any k guys ≤4T + MST(XT). Lemma 2: For T=T*/8, cost of MST(XT) ≥ 4T. Proof: if not, we could connect up any k points at cost < 8T = T*. if |XT| ≤ k, it is a set with cost ≥ 4T = T*/2 and small size have found k-subset with cost > T*/4 if |XT| is large, pick any k points from it. their mutual distance > 4T/k, so Steiner-tree cost > 2T = T*/4.

25. remains to prove… Theorem 1: For T = T*, cost of MST(XT) < 4T. optimal tree on XT ≤ |XT| × 4T*/k × ½ ≤ |XT|/k × T* ≤ (|XT|/k+ 1) × T* ≤ 2T* |XT| × T*/k ≤ T*  |XT| ≤ k MST is a 2-approximation, hence the 4T*

26. wrapping up maxmin Steiner Algorithm: For each value T do: • add the root r to XT. • while exists a vertex v such that distance(v, XT) > 4T/k add v to XT Talg = smallest T such that MST(XT) < 4T. This value Talgshows all k-subsets can be connected at cost O(Talg) the set XTfor Talg/2gives a k-subset of cost (Talg) so, constant approximation.

27. rest of the talk Generic Algorithm: Show how generic algorithm applies to: • maxmin Steiner tree • (time permitting) maxmin set cover For each value T do the following: While some element costs “much more” than T/k to satisfy, add it to a set XT Talg = smallest T such that cost(XT) not “much more” than T

28. to conclude studied constrained maximization problems good algorithms for submodular cases algorithms for max-min problems (f = minimization covering problem) O(log m + log n)-approximation for Set Cover also for Steiner tree, Steiner forest, min-cut, multicut “same” simple algorithm “dual-rounding” analysis for which other functions can we do constrained maximization well? papers are on the arxiv thanks!! max s.t. S in I f(S)