Geometry and Expansion: A survey of some results

1. Geometry and Expansion: A survey of some results Sanjeev Arora Princeton ( touches upon: S. A., Satish Rao, Umesh Vazirani, STOC’04; S. A., Elad Hazan, and Satyen Kale, FOCS’04; S. A., James Lee, and Assaf Naor, STOC’05 & JAMS’08 S.A., S. Kale STOC 2007. + papers that are not mine)

2. Outline: • Graph partitioning problems: intro and history • New approximation via expander flows. • New approximation algorithm via semidefinite programming (+ analysis using “Structure Theorem”) [A., Rao, Vazirani] • Outline of proof of “S. T.” • Uses of “S. T.” in geometric embeddings • Open problems

3. | E(S, Sc)| (G) = min S S S µ V |S| |S| < |V|/2 | E(S, Sc)| c(G) = min S µ V |S| c |V| < |S| < |V|/2 Sparsest Cut / Edge Expansion G = (V, E) c- balanced separator Both NP-hard

4. Why these problems are important • Analysis of random walks, PRAM simulation, packet routing, clustering, VLSI layout etc. • Underlie many divide-and-conquer graph algorithms (surveyed by Shmoys’95) • Discrete analog of isoperimetry; useful in Riemannian geometry (via 2nd eigenvalue of Laplacian (Cheeger’70) • Graph-theoretic parameters of inherent interest (cf. Lipton-Tarjan planar separator theorem)

5. 3) Embeddings of finite metric spaces into l1 • Geometric approach; more general result (but still O(log n) approximation) Previous approximation algorithms • Eigenvalue approaches (Cheeger’70, Alon’85, Alon-Milman’85)Only yield factor n approximation. 2c(G) ¸ (G) ¸ c(G)2 /2 2) O(log n) -approximationvia LP (multicommodity flows) (Leighton-Rao’88) • Approximate max-flow mincut theorems • Region-growing argument (Linial, London, Rabinovich’94, AR’94)

6. log n log n New results of [ARV’04] • O( ) -approximation to sparsest cut and conductance • O( )-pseudoapproximation to c-balanced separator (algorithm outputs a c’-balanced separator, c’ < c) • Existence of expander flows in every graph (approximate certificates of expansion) Disparate approaches from previous slide get “unified” Subsequent work: [AHK’05],[AK’07],[S’09]: O(m + n1.5 ) time!

7. The three main characters Expansion Isoperimetry (continuous analog of expansion) Geometry (and geometric embeddings of finite metric spaces)

8. Identifying sparse cuts via “traffic flows”

9. Approach 1: traffic congestion identifies sparse cuts [SM’87]: Stress a network bypassing traffic “flow” throughit. Look at congested edgesto identify sparse cuts [LR88] O(log n) approximationto sparsest cut. Route 1 unit of traffic between every pair of nodes [ARV’04] Traffic flow is like embedding a weighted graph.wij = amount of traffic from i to jSolve a math program to find the “right” flow pattern ([AHK’05] Do it in O(n2) time)

10. S Our Thm: If G has expansion , then a D-regular expander flow exists in it where D= Expander traffic flows [ARV’04] G = (V, E) A D-regular flow graph s.t. 8 S w(S, Sc) = ( D |S|) (*) S (certifies expansion = (D) ) Weighted Graph w satisfies (*) iff L(w) = (1) [Cheeger]

11. log n Formal statement : 90 >0 s.t. foll. LP is feasible for D = (G) Pij = paths whose endpoints are i, j 8i jp 2 Pij fp = D (degree) 8e 2 E p 3 e fp· 1 (capacity) 8S µ V i 2 S j 2 Scp 2 Pij fp¸0 D |S| (demand graph is an expander) fp¸ 0 8 paths p in G WHY IS THIS FEASIBLE???

12. Feasibility Criterion for LP on prev. slide(via Farkas’s Lemma) Existence of such i, j proved in [ARV’04]. When fail tofind such i, j, we find a cut of small expansion

13. Overall approximation algorithm via flows Try to solve above LP to find D-regular expander flow If succeed, have verified that expansion is > D/10. If fail, then use [ARV04] ideas to find a cut of capacity Note: Before finding this cut already had D/2-regular flow

14. Next: The SDP-based approach to Graph partitioning (ARV’04)

15. S | E(S, Sc)| c(G) = min S µ V |S| c |V| < |S| < |V|/2 Semidefinite relaxation for c-balanced separator |vi –vj|2/4 =1 |vi –vj|2 =0 +1 S -1 “cut semimetric” Find unit vectors in <n Assign {+1, -1} to v1, v2, …, vn to minimize (i, j) 2 E |vi –vj|2/4 Subject to i < j |vi –vj|2/4 ¸ c(1-c)n2 Triangle inequality |vi –vj|2 + |vj –vk|2¸ |vi –vk|28 i, j, k

16. Vi Vj Vk Unit l22 space Unit vectors v1, v2,… vn2<d |vi –vj|2 + |vj –vk|2¸ |vi –vk|28 i, j, k non obtuse ! Example: Hypercube {-1, 1}k |u – v|2 = i |ui – vi|2 = 2 i |ui – vi| = 2 |u – v|1 In fact, l2 and l1 are subcases of l22

17. log n Structure Theorem for l22 spaces [ARV’04] Subsets S and T are -separated if for every vi2 S, vj2 T |vi –vj|2¸ <d ¸  Thm: If i< j |vi –vj|2 = (n2) then 9S, T of size (n) that are  -separated for  = ( 1 )

18. log n ) |E(R, Rc)| · SDPopt / · O( SDPopt) Main thm ) O( )-approximation log n v1, v2,…, vn2<d is optimum SDP soln; SDPopt = (i, j) 2 E |vi –vj|2 S, T :  –separated sets of size (n) Do BFS from S until you hit T. Take the level of the BFS tree with the fewest edges and output the cut (R, Rc) defined by this level  d(S, j) d(S, i) (i, j) 2 E |vi –vj|2¸ |E(R, Rc)| £ j S i

19. log n Other new -approximation algorithms • MIN-2-CNF deletion and several graph deletion problems. [Agarwal, Charikar, Makarychev, Makarychev’04]. Weighted version of S.T. • MIN-LINEAR ARRANGEMENT [Charikar, Karloff, Rao’04] • General SPARSEST CUT [A., Lee, Naor ’04] • Min-ratioVERTEX SEPARATORS and Balanced VERTEX SEPARATORS[ Feige, Hajiaghayi, Lee, ’04] All use the Structure Theorem (+ other ideas)

20. T S • Outline: • Graph partitioning problems: intro and history • New approximation algorithm via semidefinite programming (+ analysis using “Structure Theorem”) [A., Rao, Vazirani] • Outline of proof of “S. T.” • Uses of “S. T.” in geometric embeddings • Introduction to expander flows and O(n2) time algorithms • Open problems (Algorithm to produce  -separatedsets S, T, of size (n) )

21. 0.01 d “Stretched pair”: vi, vj such that |vi –vj|2· and | h vi –vj, u i | ¸ 0.01 d Algorithm to produce two  –separated sets <d Easy: Su and Tu likely to have size (n) u Tu Delete any vi2 Su, vj2 Tu s.t. |vi –vj|2 < . (till no such pair remains) Su If Su, Tu still have size (n), output them Main difficulty: Show that whp only o(n) points get deleted Obs: Deleted pairs are stretched and they form a matching.

22. -t2 /2 e 1 1 d d = O( 1 ) Stretched pair: |vi –vj|2 < ; |<vi –vj, u>| > 0.01 d  Naïve analysis of random projection fails v <d u <u, v> ?? standard deviations E[# of stretched pairs] = n2 exp(-) À n

23. Vi 0.01 d Proof by contradiction: Suppose matching of (n) size exists with probability (1)… ….stretched pairs are almost everywhere you look! Vj u Ball (vi , ) Idea: Put stretched pairs together; derive very improbable event

24. Vi Vj Vk s s s s Walks in unit l22 space Unit vectors v1, v2,… vn2<d |vi –vj|2 + |vj –vk|2¸ |vi –vk|28 i, j, k Angles are non obtuse Taking r steps of length s only takes you squared distance rs2 (i.e. distance r s)

25. r steps of length s ) squared distance rs2 (distance r s) s s 0.01 <vfinal –v0, u> ¸ r 0.01 0.01 d d d s s r Projection = £ standard deviation  Proof by contradiction (contd.) Claim: 9walk on stretched edges VERY UNLIKELY IF r large enough) Walk impossible (CONTRADICTION) Stretched pair: |vi –vj|2 < ; |<vi –vj, u> ¸ 0.01 d    …. u Why walk is possible: delicate argument; measure concentration |vfinal –v0|· r 

26. Outline: • Graph partitioning problems: intro and history • New approximation algorithm via semidefinite programming (+ analysis using “Structure Theorem”) [A., Rao, Vazirani] • Outline of proof of “S. T.” • Geometric embeddings of metric spaces • Open problems

27. <k(with l2 norm) Finite metric space (X, d) f(x) y f d(x,y) x f(y) distortion of f is minimum C>1 such that d( x, y) · |f(x ) – f( y)|2·C d( x, y) 8 x, y Thm (Bourgain’85): For every n-point metric space, a map exists with distortion O(log n) [LLR’94]: Efficient algorithm to find the map; Proof that O(log n) cannot be improved in general Qs: Improve O(log n) for X = l22 (say) or l1 ?

28. Embeddings and Cuts (LLR’94, AR’94) Recall: Cut semi-metric Fact: Metric (X, d) embeds isometricallyin l1 iff it can be written as a positive combination of cut semimetrics 1 0 Embedding l22 into l1 gives a way to produce cuts from SDP solution

29. Status report of this area Best upperbound Best lowerbound Disproves Goemans-Linial conjecture log n [Bourgain’85] Uses fourier techniques developed for PCPs! log0.75 n [Chawla,Gupta,Racke ’04] Exactly the integrality gap of SDP for general SPARSEST CUT [LLR’94, AR’94] log0.5 n log log n [A., Lee, Naor’04] Uses new metric differentiation techniques

30. x Ai Upperbounds:Frechet’s recipe to embed metric space (X, d) into Rk Pick k suitable subsets A1, A2, …, Ak of X Map x 2 X to (d(x, A1), d(x, A2), … , d(x, Ak)) Note: d(x, A1) – d(y, A1) · d(x, y) In recent embeddings, Ai’s are chosen using S.T.and “Measured descent” idea of [Krauthgamer, Lee, Naor, and Mendel’04]

31. Embedding lowerbounds (Khot-Vishnoi’05) Explicit unit- l22 space (X, d) that requires distortion log log log n into l1 Main observation: Need good handle on cut structure of X Use hypercube as building block ! Cut ´ Boolean Function Number of cut edges = average sensitivity (Fourier analysis a la KKL, Friedgut, Hastad, Bourgain etc. ) isoperimetric theorems)

32. OPEN PROBLEMS • Better approximation factor than O( )? (log log n “lowerbound” assuming UGC ) • Better distortion bound for embedding l22 into l1?.) ( upperbound v/s lowerbound • Combinatorial approximation algorithms for other problems ? (similar to one for SPARSEST CUT from [A., Hazan, Kale] ) • Other applications of expander flows? (Useful in some geometric results [Naor, Rabani, Sinclair’04]) • Ways to use spectral ideas a la [ABS’10] for SPARSEST CUT?

33. Example of expander flow n-cycle Take any 3-regular expander on n nodes Put a weight of 1/3n on each edge Embed this into the n-cycle Routing of edges does not exceed any capacity ) expansion =(1/n)

34. Other extensions of flow-based techniques • Generalization to problems other than sparsest cut [A., Kale07] “Primal-dual approach to SDP.” • Very fast algorithms for O(log n) approximation: O(n1.5 + m) time (Faster than [LR88] type algorithms!) • Very simple algorithms; use only maxflow and eigenvalue computations [KRV06]

35. Looking forward to more progress… Thanks !

36. ) (D) ·(G) ·O(D ) log n log n New Result (A., Hazan, Kale;FOCS’04) O(n2) time algorithm that given any graph G finds for some D >0 • a D-regular expander flow • a cut of expansion O( D ) Ingredients: Approximate eigenvalue computations; Approximate flow computations (Garg-Konemann; Fleischer) Random sampling (Benczur-Karger + some more) Idea: Define a zero-sum game whose optimum solution is an expander flow; solve approximately using Freund-Schapire approximate solver.

37. Expander flows: LP view · 1 LP feasible )¸(D) · D Thm [ARV]:90 s.t. the LP is feasible with D = /√log n G

38. Open problems (circa April’04) O(n2) time; [A., Hazan, Kale] • Better running time/combinatorial algorithm? • Improve approximation ratio to O(1); better rounding??(our conjectures may be useful…) • Extend result to other expansion-like problems (multicut, general sparsest cut; MIN-2CNF deletion) • Resolve conjecture about embeddability of l22 into l1; of l1 into l2 • Any applications of expander flows? Integrality gap is (log n) [Charikar] log3/4 n distortion; [Chawla,Gupta, Racke] Yes [Naor,Sinclair,Rabani] Better embeddings of lp into lq [Lee]

39. Various new results O(n2) time combinatorial algorithm for sparsest cut (does not use semidefinite programs) [A., Hazan, Kale’04] New results about embeddings: (i) lp into lq[J. Lee’04] (ii) l22 and l1 into l2[CGR’04] (approx for general sparsest cut) Clearer explanation of expander flows and their connection to embeddings [NRS’04]

40. A concrete conjecture (prove or refute) G = (V, E);  = (G) For every distribution on n/3 –balanced cuts {zS} (i.e., SzS =1) there exist (n) disjoint pairs (i1, j1), (i2, j2), ….. such that for each k, • distance between ik, jk in G is O(1/ ) • ik, jk are across (1) fraction of cuts in {zS} (i.e., S: i 2 S, j 2 Sc zS = (1) ) Conjecture ) existence of d-regular expander flows for d = 

41. log n log n

42. Example of l22 space: hypercube {-1, 1}k |u – v|2 = i |ui – vi|2 = 2 i |ui – vi| = 2 |u – v|1 In fact, every l1 space is also l22 Conjecture (Goemans, Linial): Every l22 space is l1 up to distortion O(1)

43. 1 0 0 1 1 Semidefinite LP Relaxations for c-balanced separator Min (i, j) 2 E Xij 0 · Xij· 1 Motivation: Every cut (S, Sc) defines a (semi) metric Xij2 {0,1} Xij + Xj k¸ Xik  i< j Xij¸ c(1-c)n2 There exist unit vectors v1, v2, …, vn2<n such that Xij = |vi - vj|2 /4

44. Semidefinite relaxation (contd) Min (i, j) 2 E |vi –vj|2/4 |vi|2 = 1 |vi –vj|2 + |vj –vk|2¸ |vi –vk|28 i, j, k i < j |vi –vj|2¸ 4c(1-c)n2 Unit l22 space Many other NP-hard problems have similar relaxations.

45. If any vi2 Su and vj2 Tu satisfy |vi –vj|2·, delete them and repeat until no such vi, vj remain 0.01 d “Stretched pair”: vi, vj such that |vi –vj|2· and | h vi –vj, u i | ¸ 0.01 d Algorithm to produce two  –separated sets <d Check if Su and Tu have size (n) u Tu Su If Su, Tu still have size (n), output them Main difficulty: Show that whp only o(n) points get deleted Obs: Deleted pairs are stretched and they form a matching.

46. T S Next 10-12 min: Proof-sketch of Structure Thm ( algorithm to produce  -separated S, T of size (n);  = 1/ )

47. “Stretched pair”: vi, vj such that |vi –vj|2· and | h vi –vj, u i | ¸ 0.01 d O( 1 ) £ standard deviation  = exp( - ) log n ) PrU [ vi, vj get stretched] = exp( - 1 )  E[# of stretched pairs] = O( n2 ) £ exp(- ) logn “Matching is of size o(n) whp” : naive argument fails

48. |vfinal - vi| < r  = O( r ) x standard dev. | <vfinal – vi, u>| ¸ 0.01r 0.01 0.01 0.01 r d d d  d Generating a contradiction: the walk on stretched pairs Contradiction if r is large enough! Vj vfinal    Vi r steps u

49. Reason: Isoperimetric inequality for spheres  Measure concentration (P. Levy, Gromov etc.) <d A : measurable set with (A) ¸ 1/4 A : points with distance · to A A (A) ¸ 1 – exp(-2 d) A

50. Expander flows (approximate certificates of expansion)