Expander Flows, Graph Spectra and Graph Separators

1. Expander Flows, Graph Spectra and Graph Separators Umesh Vazirani U.C. Berkeley Based on joint work with Khandekar and Rao and with Orrechia, Schulman and Vishnoi

2. Graph Separators S T Sparsest Cut/Edge Expansion: c-Balanced Separator:

3. Applications • Clustering • Image segmentation • VLSI layout • Underlie many divide-and-conquer graph algorithms

4. Interesting Techniques • Spectral methods. Connection to differential • geometry, discrete isoperimetric inequalities. • Linear/semidefinite programming • Measure concentration • Metric embeddings

5. Geometrical view • Map vertices to points in some abstract space: • - points well-spread • - edges short

6. Geometrical view • Map vertices to points in some abstract space: • - points well-spread • - edges short • “Good bisection” of the space yields sparse cut in graph

7. Spectral Method Cut at random Minimize sum of “edge lengths”: Spread out vertices: [Cheeger’70] [Alon, Milman ’85][Jerrum, Sinclair’89]

8. Leighton-Rao ‘89 1 4 1 1 5 2 1 2 1 2 5 2 2 4 1 5 Cut along ball of random radius Minimize sum of “edge lengths”: Spread out vertices: Distances form a metric: satisfy triangle inequality. wij + wjk >= wik O(log n) approximation: Approximate max-flow min-cut thm for multi-commodity flows.

9. ARV ‘04 Unit sphere in Rd No angles obtuse Unit L22 embedding: Triangle inequality: Minimize sum of “edge lengths” Spread out vertices Procedure to recover cut of size

10. ARV Procedure to recover cut Unit sphere in Rd • Slice a randomly oriented “fat”-hyperplane of width

11. ARV Procedure to recover cut Unit sphere in Rd S • Slice a randomly oriented “fat”-hyperplane of width • Discard pairs of points (u,v): • Arrange points according to distance from S • Cut along ball of random radius r:

12. Metric Embeddings Finite Metric Space (X, d) Rk with L2 norm y f(y) f() x f(x) Distortion of f is min c: [Bourgain ’85] Every finite metric space can be embedded in L2 with distortion O(log n). Longstanding open question: Better bound for L1? [Enflo ’69] [Arora, Lee, Naor ’05] Any finite L1 metric can be embedded in L2 with distortion

13. Today’s Talk Multi-commodity flow: • Leighton-Rao: multi-commodity flow O(n2). • Arora, Hazan, Kale: O*(n2) ARV implementation • based on expander-flow formalism • Much faster in practice. • [Khandekar-Rao-V] : O*(min{n1.5, n/α(G)}) single commodity • flow based algorithm. O(log2 n) approx. ratio. • [Arora, Kale]: matrix multiplicative weights algorithm • based O(log n) approx • [Orrechia, Schulman, V, Vishnoi] O(log n) approx using • KRV style algorithm Single commodity flow:

14. Expander Flows • Any algorithm for approximating sparse cuts must • find a good cut, of expansion say β • Must also certify no cut is much smaller. • To give a k-approximation • must certify that no cut • has expansion less than • β/k. • Problem: there are exponentially many cuts. T S

15. Expander Flows G = H = • For each edge of H, route one unit of flow through G

16. Expander Flows T G = H = S • For each edge of H, route one unit of flow through G • Must route Ώ(|S|) units of flow from S to T. • Therefore |ES,T| = Ώ(|S|/c) expansion = Ώ(1/c) • Ideally c = O(1/α(G)) max congestion = c expansion = Ώ(1/c)

17. Expander Flows • max congestion = c. expansion = Ώ(1/c). • ARV: max congestion = • Leighton-Rao: H = complete graph. max cong = O(logn/α(G)) • tight example: G = expander graph. • Motivating idea for ARV: write LP to find best embedding • of H in G + exponentially many constraints saying H expander • eigenvalue bound gives efficient test for expansion! • Therefore poly time using Ellipsoid algorithm. • [Arora, Hazan, Kalle] O*(n2) implementation of ARV

18. KRV t s • Know large number of vertices on each side of cut. • A max-flow, min-cut computation should reveal sparse cut. • But this is circular…

19. Outline of Algorithm • H Φ • Embed candidate expander H in G with small congestion. • Test whether H is expander (if so done!) • Else non-expanding cut in H gives a bipartition of G; • route a flow in G across this bipartition. • Decompose flow into flow paths and add the resulting • matching to H.

20. Cut-Matching Game H Φ • Matching Player • Pick a perfect matching • across cut • Goal: max # iterations until • H is an expander • Cut Player • Find bad 50-50 cut in H • Goal: min # iterations until • H is an expander Claim: There is a cut player strategy that succeeds in O(log 2 n) rounds.

21. Finding a cut: Spectral-like-method After t iterations, H = { M1, M2, …, Mt }. = +1 charge Random assignment of charge = –1 charge (x+y)/2 x V: Vertex set Mix the charges along the matchings { M1, M2, …, Mt } y (x+y)/2

22. Finding a cut: Spectral-like-method Order the vertices according to the final charge present and cut in half. S S n/2 n/2 But how to formalize intuition?

23. Lift to Rn • Cannot directly formalize previous intuition • Therefore lift random walk to Rn – walk embedding of H. • n-dimensional vector associated with each vertex • In each step, replace vectors at endpoints of matched • edge by their average vector. • Potential function to measure progress of this process. • Potential function small implies H expander. • Relate lifted process to original random walk: • each successive matching decreases potential function.

24. Walk Embedding Ht = { M1, M2, …, Mt }. H Rn, Vertex i mapped to Pi = (pi1, …, pin) pij = P[walk started at j ends at i] P3 P2 P1 Small cut in graph shows up as clusters in walk embedding. (1/n, …, 1/n) Pn Potential: Ψ(0) = n-1 Claim: ψ(t) ≤1/4n2 implies α(Ht)≥ ½ Will show potential reduces by (1 – 1/log n) in each iteration.

25. The Lifted Walk Main Question: How to augment Ht = { M1, M2, …, Mt } by Mt+1 so H closer to expander? P3 Potential: P2 If Mt+1 matches vertex u to vertex v, then potential reduction in t+1-st step Since each of Pu and Pv replaced by So potential reduction = P1 (1/n, …, 1/n) Pn

26. Potential Reduction Original random walk = projection of lifted walk on random vector Pv  = v |Pv1/n|2 Reduction in  = |green|2 1-d: reduction = () n-d  1-d: log n stretch  Actual potential reduction = /log n

27. Running time • Number of iterations = O(log2 n) • Each iteration = 1 max-flow + O*(n) work = O*(m3/2) • [Benczur-Karger’96] In O*(m) time, we can transform any graph G on n vertices into G’ on same vertices: • G’ has O(n log (n)/ε2) edges • All cuts in G’ have size within (1 ± ε) of those in G • Overall running time = O*(m + n3/2)

28. Improving to O(log n) approximation • [Arora, Kale]: matrix multiplicative weights algorithm based • combinatorial primal-dual schema for semidefinite progs • [Orrechia, Schulman, V, Vishnoi]: simple KRV style algorithm Idea: To find Mt+1 perform t steps of natural random walk (instead of round-robin walk) on Ht = { M1, M2, …, Mt }

29. Brief Sketch • Instead of showing that H has constant edge • expansion after O(log2 n) steps, will show that • the spectral gap of H is at least 1/log n, and therefore • the conductance of H is at least 1/log n. • Since degree of H is log2 n, this means its edge • edge expansion is at least log n.

30. Why natural walk? Mk+1 S T Suppose round robin walk on M1, … , Mk mixes perfectly on each of S, T. Now a single averaging step on Mk+1 ensures perfect mixing on entire graph!

31. Matrix inequality: Gives a way of relating round robin walk to time independent walk. Question: Replace ½ self-loop with a ¾ self-loop in round-robin random walk! (3x/4+y/4) x y (3y/4 + x/4

32. Conclusions and Open Questions • Our algorithm is very similar to some heuristics. • [Lang’04] similar to one iteration of our algorithm. • METIS [Karypis-Kumar’99] • collapses random edges • finds a good partition in collapsed graph • induces it up to original graph, using local search • Connections with these heuristics? Rigorous analysis?

33. When the Expansion is large … • Could have used [Spielman-Teng’04] “nibble” algorithm instead of walk-embedding. But: • Conjecture: A single iteration of round-robin walk + • max-flow should give a sparse cut.

34. Limits to these methods • [Khot, Vishnoi] Ώ(loglog n) integrality gap • [Orrechia, Schulman, V, Vishnoi] Ώ(√logn) bound on cut-matching game. • Is it possible to obtain a O(√log n) approximation algorithm using single commodity flows via the cut-matching game?