1 / 38

Graph Sparsifiers : A Survey

Graph Sparsifiers : A Survey. Based on work by: Batson, Benczur , de Carli Silva, Fung, Hariharan , Harvey, Karger , Panigrahi , Sato, Spielman , Srivastava and Teng. Nick Harvey. Approximating Dense Objects by Sparse Ones. Floor joists Image compression.

merlin
Download Presentation

Graph Sparsifiers : A Survey

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Graph Sparsifiers: A Survey Based on work by: Batson, Benczur, de Carli Silva, Fung, Hariharan, Harvey, Karger, Panigrahi, Sato, Spielman, Srivastava and Teng Nick Harvey

  2. Approximating Dense Objects by Sparse Ones • Floor joists • Image compression

  3. Approximating Dense Graphsby Sparse Ones • Spanners: Approximate distances to within ®using only = O(n1+2/®) edges • Low-stretch trees: Approximate most distancesto within O(log n) using only n-1 edges (n = # vertices)

  4. Overview • Definitions • Cut & Spectral Sparsifiers • Cut Sparsifiers • A combinatorial construction • Spectral Sparsifiers • A random sampling construction • Derandomization

  5. (Karger ‘94) Cut Sparsifiers • Input: An undirected graph G=(V,E) with weights u : E !R+ • Output: A subgraph H=(V,F) of G with weightsw : F!R+ such that |F| is small and w(±H(U)) = (1§²) u(±G(U)) 8Uµ V weight of edges between U and V\U in H weight of edges between U and V\U in G U U

  6. Cut Sparsifiers • Input: An undirected graph G=(V,E) with weights u : E !R+ • Output: A subgraph H=(V,F) of G with weightsw : F!R such that |F| is small and w(±H(U)) = (1§²) u(±G(U)) 8Uµ V weight of edges between U and V\U in H weight of edges between U and V\U in G

  7. Generic Applicationof Cut Sparsifiers (Slow) Algorithm A for some problem P (Dense) Input graph G Exact/Approx Output Min s-t cut, Sparsest cut,Max cut, … (Efficient) Sparsification Algorithm S Algorithm A(now faster) Sparse graph H approx preserving solution of P Approximate Output

  8. Relation to Expander Graphs • Graph H on V is an expander if, for some constant c,|±H(U)| ¸c¢ |U| 8Uµ V, |U|·n/2 • Let G be the complete graph on V. Note that|±G(U)| = |U|¢|VnU| · n¢|U| • If we give all edges of H weight w=n, thenw(±H(U)) ¸ c¢|±G(U)| 8Uµ V, |U|·n/2 • Expanders are similar to sparsifiers of complete graph G H

  9. Relation to Expander Graphs • Fact: Pick a random graph where each edge appears independently with probability p=(log(n)/n). Gives an expander with O(n log n) edges with high probability. G H

  10. (Spielman-Teng ‘04) Spectral Sparsifiers • Input: An undirected graph G=(V,E) withweights u : E !R+ • Def: The Laplacianis the matrix LG such thatxTLGx = st2Eust (xs-xt)28x2RV. • LG is positive semidefinite since this is ¸ 0. • Example: Electrical Networks • View edge st as resistor of resistance 1/ust. • Impose voltage xv at every vertex v. • Ohm’s Power Law: P = V2/R. • Power consumed on edge st is ust (xs-xt)2. • Total power consumed is xTLGx.

  11. (Spielman-Teng ‘04) Spectral Sparsifiers • Input: An undirected graph G=(V,E) withweights u : E !R+ • Def: The Laplacianis the matrix LG such thatxTLGx = st2Eust (xs-xt)28x2RV. • Output: A subgraphH=(V,F) of G with weightsw : F!R such that |F| is small and xTLHx = (1 §²) xTLGx8x2RV w(±H(U)) = (1 § ²) u(±G(U)) 8Uµ V SpectralSparsifier ) ) CutSparsifier

  12. Cut vs Spectral Sparsifiers • Number of Constraints: • Cut: w(±H(U)) = (1§²) u(±G(U)) 8UµV (2n constraints) • Spectral: xTLHx = (1§²) xTLGx8x2RV (1 constraints) • Spectral constraints are SDP feasibility constraints: (1-²) xTLGx· xTLHx· (1+²) xTLGx8x2RV , (1-²) LG¹LH¹ (1+²) LG • Spectral constraints are actually easier to handle • Checking “Is H is a spectral sparsifier of G?” is in P • Checking “Is H is a cut sparsifier of G?” isnon-uniform sparsest cut, so NP-hard Here X ¹ Y means Y-X is positive semidefinite

  13. Application of Spectral Sparsifiers • Consider the linear system LGx = b.Actual solution is x := LG-1 b. • Instead, compute y := LH-1b,where H is a spectral sparsifier of G. • We know: (1-²) LG¹LH¹(1+²) LG )y has low multiplicative error: ky-xkLG·2²kxkLG • Computing y is fast since H is sparse:conjugate gradient method takes O(n|F|) time (where |F| = # nonzero entries of LH)

  14. Application of Spectral Sparsifiers • Consider the linear system LGx = b.Actual solution is x := LG-1 b. • Instead, compute y := LH-1b,where H is a spectral sparsifier of G. • We know: (1-²) LG¹LH¹(1+²) LG )y has low multiplicative error: ky-xkLG·2²kxkLG • Theorem:[Spielman-Teng ‘04, Koutis-Miller-Peng ‘10]Can compute avector y with low multiplicative error in O(m log n (log log n)2) time. (m = # edges of G)

  15. Results on Sparsifiers Cut Sparsifiers Spectral Sparsifiers Karger ‘94 Benczur-Karger ‘96 Combinatorial Fung-Hariharan-Harvey-Panigrahi ‘11 Spielman-Teng ‘04 Spielman-Srivastava ‘08 Linear Algebraic Batson-Spielman-Srivastava ‘09 de Carli Silva-Harvey-Sato ‘11 These construct sparsifiers with n logO(1) n / ²2 edges These construct sparsifiers with O(n / ²2) edges

  16. Sparsifiers by Random Sampling • The complete graph is easy!Random sampling gives an expander (ie. sparsifier) with O(n log n) edges.

  17. Sparsifiers by Random Sampling Eliminate most of these • Can’t sample edges with same probability! • Idea [BK’96]Sample low-connectivity edges with high probability, and high-connectivity edges with low probability Keep this

  18. Non-uniform sampling algorithm [BK’96] • Input: Graph G=(V,E), weights u : E !R+ • Output: A subgraph H=(V,F) with weights w : F !R+ • Choose parameter ½ • Compute probabilities { pe:e2E} • For i=1 to ½ • For each edge e2E • With probability pe, Add e to F Increase we by ue/(½pe) Can we dothis so that thecut values aretightly concentratedand E[|F|]=nlogO(1)n? • Note: E[|F|] ·½¢epe • Note: E[ we ] = ue8e2E • ) For every UµV, E[ w(±H(U)) ] = u(±G(U))

  19. Benczur-Karger ‘96 • Input: Graph G=(V,E), weights u : E !R+ • Output: A subgraph H=(V,F) with weights w : F !R+ • Choose parameter ½ • Compute probabilities { pe:e2E} • For i=1 to ½ • For each edge e2E • With probability pe, Add e to F Increase we by ue/(½pe) Can we dothis so that thecut values aretightly concentratedand E[|F|]=nlogO(1)n? Can approximateall values in m logO(1)n time. • Set ½ = O(logn/²2). • Let pe = 1/“strength” of edge e. • Cuts are preserved to within (1§²) and E[|F|] = O(n logn/²2)

  20. Fung-Hariharan-Harvey-Panigrahi ‘11 • Input: Graph G=(V,E), weights u : E !R+ • Output: A subgraph H=(V,F) with weights w : F !R+ • Choose parameter ½ • Compute probabilities { pe:e2E} • For i=1 to ½ • For each edge e2E • With probability pe, Add e to F Increase we by ue/(½pe) Can we dothis so that thecut values aretightly concentratedand E[|F|]=nlogO(1)n? Can approximateall values in O(m + n log n) time • Set ½ = O(log2n/²2). • Let pst = 1/(min cut separating s and t) • Cuts are preserved to within (1§²) and E[|F|] = O(n log2n/²2)

  21. Letkuv = min size of a cut separating u and v.Recall sampling probability is pe = 1/ke • Partition edges into connectivity classesE = E1[E2[ ... Elog nwhere Ei = { e : 2i-1·ke<2i } • Prove weight of sampled edges that each cuttakes from each connectivity class has low error • Key point: Edges in ±(U)ÅEi have roughly same pe • This yields a sparsifier U

  22. Notation: • C = ±(U) is a cut • Ci= ±(U) ÅEi is a cut-induced set • Need to prove: for every Ci Prove weight of sampled edges that each cuttakes from each connectivity class has low error C2 C3 C1 C4

  23. Notation:Ci= ±(U) ÅEi is a cut-induced set Prove 8 cut-induced set Ci • Key Ingredients • Hoeffding bound: Provesmall • Bound on # small cut-induced sets:For most of these events, u(C) is large.In other words, #{ cut-induced sets Ciinduced by a small cut C }is small. C2 C3 C1 C4

  24. Counting Small Cut-Induced Sets • Theorem:[Fung-Hariharan-Harvey-Panigrahi ‘11] Let G=(V,E) be a graph. Fix any BµE. Suppose ke¸K for all e in B. (kuv = min size of a cut separating u and v) Then, for every ®¸1,|{ ±(U) ÅB : |±(U)|·®K }| < n2®. • Corollary: [Karger ‘93] Let G=(V,E) be a graph. Let K be the edge-connectivity of G. (i.e., global min cut value) Then, for every ®¸1,|{ ±(U) : |±(U)|·®K }| < n2®.

  25. Summary for Cut Sparsifiers • Do non-uniform sampling of edges,with probabilities based on “connectivity” • Analysis involves: • Decomposing the graph • Hoeffding bounds to analyze each “cut” • Cut-counting theorem: “few small cuts” • BK’96 had weaker cut-counting theorem, but had more complicated “connectivity” notion. • Can get sparsifiers with O(n log n / ²2) edges • Optimal for any independent sampling algorithm

  26. Spectral Sparsification • Input: Graph G=(V,E), weights u : E !R+ • Recall: xTLGx = st2Eust (xs-xt)2 • Goal: Find weights w : E !R+ such that very fewwe are non-zero, and(1-²) xTLGx·e2EwexTLex·(1+²) xTLGx8x2RV , (1- ²) LG¹e2EweLe¹ (1+²) LG • General Problem: Given matrices Le satisfying eLe=LG, find coefficients we, mostly zero, such that (1-²) LG¹eweLe¹ (1+²) LG Call this xTLstx

  27. The General Problem:Sparsifying Sums of PSD Matrices • General Problem: Given PSD matrices Les.t. eLe=LG, find coefficients we, mostly zero, such that (1-²) LG¹eweLe¹ (1+²) LG • Theorem:[Ahlswede-Winter ’02]Randomized alg gives w with O( n log n/²2 ) non-zeros. • Theorem:[de Carli Silva-Harvey-Sato ‘11],building on [Batson-Spielman-Srivastava ‘09]Deterministic alg gives w with O( n/²2 ) non-zeros. • Cut & spectral sparsifiers with O(n/²2) edges [BSS’09] • Sparsifiers with more properties and O(n/²2) edges [dHS’11]

  28. Vector Case Vector Case • Vector problem: Given vectors ve2[0,1]ns.t. eve = v,find coefficients we, mostly zero, such thatkeweve - v k1 · ² • Theorem [Althofer ‘94, Lipton-Young ‘94]:There is a w with O(log n/²2) non-zeros. • Proof: Random sampling & Hoeffding inequality. • Multiplicative version: There is a w with O(n log n/²2) non-zeros such that (1-²) v·eweve· (1+²) v • General Problem: Given PSD matrices Les.t. eLe = L, find coefficients we, mostly zero, such that (1-²) L¹eweLe¹ (1+²) L

  29. Concentration Inequalities • Theorem: [Chernoff ‘52, Hoeffding ‘63]Let Y1,…,Yk be i.i.d. randomnon-negative real numberss.t. E[ Yi ] = Z andYi·uZ. Then • Theorem: [Ahlswede-Winter ‘02]Let Y1,…,Yk be i.i.d. randomPSD nxn matricess.t. E[ Yi ] = Z andYi¹uZ. Then The only difference

  30. “Balls & Bins” Example • Problem: Throw k balls into n bins. Wantmax load / min load · 1+². How big should k be? • AW Theorem: Let Y1,…,Yk be i.i.d. random PSD matricessuch that E[ Yi ] = Z andYi¹uZ. Then • Solution: Let Yi be all zeros, except for a single n in a random diagonal entry. Then E[ Yi ] = I =: Z, and ¸max(Yi Z-1) = n =: u.Set k = £(n logn/²2). Then, with high probability,every diagonal entry of i Yi/k is in [1-²,1+²].

  31. Solving the General Problem • General Problem: Given PSD matrices Les.t. eLe = LG, find coefficients we, mostly zero, such that (1-²) LG¹eweLe¹ (1+²) LG • AW Theorem: Let Y1,…,Yk be i.i.d. random PSD matricessuch that E[ Yi ] = Z andYi¹uZ. Then • Solve General Problem with O(nlogn/²2) non-zeros • Repeat k:=£(nlogn/²2) times • Pick an edge e with probability pe := Tr(LeLG-1)/n • Increment we by 1/k¢pe

  32. Derandomization • Vector problem: Given vectors ve2[0,1]ns.t. eve = v,find coefficients we, mostly zero, such thatkeweve - vk1 · ² • Theorem[Young ‘94]: The multiplicative weights method deterministically gives w with O(log n/²2) non-zeros • Or, use pessimistic estimators on the Hoeffding proof • General Problem: Given PSD matrices Les.t. eLe = LG, find coefficients we, mostly zero, such that (1-²) LG¹eweLe¹ (1+²) LG • Theorem [de Carli Silva-Harvey-Sato ‘11]:The matrix multiplicative weights method (Arora-Kale ‘07)deterministically gives w with O(n log n/²2) non-zeros • Or, use matrix pessimistic estimators (Wigderson-Xiao ‘06)

  33. MWUM for “Balls & Bins” • Let ¸i = load in bin i. Initially ¸=0. Want: 1·¸i and ¸i·1. • Introduce penalty functions “exp(l-¸i)” and “exp(¸i-u)” • Find a bin ¸i to throw a ball into such that,increasing l by ±l and u by ±u, the penalties don’t grow. i exp(l+±l - ¸i’) · i exp(l-¸i) i exp(¸i’-(u+±u)) · i exp(¸i-u) • Careful analysis shows O(n log n/²2) balls is enough ¸ values: u l 0 1

  34. MMWUM for General Problem • Let A=0 and ¸ its eigenvalues. Want: 1·¸i and ¸i·1. • Use penalty functions Tr exp(lI-A) and Tr exp(A-uI) • Find a matrix Le such that adding ®Le to A,increasing l by ±l and u by ±u, the penalties don’t grow. Trexp((l+±l)I- (A+®Le)) · Trexp(lI-A) Trexp((A+®Le)-(u+±u)I) · Tr exp(A-uI) • Careful analysis shows O(n log n/²2) matrices is enough ¸ values: u l 0 1

  35. Beating Sampling & MMWUM • To get a better bound, try changing the penalty functions to be steeper! • Use penalty functions Tr (A-lI)-1 and Tr (uI-A)-1 • Find a matrix Le such that adding ®Le to A,increasing l by ±l and u by ±u, the penalties don’t grow. Tr((A+®Le)-(l+±l)I)-1· Tr(A-lI)-1 Tr((u+±u)I-(A+®Le))-1· Tr (uI-A)-1 All eigenvaluesstay within [l, u] ¸ values: u l 0 1

  36. Beating Sampling & MMWUM • To get a better bound, try changing the penalty functions to be steeper! • Use penalty functions Tr (A-lI)-1 and Tr (uI-A)-1 • Find a matrix Le such that adding ®Le to A,increasing l by ±l and u by ±u, the penalties don’t grow. Tr((A+®Le)-(l+±l)I)-1· Tr(A-lI)-1 Tr((u+±u)I-(A+®Le))-1· Tr (uI-A)-1 • General Problem: Given PSD matrices Les.t. eLe = LG, find coefficients we, mostly zero, such that (1-²) LG¹eweLe¹ (1+²) LG • Theorem:[Batson-Spielman-Srivastava ‘09] in rank-1 case,[de Carli Silva-Harvey-Sato ‘11] for general caseThis gives a solution w with O( n/²2 ) non-zeros.

  37. Applications • Theorem:[de Carli Silva-Harvey-Sato ‘11]Given PSD matrices Les.t. eLe = L, there is analgorithm to find w with O( n/²2 ) non-zeros such that (1-²) L¹eweLe¹ (1+²) L • Application 1: Spectral Sparsifiers with CostsGiven costs on edges of G, can find sparsifier H whose cost isat most (1+²) the cost of G. • Application 2: Simultaneous Spectral SparsifiersGiven two graphs G1 & G2 with a bijection on their edges,can choose edges that simultaneously sparsify G1 & G2. • Application 3: Sparse SDP Solutionsmin { cTy : iyiAiº B, y¸0 } where Ai’s and B are PSDhas nearly optimal solution with O(n/²2) non-zeros.

  38. Open Questions • Use of sparsifiers in other areas (infoviz, etc.) • Sparsifiers for directed graphs • Construction of expander graphs • More control of the weights we • A combinatorial proof of spectral sparsifiers • More applications of our general theorem

More Related