1 / 73

Approximations for Isoperimetric and Spectral Profile and Related Parameters

Approximations for Isoperimetric and Spectral Profile and Related Parameters. S. Prasad Raghavendra MSR New England. joint work with. David Steurer Princeton University. Prasad Tetali Georgia Tech. Graph Expansion. d -regular graph G with n vertices. d.

urbain
Download Presentation

Approximations for Isoperimetric and Spectral Profile and Related Parameters

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Approximations for Isoperimetric and Spectral Profile and Related Parameters S • Prasad Raghavendra • MSR New England joint work with • David Steurer • Princeton University • Prasad Tetali • Georgia Tech

  2. Graph Expansion d-regular graph G with n vertices d A random neighbor of a random vertex in S is outside of S with probability expansion(S) # edges leaving S expansion(S) = d |S| Extremely well-studied, many different contexts pseudo-randomness, group theory, online routing, Markov chains, metric embeddings, … Conductance of Graph G vertex set S minimum |S| ≤ n/2 ФG = expansion(S) Uniform Sparsest Cut Problem Given a graph G compute ФGand find the set S achieving it. S

  3. Measuring Graph Expansion d-regular graph G with n vertices d # edges leaving S expansion(S) = d |S| Conductance of Graph G Complete Graph Complete Graph vertex set S minimum |S| ≤ n/2 ФG = expansion(S) Path Complete graphs with a perfect matching Typically, small sets expand to a greater extent.

  4. Isoperimetric Profile S # edges leaving S expansion(S) = d |S| 1 Isoperimetric Profile of Graph G • Conductance function – defined by [Lovasz-Kannan] used to obtain better mixing time bounds for Markov chains. • Decreasing function of δ minimum |S| ≤ δn Ф(δ) = expansion(S) Ф(δ) 0.5 Set Size δ

  5. Approximating Isoperimetric Profile S # edges leaving S What is the value of Ф(δ) for a given graph G and a constant δ > 0? expansion(S) = Uniform Sparsest Cut: Determine the lowest point on the curve. d |S| 1 Isoperimetric Profile of Graph G Gap Small Set Expansion Problem (GapSSE)(η, δ) Given a graph G and constants δ, η > 0, Is Ф(δ) < η OR Ф(δ) > 1- η? minimum |S| ≤ δn Ф(δ) = expansion(S) Ф(δ) ----Closely tied to Unique Games Conjecture (Last talk of the day “Graph Expansion and the Unique Games Conjecture” [R-Steurer 10]) 0.5 Set Size δ

  6. Algorithm Theorem For every constant δ > 0, there exists a poly-time algorithm that finds a set S of size O(δ) such that expansion (S) ≤ -- A (Ф(δ) vs ) approximation • For small enough δ, the algorithm cannot distinguish between • Ф(δ) < η OR Ф(δ) > 1- η Theorem [R-Steurer-Tulsiani 10] • An improvement over above algorithm by better than constant factor, would help distinguish Ф(δ) < η OR Ф(δ) > 1- η

  7. A Spectral Relaxation 0 Letx = (x1 , x2 , .. xn) be the indicator function of the unknown least expanding setS SC 1 S |S| = Relaxing 0,1 to real numbers Number of edges leaving S = |Support(x)| < δn

  8. Why is it spectral? Spectral Profile: [Goel-Montenegro-Tetali] |Support(x)| < δn |Support(x)| < δn Smallest Eigen Value of Laplacian: Observation:Λ(δ) is the smallest possible eigenvalue of a submatrixL(S,S) of size at most < δn of the LaplacianL.

  9. Rounding Eigenvectors Smallest Eigen Value of Laplacian Cheeger’s inequality There is a sparse cut of value at most Rounding x |Support(x)| < δn |Support(x)| < δn 0 Lemma: There exists a set S of volume at most δ whose expansion is at most

  10. Spectral Profile[Goel-Montenegro-Tetali] |Support(x)| < δn |Support(x)| < δn Unlike eigen values, Λ(δ) is not the optimum of a convex program. We show an efficiently computable SDP that gives good guarantee. Theorem: There exists an efficiently computable SDP relaxation Λ* (δ) of Λ(δ) such that for every graph G Λ* (δ) ≤ Λ(δ) ≤ Λ* (δ/2)∙O(log 1/δ)

  11. Recap Theorem: (Approximating Spectral Profile) There exists an efficiently computable SDP relaxation Λ* (δ) of Λ(δ) such that for every graph G Λ* (δ) ≤ Λ(δ) ≤ Λ* (δ/2)∙O(log 1/δ) Theorem (Approximating Isoperimetric Profile) For every constant δ > 0, there exists a poly-time algorithm that finds a set S of size O(δ) such that expansion (S) ≤ Lemma: (Cheeger Style Rounding) There exists a set S of volume at most δ whose expansion is at most

  12. Restricted Eigenvalue Problem Given a matrix A, find a submatrix A[S,S] of size at most δn X δn matrix with the least eigenvalue. -- Our algorithm is applicable to diagonally dominant matrices (yields a log(1/δ) approximation).

  13. Approximating Spectral Profile |Support(x)| < δn

  14. SDP Relaxation for Replace each xi by a vector vi: Denominator = n Without loss of generality, xi can be assumed positive. This yields the constraint: vi∙vj ≥ 0 Enforcing Sparsity: |Support(x)| < δn By Cauchy-Schwartzinequality:

  15. SDP Relaxation for Minimize (Sum of Squared Edge Lengths) Subject to Positive Inner Products: vi∙vj ≥ 0 Average squared length =1 |Support(x)| < δn Average Pairwise Correlation < δ

  16. Rounding • Two Phase Rounding: • Transform SDP vectors in to a SDP solution with only non-negative coordinates. • Use thresholding to convert non-negative vectors in to sparse vectors.

  17. Making SDP solution nonnegative Let v be a n-dimensional real vector. Let v*denote the unit vector along direction v. Map the vector v to the following function over Rn: fv= ||v||∙ (Square Root of Probability Density Function of n-dimensional Gaussian centered at v*) Formally, Where: Ф(x) = probability density function of a mean 0, varianceσ spherical Gaussian in n-dimensions.

  18. Properties Where: Ф(x) = probability density function of a mean 0, varianceσ spherical Gaussian in n-dimensions. SDP Constraint Average Pairwise Correlation < δ Lemma: (Pairwise correlation remains low if σ is small) Pick σ = 1/sqrt{log(1/δ)}

  19. Properties Where: Ф(x) = probability density function of a mean 0, varianceσ spherical Gaussian in n-dimensions. Lemma:(Squared Distances get stretched by at most 1/σ2) |f1– f2|2 ≤ O(1/σ2) |v1 – v2 |2 For our choice of σ, squared distances are stretched by log(1/δ) . With a log(1/δ) factor loss, we obtaining a non-negative SDP solution.

  20. Rounding a positive vector solution Let us pretend the vectors vi are non-negative. i.e., vi(t) ≥ 0 for all t Rounding Non-Negative Vectors • Sample t • Compute threshold θ = (average of vi(t) over i) * (2/δ ) • Set xi = max{ vi(t) – θ, 0 } for all i Observation:|Support(x)| <δ/2 ∙ n Observation:

  21. Open Problem Minimize (Sum of Squared Edge Lengths) Subject to Positive Inner Products: vi∙vj ≥ 0 Find integrality gaps for the SDP relaxation: Current best have δ = 1/poly(logn), Do there exist integrality gaps with δ = 1/poly(n)? Average squared length =1 Average Pairwise Correlation < δ

  22. Thank You

  23. Spectral Profile Second Eigen Value: Spectral Profile |Support(x)| < δn • Remarks: • Λ(δ) ≤ minimum of Ф(δ0) over all δ0 ≤ δ • Unlike second eigen value, Λ(δ) is not the optimum of a convex program. • Λ(δ) is the smallest possible eigenvalue of a submatrix of size at most < δn of the matrix L • xi can be assumed to be all positive.

  24. Small Sets via Spectral Profile Using an analysis along the lines of analysis of Cheeger’s inequality, it yields: Theorem [Raghavendra-Steurer-Tetali 10] There exists a poly-time algorithm that finds a set S of size O(δ) such that expansion (S) ≤ sqrt (Λ* (δ)∙O(log 1/δ)) So if there is a set S with expansion(S) = ε, then the algorithm finds a cut of size Similar behaviour as the Gaussian expansion profile

  25. Spectral Profile Second Eigen Value: Spectral Profile |Support(x)| < δn Replace xi by vectors vi Subject to vi∙vj ≥ 0

  26. Graph Expansion A random neighbor of a random vertex in S is outside of S with probability expansion(S) d-regular graph G d Extremely well-studied, many different contexts pseudo-randomness, group theory, online routing, Markov chains, metric embeddings, … Uniform Sparsest Cut Problem Given a graph G compute ФGand find the set S achieving it. # edges leaving S expansion(S) = d |S| • Approximation Algorithms: • Cheeger’s Inequality [Alon][Alon-Milman] • Given a graph G, if the normalized adjacency matrix has second eigen value λ2 then, • A log n approximation algorithm [Leighton-Rao]. • A sqrt(log n) approximation algorithm using semidefinite programming [Arora-Rao-Vazirani]. Conductance of Graph G vertex set S minimum |S| ≤ n/2 ФG = expansion(S)

  27. Limitations of Eigenvalues • The best lower bound that Cheeger’s inequality gives on expansion is (1-λ2)/2 < ½, while Ф(δ) can be close to 1. • Consider graph G Connect pairs of points on {0,1}nthat are εn Hamming distance away. Then Second eigenvalue≈ 1- εand ф(1/2) ≈ ε yet Ф(δ) ≈ 1 (small sets have near-perfect expansion) • A simple Sdp relaxation cannot distinguish between • all small sets expand almost completely • exists small set with almost no expansion

  28. A Conjecture Small-Set Expansion Conjecture: 8η>0, 9± >0 such that GapSSE(η, ± ) is NP-hard, i.e., Given a graph G, it is NP-hard to distinguish YES: expansion(S) <η for some S of volume ¼± NO: expansion(S) > 1- η for all S of volume ¼±

  29. Road Map • Algorithm: • Spectral Profile. • Reductions within Expansion • Relationship with Unique Games Conjecture

  30. Gaussian Curve Γε(δ) = Gaussian noise sensitivity of a set of measure δ = least expanding sets are caps/thresholds of measure δ = Gaussian Graph Vertices: all points in Rd (d dimensional real space) Edges: Weights according to the following sampling procedure: • Sample a random Gaussian variable x in Rd • Perturb x as follows to get y in Rd Add an edge between x an y 1 Ф(δ) 0.5 Set Size δ

  31. Approximating Expansion Profile

  32. Reductions within Expansion

  33. Reductions within Expansion • Theorem [Raghavendra-Steurer-Tulsiani 10] • For every positive integer q, and constantsε,δ,γ, given a graph it is SSE-hard to distinguish between: • There exists q disjoint small sets S1 , S2 , .. Sq of size close to 1/q, such that expansion(Si) ≤ ε • No set of size μ> δ has expansion less than size Γε/2(μ) -- expansion of set of size μ in Gaussian graph with parameter ε/2 • expansion (S) < sqrt (Λ* (δ)∙O(log 1/ μ))

  34. Informal Statement 1 • Quantitative Statement • Given a graph G , it is SSE-hard to distinguish whether, • There is a small set of size whose expansion is ε. • Every small set of size μ (in a certain range) expands at least as much as the corresponding Gaussian graph with noise ε 1 Set Size δ Qualitative Assumption GapSSE is NPhard Ф(δ) 0.5 Set Size δ

  35. Corollaries Corollary: The algorithm in[Raghavendra-SteurerTetali 10] has near-optimal guarantee assuming SSE conjecture. Corollary: Assuming GapSSE, there is no constant factor approximation for Balanced Separator or Uniform Sparsest Cut.

  36. Relation with Unique Games Conjecture

  37. Unique Games Unique game ¡ : Referee sample (A,B,¼) graph of size n label set L(A) A ¼ B Each edge (A,B) has an associated map ¼ ¼ is bijection from L(A) to L(B) A a B b Referee A labelling satisfies (A,B) if ¼ (label of A) = label of B players win if ¼(a) = b Player 1 Player 2 pick a in L(A) pick b in L(B) Goal: Find an assignment of labels to vertices such that maximum number of edges are satisfied. value( ¡ ): maximum success probability over all strategies of the players no communication between players

  38. Unique Games Conjecture [Khot02] Unique Games Conjecture: [Khot ‘02] 8²>0, 9 q >0: NP-hard to distinguish for ¡ with label set size q YES: value( ¡ ) > 1-² NO: value( ¡ ) < ²

  39. Implications of UGC Basic Sdp is optimal for … • Constraint Satisfaction Problems [Raghavendra`08] • Max Cut, Max 2Sat Metric Labeling Problems [MNRS`08] • Multiway Cut, 0-extension • Ordering CSPs [GMR`08] • Max Acyclic Subgraph, Betweeness UGC • Strict Monotone CSPs [KMTV`10] • Vertex Cover, Hypergraph Vertex Cover Kernel Clustering Problems [KN`08,10] Grothendieck Problems[KNS`08, RS`09] …

  40. “Reverse Reductions” ? Basic Sdp is optimal for lots of optimization problems, e.g.: Max Cut and Vertex Cover UGC Basic Sdp optimal for Problem X * Win-Win Situation If we could show , then a refutation of UGC would imply an improved algorithm for Problem X * Problem X = Max Cut Parallel Repetition is natural candidate reduction for [FeigeKinderO’Donnell’07] * Bad news: this reduction cannot work [Raz’08, BHHRRS’08]

  41. Small Set Expansion and Unique Games • Solving Unique Games  Finding a small non-expanding set in the “label extended graph” Theorem [Raghavendra-Steurer 10] Small Set Expansion Conjecture  Unique Games Conjecture Establishes a reverse connection from a natural problem.

  42. Implications of UGC Basic Sdp is optimal for … • Constraint Satisfaction Problems [Raghavendra`08] • Max Cut, Max 2Sat Metric Labeling Problems [MNRS`08] • Multiway Cut, 0-extension • Ordering CSPs [GMR`08] • Max Acyclic Subgraph, Betweeness UGC UGC With expansion • Strict Monotone CSPs [KMTV`10] • Vertex Cover, Hypergraph Vertex Cover Gap SSE Kernel Clustering Problems [KN`08,10] Grothendieck Problems[KNS`08, RS`09] … Uniform Sparsest Cut [KNS`08, RS`09] Minimum Linear Arrangement[KNS`08, RS`09]

  43. Most known SDP integrality gap instances for problems like MaxCut, Vertex Cover, Unique games have graphs that are “small set expanders” Theorem [Raghavendra-Steurer-Tulsiani 10] Small Set Expansion Conjecture  MaxCut or Unique Games on Small Set Expanders is hard. Reverse Connections?

  44. Approximating Spectral Profile

  45. Roadmap Introduction Graph Expansion: Cheeger’s Inequality, Leighton Rao, ARV Expansion Profile: Small Sets expand more than large ones. Cheeger’s inequality and SDPs fail GapSSE Problem Relation to Unique Games Conjecture: Unique Games definition, Applications, lack of reverse reductions. Label extended graph  Small sets Small Set Expansion Conjecture  UGC UGC with SSE is easy Algorithm for SSE: Spectral Profile, SDP for Spectral Profile, Rounding algorithm Relations within expansion: GapSSE Balanced Separator Hardness

  46. NP-hard Optimization Example: Max Cut: partition vertices of a graph into two sets so as to maximize number of cut edges fundamental graph partitioning problem benchmark for algorithmic techniques

  47. Approximation Max Cut Trivial approximation Random assignment, cut ½ of the edges approx-ratio 0.878 First non-trivial approximation Goemans–Williamson algorithm (‘95) based on a semidefinite relaxation (Basic Sdp) Beyond Max Cut: Analogous Basic Sdprelaxation for many other problems Almost always, Basic Sdpgives best known approximation (often strictly better than non-SDP methods)

  48. Approximation Max Cut Trivial approximation Random assignment, cut ½ of the edges approx-ratio 0.878 First non-trivial approximation Goemans–Williamson algorithm (‘95) based on a semidefinite relaxation (Basic Sdp) Can we beat the approximation guarantee of Basic Sdp?

More Related