Approximations for Isoperimetric and Spectral Profile and Related Parameters

1. Approximations for Isoperimetric and Spectral Profile and Related Parameters S • Prasad Raghavendra • MSR New England joint work with • David Steurer • Princeton University • Prasad Tetali • Georgia Tech

2. Graph Expansion d-regular graph G with n vertices d A random neighbor of a random vertex in S is outside of S with probability expansion(S) # edges leaving S expansion(S) = d |S| Extremely well-studied, many different contexts pseudo-randomness, group theory, online routing, Markov chains, metric embeddings, … Conductance of Graph G vertex set S minimum |S| ≤ n/2 ФG = expansion(S) Uniform Sparsest Cut Problem Given a graph G compute ФGand find the set S achieving it. S

3. Measuring Graph Expansion d-regular graph G with n vertices d # edges leaving S expansion(S) = d |S| Conductance of Graph G Complete Graph Complete Graph vertex set S minimum |S| ≤ n/2 ФG = expansion(S) Path Complete graphs with a perfect matching Typically, small sets expand to a greater extent.

4. Isoperimetric Profile S # edges leaving S expansion(S) = d |S| 1 Isoperimetric Profile of Graph G • Conductance function – defined by [Lovasz-Kannan] used to obtain better mixing time bounds for Markov chains. • Decreasing function of δ minimum |S| ≤ δn Ф(δ) = expansion(S) Ф(δ) 0.5 Set Size δ

5. Approximating Isoperimetric Profile S # edges leaving S What is the value of Ф(δ) for a given graph G and a constant δ > 0? expansion(S) = Uniform Sparsest Cut: Determine the lowest point on the curve. d |S| 1 Isoperimetric Profile of Graph G Gap Small Set Expansion Problem (GapSSE)(η, δ) Given a graph G and constants δ, η > 0, Is Ф(δ) < η OR Ф(δ) > 1- η? minimum |S| ≤ δn Ф(δ) = expansion(S) Ф(δ) ----Closely tied to Unique Games Conjecture (Last talk of the day “Graph Expansion and the Unique Games Conjecture” [R-Steurer 10]) 0.5 Set Size δ

6. Algorithm Theorem For every constant δ > 0, there exists a poly-time algorithm that finds a set S of size O(δ) such that expansion (S) ≤ -- A (Ф(δ) vs ) approximation • For small enough δ, the algorithm cannot distinguish between • Ф(δ) < η OR Ф(δ) > 1- η Theorem [R-Steurer-Tulsiani 10] • An improvement over above algorithm by better than constant factor, would help distinguish Ф(δ) < η OR Ф(δ) > 1- η

7. A Spectral Relaxation 0 Letx = (x1 , x2 , .. xn) be the indicator function of the unknown least expanding setS SC 1 S |S| = Relaxing 0,1 to real numbers Number of edges leaving S = |Support(x)| < δn

8. Why is it spectral? Spectral Profile: [Goel-Montenegro-Tetali] |Support(x)| < δn |Support(x)| < δn Smallest Eigen Value of Laplacian: Observation:Λ(δ) is the smallest possible eigenvalue of a submatrixL(S,S) of size at most < δn of the LaplacianL.

9. Rounding Eigenvectors Smallest Eigen Value of Laplacian Cheeger’s inequality There is a sparse cut of value at most Rounding x |Support(x)| < δn |Support(x)| < δn 0 Lemma: There exists a set S of volume at most δ whose expansion is at most

10. Spectral Profile[Goel-Montenegro-Tetali] |Support(x)| < δn |Support(x)| < δn Unlike eigen values, Λ(δ) is not the optimum of a convex program. We show an efficiently computable SDP that gives good guarantee. Theorem: There exists an efficiently computable SDP relaxation Λ* (δ) of Λ(δ) such that for every graph G Λ* (δ) ≤ Λ(δ) ≤ Λ* (δ/2)∙O(log 1/δ)

11. Recap Theorem: (Approximating Spectral Profile) There exists an efficiently computable SDP relaxation Λ* (δ) of Λ(δ) such that for every graph G Λ* (δ) ≤ Λ(δ) ≤ Λ* (δ/2)∙O(log 1/δ) Theorem (Approximating Isoperimetric Profile) For every constant δ > 0, there exists a poly-time algorithm that finds a set S of size O(δ) such that expansion (S) ≤ Lemma: (Cheeger Style Rounding) There exists a set S of volume at most δ whose expansion is at most

12. Restricted Eigenvalue Problem Given a matrix A, find a submatrix A[S,S] of size at most δn X δn matrix with the least eigenvalue. -- Our algorithm is applicable to diagonally dominant matrices (yields a log(1/δ) approximation).

13. Approximating Spectral Profile |Support(x)| < δn

14. SDP Relaxation for Replace each xi by a vector vi: Denominator = n Without loss of generality, xi can be assumed positive. This yields the constraint: vi∙vj ≥ 0 Enforcing Sparsity: |Support(x)| < δn By Cauchy-Schwartzinequality:

15. SDP Relaxation for Minimize (Sum of Squared Edge Lengths) Subject to Positive Inner Products: vi∙vj ≥ 0 Average squared length =1 |Support(x)| < δn Average Pairwise Correlation < δ

16. Rounding • Two Phase Rounding: • Transform SDP vectors in to a SDP solution with only non-negative coordinates. • Use thresholding to convert non-negative vectors in to sparse vectors.

17. Making SDP solution nonnegative Let v be a n-dimensional real vector. Let v*denote the unit vector along direction v. Map the vector v to the following function over Rn: fv= ||v||∙ (Square Root of Probability Density Function of n-dimensional Gaussian centered at v*) Formally, Where: Ф(x) = probability density function of a mean 0, varianceσ spherical Gaussian in n-dimensions.

18. Properties Where: Ф(x) = probability density function of a mean 0, varianceσ spherical Gaussian in n-dimensions. SDP Constraint Average Pairwise Correlation < δ Lemma: (Pairwise correlation remains low if σ is small) Pick σ = 1/sqrt{log(1/δ)}

19. Properties Where: Ф(x) = probability density function of a mean 0, varianceσ spherical Gaussian in n-dimensions. Lemma:(Squared Distances get stretched by at most 1/σ2) |f1– f2|2 ≤ O(1/σ2) |v1 – v2 |2 For our choice of σ, squared distances are stretched by log(1/δ) . With a log(1/δ) factor loss, we obtaining a non-negative SDP solution.

20. Rounding a positive vector solution Let us pretend the vectors vi are non-negative. i.e., vi(t) ≥ 0 for all t Rounding Non-Negative Vectors • Sample t • Compute threshold θ = (average of vi(t) over i) * (2/δ ) • Set xi = max{ vi(t) – θ, 0 } for all i Observation:|Support(x)| <δ/2 ∙ n Observation:

21. Open Problem Minimize (Sum of Squared Edge Lengths) Subject to Positive Inner Products: vi∙vj ≥ 0 Find integrality gaps for the SDP relaxation: Current best have δ = 1/poly(logn), Do there exist integrality gaps with δ = 1/poly(n)? Average squared length =1 Average Pairwise Correlation < δ

22. Thank You

23. Spectral Profile Second Eigen Value: Spectral Profile |Support(x)| < δn • Remarks: • Λ(δ) ≤ minimum of Ф(δ0) over all δ0 ≤ δ • Unlike second eigen value, Λ(δ) is not the optimum of a convex program. • Λ(δ) is the smallest possible eigenvalue of a submatrix of size at most < δn of the matrix L • xi can be assumed to be all positive.

24. Small Sets via Spectral Profile Using an analysis along the lines of analysis of Cheeger’s inequality, it yields: Theorem [Raghavendra-Steurer-Tetali 10] There exists a poly-time algorithm that finds a set S of size O(δ) such that expansion (S) ≤ sqrt (Λ* (δ)∙O(log 1/δ)) So if there is a set S with expansion(S) = ε, then the algorithm finds a cut of size Similar behaviour as the Gaussian expansion profile

25. Spectral Profile Second Eigen Value: Spectral Profile |Support(x)| < δn Replace xi by vectors vi Subject to vi∙vj ≥ 0

26. Graph Expansion A random neighbor of a random vertex in S is outside of S with probability expansion(S) d-regular graph G d Extremely well-studied, many different contexts pseudo-randomness, group theory, online routing, Markov chains, metric embeddings, … Uniform Sparsest Cut Problem Given a graph G compute ФGand find the set S achieving it. # edges leaving S expansion(S) = d |S| • Approximation Algorithms: • Cheeger’s Inequality [Alon][Alon-Milman] • Given a graph G, if the normalized adjacency matrix has second eigen value λ2 then, • A log n approximation algorithm [Leighton-Rao]. • A sqrt(log n) approximation algorithm using semidefinite programming [Arora-Rao-Vazirani]. Conductance of Graph G vertex set S minimum |S| ≤ n/2 ФG = expansion(S)

27. Limitations of Eigenvalues • The best lower bound that Cheeger’s inequality gives on expansion is (1-λ2)/2 < ½, while Ф(δ) can be close to 1. • Consider graph G Connect pairs of points on {0,1}nthat are εn Hamming distance away. Then Second eigenvalue≈ 1- εand ф(1/2) ≈ ε yet Ф(δ) ≈ 1 (small sets have near-perfect expansion) • A simple Sdp relaxation cannot distinguish between • all small sets expand almost completely • exists small set with almost no expansion

28. A Conjecture Small-Set Expansion Conjecture: 8η>0, 9± >0 such that GapSSE(η, ± ) is NP-hard, i.e., Given a graph G, it is NP-hard to distinguish YES: expansion(S) <η for some S of volume ¼± NO: expansion(S) > 1- η for all S of volume ¼±

29. Road Map • Algorithm: • Spectral Profile. • Reductions within Expansion • Relationship with Unique Games Conjecture

30. Gaussian Curve Γε(δ) = Gaussian noise sensitivity of a set of measure δ = least expanding sets are caps/thresholds of measure δ = Gaussian Graph Vertices: all points in Rd (d dimensional real space) Edges: Weights according to the following sampling procedure: • Sample a random Gaussian variable x in Rd • Perturb x as follows to get y in Rd Add an edge between x an y 1 Ф(δ) 0.5 Set Size δ

31. Approximating Expansion Profile

32. Reductions within Expansion

33. Reductions within Expansion • Theorem [Raghavendra-Steurer-Tulsiani 10] • For every positive integer q, and constantsε,δ,γ, given a graph it is SSE-hard to distinguish between: • There exists q disjoint small sets S1 , S2 , .. Sq of size close to 1/q, such that expansion(Si) ≤ ε • No set of size μ> δ has expansion less than size Γε/2(μ) -- expansion of set of size μ in Gaussian graph with parameter ε/2 • expansion (S) < sqrt (Λ* (δ)∙O(log 1/ μ))

34. Informal Statement 1 • Quantitative Statement • Given a graph G , it is SSE-hard to distinguish whether, • There is a small set of size whose expansion is ε. • Every small set of size μ (in a certain range) expands at least as much as the corresponding Gaussian graph with noise ε 1 Set Size δ Qualitative Assumption GapSSE is NPhard Ф(δ) 0.5 Set Size δ

35. Corollaries Corollary: The algorithm in[Raghavendra-SteurerTetali 10] has near-optimal guarantee assuming SSE conjecture. Corollary: Assuming GapSSE, there is no constant factor approximation for Balanced Separator or Uniform Sparsest Cut.

36. Relation with Unique Games Conjecture

37. Unique Games Unique game ¡ : Referee sample (A,B,¼) graph of size n label set L(A) A ¼ B Each edge (A,B) has an associated map ¼ ¼ is bijection from L(A) to L(B) A a B b Referee A labelling satisfies (A,B) if ¼ (label of A) = label of B players win if ¼(a) = b Player 1 Player 2 pick a in L(A) pick b in L(B) Goal: Find an assignment of labels to vertices such that maximum number of edges are satisfied. value( ¡ ): maximum success probability over all strategies of the players no communication between players

38. Unique Games Conjecture [Khot02] Unique Games Conjecture: [Khot ‘02] 8²>0, 9 q >0: NP-hard to distinguish for ¡ with label set size q YES: value( ¡ ) > 1-² NO: value( ¡ ) < ²

39. Implications of UGC Basic Sdp is optimal for … • Constraint Satisfaction Problems [Raghavendra`08] • Max Cut, Max 2Sat Metric Labeling Problems [MNRS`08] • Multiway Cut, 0-extension • Ordering CSPs [GMR`08] • Max Acyclic Subgraph, Betweeness UGC • Strict Monotone CSPs [KMTV`10] • Vertex Cover, Hypergraph Vertex Cover Kernel Clustering Problems [KN`08,10] Grothendieck Problems[KNS`08, RS`09] …

40. “Reverse Reductions” ? Basic Sdp is optimal for lots of optimization problems, e.g.: Max Cut and Vertex Cover UGC Basic Sdp optimal for Problem X * Win-Win Situation If we could show , then a refutation of UGC would imply an improved algorithm for Problem X * Problem X = Max Cut Parallel Repetition is natural candidate reduction for [FeigeKinderO’Donnell’07] * Bad news: this reduction cannot work [Raz’08, BHHRRS’08]

41. Small Set Expansion and Unique Games • Solving Unique Games  Finding a small non-expanding set in the “label extended graph” Theorem [Raghavendra-Steurer 10] Small Set Expansion Conjecture  Unique Games Conjecture Establishes a reverse connection from a natural problem.

42. Implications of UGC Basic Sdp is optimal for … • Constraint Satisfaction Problems [Raghavendra`08] • Max Cut, Max 2Sat Metric Labeling Problems [MNRS`08] • Multiway Cut, 0-extension • Ordering CSPs [GMR`08] • Max Acyclic Subgraph, Betweeness UGC UGC With expansion • Strict Monotone CSPs [KMTV`10] • Vertex Cover, Hypergraph Vertex Cover Gap SSE Kernel Clustering Problems [KN`08,10] Grothendieck Problems[KNS`08, RS`09] … Uniform Sparsest Cut [KNS`08, RS`09] Minimum Linear Arrangement[KNS`08, RS`09]

43. Most known SDP integrality gap instances for problems like MaxCut, Vertex Cover, Unique games have graphs that are “small set expanders” Theorem [Raghavendra-Steurer-Tulsiani 10] Small Set Expansion Conjecture  MaxCut or Unique Games on Small Set Expanders is hard. Reverse Connections?

44. Approximating Spectral Profile

46. Roadmap Introduction Graph Expansion: Cheeger’s Inequality, Leighton Rao, ARV Expansion Profile: Small Sets expand more than large ones. Cheeger’s inequality and SDPs fail GapSSE Problem Relation to Unique Games Conjecture: Unique Games definition, Applications, lack of reverse reductions. Label extended graph  Small sets Small Set Expansion Conjecture  UGC UGC with SSE is easy Algorithm for SSE: Spectral Profile, SDP for Spectral Profile, Rounding algorithm Relations within expansion: GapSSE Balanced Separator Hardness

48. NP-hard Optimization Example: Max Cut: partition vertices of a graph into two sets so as to maximize number of cut edges fundamental graph partitioning problem benchmark for algorithmic techniques

49. Approximation Max Cut Trivial approximation Random assignment, cut ½ of the edges approx-ratio 0.878 First non-trivial approximation Goemans–Williamson algorithm (‘95) based on a semidefinite relaxation (Basic Sdp) Beyond Max Cut: Analogous Basic Sdprelaxation for many other problems Almost always, Basic Sdpgives best known approximation (often strictly better than non-SDP methods)

50. Approximation Max Cut Trivial approximation Random assignment, cut ½ of the edges approx-ratio 0.878 First non-trivial approximation Goemans–Williamson algorithm (‘95) based on a semidefinite relaxation (Basic Sdp) Can we beat the approximation guarantee of Basic Sdp?