On the Unique Games Conjecture

On the Unique Games Conjecture Subhash Khot NYU Courant CCC, June 10, 2010

Approximation Algorithms A C-approximation algorithm for an NP-complete problem computes (C > 1), for problem instance I , solution A(I) s.t. Minimization problems : A(I)  C  OPT(I) Maximization problems : A(I)  OPT(I) / C

PCP Theorem [B’85, GMR’89, BFL’91, LFKN’92, S’92,……] [PY’91] [FGLSS’91, AS’92 ALMSS’92] Theorem : It is NP-hard to tell whether a MAX-3SAT instance is * Satisfiable (i.e. OPT = 1) or * No assignment satisfies more than 99% clauses (i.e. OPT  0.99). i.e. MAX-3SAT is 1.01 hard to approximate.

(In)approximability : Towards Tight Hardness Results • [Hastad’96]Clique n1- • [Hastad’97] MAX-3SAT 8/7 -  • [Feige’98] Set Cover (1- ) ln n [Dinur’05]Combinatorial Proof of PCP Theorem !

Open Problems in (In) Approximability • Vertex Cover (1.36 vs. 2) [DinurSafra’02] • Coloring 3-colorable graphs (5 vs. n3/14) [KhannaLinialSafra’93, BlumKarger’97] • Sparsest Cut (1+ε vs. (logn)1/2) [AMS’07,AroraRaoVazirani’04] • Max Cut (17/16 vs 1/0.878… ) [Håstad’97, GoemansWilliamson’94] ………………………..

Unique Games Conjecture [K’02] Supporting Evidence Connections to: • Inapproximability (UGC  several problems inapproximable) • Discrete Fourier Analysis (e.g. KKL, Majority is Stablest) • Geometry (Isoperimetry, Metric geometry, Integrality gaps) • Algorithms (Attempts to disprove UGC) • Parallel Repetition (Gap amplification, Foam construction)

Example of Unique Game (2CSP) OPT = max fraction of equations that can be satisfied by any assignment. x1 - x3 = 2 (mod k) x5 -x2 = -1 (mod k) x2 - x1 = k-7 (mod k) …………. ………….

Unique Game 2CSP w/ Permutation Constraints variable k labels Here k=4 constraints 

Unique Game2CSP w/ Permutation Constraints variable k labels Here k=4 Permutations or matchings  : [k]  [k]

Unique Game Find a labeling that satisfies max # constraints OPT(G) = 6/7

Unique Games Considered before …… [Feige Lovasz’92] Parallel Repetition of UG reduces OPT(G). How hard is approximating OPT(G) ? Observation : Easy to decide whether OPT(G) = 1.

Unique Games Conjecture For any , , there is integer k(, ), s.t. it is NP-hard to tell whether a Unique Game with k = k(, ) labels has OPT  1-  or OPT   i.e. Gap-Unique Game (1-  , ) is NP-hard. Gap Projection Game (1, ) is NP-hard. [ PCP Theorem + Raz’s Parallel Repetition Theorem ].

Supporting Evidence [UGC] Gap-Unique Game (1-ε, ) is NP-hard. [Feige Reichman’04] Gap-Unique Game (C, ) is NP-hard. However C  --> 0 as C --> ∞. [KV’05] SDP relaxation for UG has “integrality gap” (1-, ). [KV’05] UGC based predictions were proven correct. Specifically, metric embedding lower bounds. [Wishful thinking] “There is structure in CS/math”.

Small Set Expansion Conjecture [Raghavendra Steurer’ 10]Φ (S ) = Edge expansion of set S. For every ε > 0, there exists δ > 0, such that, it is NP-hard to tell whether in a graph G(V,E), • There is a set S, |S| = δ |V|,Φ (S) ≤ ε. • For every set S, |S| ≈ δ |V|,Φ (S) ≥ 1- ε. [Raghavendra Steurer’ 10] SSE Conjecture  Unique Games Conjecture.

Unique Game and Small Set Expansion |G’| = n k. S = Optimal labeling. |S| = 1/k |G ’|. Φ(S) = 1- OPT(G). Unique Game G with n variables, k labels

Linear Equations Over Reals [K Moshkovitz’10] Homogeneous 3LIN(R): x1 – x3 + 2 x5 = 0. ∙∙∙∙∙∙∙∙ eq: xi + .5 x j -x k = 0. Theorem: It is NP-hard to tell if : • There is a “non-trivial” solution that satisfies 1-ε fraction of equations. • Any “non-trivial” solution fails on a constant fraction of equations with error Ω(√ε). 3LIN(R) to 2LIN(R) reduction ? 2LIN(R) ≈ Sparsest Cut

Unique Games Conjecture [K’02] Connections to: • Inapproximability (UGC  several problems inapproximable) • Discrete Fourier Analysis (e.g. KKL, Majority is Stablest) • Geometry (Isoperimetry, Metric geometry, Integrality gaps) • Algorithms (Attempts to disprove UGC) • Parallel Repetition (Gap amplification, Foam construction)

Generic Reduction from Unique Game[BGS’95 (Long Code), Hastad’97 (Fourier), UGC , ……]

PCP Reduction Generic Reduction from Unique Game[BGS’95 (Long Code), Hastad’97 (Fourier), UGC , ……] MAX-CUT Instance Unique Game Instance k labels Gadget: {-1,1} k Match Goemans-Williamson’s SDP rounding Algorithm 1/0.878… Hardness OPT(UG) > 1-ε   sized cut. OPT(UG) < δ No cut with size arccos (1-2) / 

x y {-1,1} k Gadget : Dictatorships (Long Codes) • Consider f: {-1, 1}k {-1,1}, i.e. Cuts. • Encode label i Є {1,2,…., k} by dictatorship function f(x) = xi. Weighted graph, total edge weight = 1. Picking random edge : x R{-1,1} k y <-- flip every co-ordinate of x with probability  (  0.8) Noise-sensitivity graph.

xi = 1 xi = -1 Gadget: Cutthat“commits” toco-ordintae i Fraction of edges cut = Pr(x,y) [xi  yi ] =  Observation : These are the maximum cuts.

Gadget : Cuts not committing to a co-ordinate Influence (i, f) = Prx [ f(x)  f(x+ei) ] How large can be cuts with no influential co-ordinate ? Random Cut : ½ Majority Cut :  > arccos (1-2) /  > ½ [KKMO’04, MOO’05]Majority Is Stablest (Under Noise) Any cut with no influential co-ordinate has size at most arccos (1-2) / .

Integrality Gap Given : Maximization Problem + SDP relaxation. • For every problem instance G, SDP(G)  OPT(G) • Integrality Gap = Sup G SDP(G) / OPT(G)

[Raghavendra’ 08] • Duality between Algorithms and Hardness. • For every CSP, write a natural SDP relaxation. • Integrality gap = β. Implies β-approximation. • Theorem: Every instance with gap β’ < β can be used to construct a gadget and prove UGC-based β’- hardness result ! • SDPs are optimal algorithm for CSPs.

Inapproximability and Fourier Analysis • f : {-1,+1} k  {-1,+1}, balanced. • Sparsest Cut[KV’05, CKKRS’05] [KKL’88] f has a co-ordinate with influence Ω(log k /k). [Bourgain’02] If NSε(f) << √ε, then f depends essentially on exp(1/ε2) co-ordinates. • MAX-CUT[KKMO’04]Majority Is Stablest [MOO’05] If f has no influential co-ordinate, then NS ε(f) ≥ NS ε(Majority) - o(1).

Inapproximability and Fourier • f : {-1,+1}k  {-1,+1}, balanced. • Vertex Cover[DinurSafra’02, K Regev’03, K Bansal’09] [Friedgut’98] If total influence is k, then f depends essentially on exp(k) co-ordintaes. [MOO’05]It Ain’t Over Till It’s Over If f has no influential co-ordinate, then on almost every subcube of {-1, +1}k of dimension k/100, f = 1 and f = -1 with constant probability.

Inapproximability and Fourier • f : {-1,+1}k  {-1,+1}, balanced. • MAX-k-CSP [Samorodintsky Trevisan ’06] If f has no influential co-ordinate, then f has low Gowers’ Uniformity norm. Open: f: [q] k  [q], q ≥ 3, no influential co-ordinate. • f balanced. Is Plurality Stablest ? • What is the maximum Fourier mass at the first level ?

Disproving UGC means .. For small enough (constant), given a UG with optimum 1- , algorithm that finds a labeling satisfying (say) 50% constraints, irrespective of k = #labels.

Algorithmic Results Algorithm that finds a labeling satisfying f(, k, n) fraction of constraints. [K’02] 1- 1/5 k2 [Trevisan’05]1- 1/3 log1/3 n [Gupta Talwar’05] 1-  log n [CMM’05] 1/k , 1- 1/2 log1/2 k [CMM’06] 1-  log1/2 k log1/2 n [AKKSTV’08 , Kolla’10] UG on “mild” expander graphs. [ABS’10] Exp ( n ) time algorithm. None of these disproves UGC. However …

If the UGC is true, then : • k >> 21/ε . • Graph of constraints cannot even be a “mild” expander. UG is easy on random graphs. • Reduction from 3SAT must blow up the size by n1/ε . • Conjecture does not hold for sub-constant ε, i.e. below 1/log n.

Orthonormal Bases for Rk v1 , v2 , … , vk v variables k labels u u1 , u2 , … , uk Matchings [k]  [k] SDP Relaxation of Unique Games [FL’92] • OPT(G) = 1- εSDP(G) ≥ 1- ε . • For i = 1, …, k,  ui , vi  ≥ 1- ε , up to permutation of indices.

vk v2 v1 uk u2 u1 [K’02, CMM’05] Rounding Algorithm r r Random r u v Pick the label closest to r. Label(u) = Label(v) = 2. Pr [ Label(u) = Label(v) ] > 1 - 1/5 k2[K’02]. Pr [ Label(u) = Label(v) ] > 1- 1/2 log1/2 k[CMM’05]. • Labeling satisfies 1- 1/2 log1/2 k fraction of constraints in expected sense.

[Trevisan’05] Algorithm Graph of variables and constraints • [Leighton Rao’88] Delete 1% of edges so that all connected components have diameter O(log n). • Algorithm to solve UG on low diameter graph.

[AKKSTV’08 Algorithm] Algorithm that works on a UG instance s.t. • 1-ε satisfiable and, • Every balanced cut in the graph cuts at least Ω ( √ε ) fraction of edges. • SDP-based. • “Mild” expansion Almost all SDP vector tuples are nearly identical  Yields a good labeling.

Unique Game and Small Set Expansion Label extended Graph |G’| = n k. S = Optimal labeling. |S| = 1/k |G ’|. Φ(S) = 1- OPT(G). UG G with n variables, k labels

[Arora Barak Steurer’10 Algorithm][Kolla’10, Naor’10] • Algo. runs in time exp(nε) on UG that is 1-ε satisfiable. • Good solution to UG  Small non-expanding set S in G’. • Small non-expanding set in label-extended graph G’ Either corresponds to a good UG solution (useful) Or is a non-expanding set in G (fake). • Iteratively remove all fake sets from G, sacrificing at most 1% edges.

[Arora Barak Steurer’10 Algorithm][Kolla’10, Naor’10] Main Lemma (Algorithmic) : If every set of size n1-ε expands by Ω(ε2), then the number of eigenvalues exceeding 1-ε is nO(ε). • The UG solution is found in the linear span of eigenvectors with eigenvalues ≥ 1-ε. [Kolla’10] • Run-time exp ( nO(ε) ).

(Gaussian) Isoperimetry . [MOO’05]Majority Is Stablest reduces via Invariance Principle, to a geometric question: P: Rn  {-1,+1} be a partition of Gaussian space into two sets of equal measure. NSε(P) = Pr [ P(x) ≠ P(y) ], Cor (x,y) = 1-2ε. Which P minimizes the noise-sensitivity? [Borell’85] NSε(P) ≥ NSε( HALF-SPACE THRU ORIGIN ).

(Gaussian) Isoperimetry Open: q ≥ 3. More Invariance. [IM’10] MAX-q-CUT Problem. Plurality is Stablest Conjecture. Partition Rn into q equal parts. (Geometric): Standard Simplex Conjecture. [K Naor’08]Kernel ClusteringProblem. Maximizing Fourier Mass at First Level. (Geometric): Propeller Conjecture.

Integrality Gap • [Feige Schechtman’01] [Goemans Williamson’92] 1/0.878.. Integrality gap for MAX-CUT. • SDP with “triangle inequality constraints” ? • ω(1) Integrality gap for Sparsest Cut? • UGC  NP-hardness  These integrality gaps exist.

Orthonormal Bases for Rk v1 , v2 , … , vk v variables k labels u u1 , u2 , … , uk Matchings [k]  [k] [KV’05] Integrality Gap for Unique Games SDP SDP(G) = 1-o(1) Unique Game G with OPT(G) = o(1)

u1 , u2 , … , uk Integrality Gap for MAX-CUT with Triangle Inequality OPT(G) = o(1) PCP Reduction No large cut Good MAX-CUT SDP solution  u1  u2  u3 ……… uk-1  uk {-1,1}k

MAX-CUT and Sparsest Cut I.G. • [KV’05] MAX-CUT gap matching Goemans-Williamson even with triangle inequality constraints. • [KV’05, KrauthgamerRabani’05, DKSV’06] (loglog n) integrality gap for Sparsest Cut SDP.  An n-point “negative type” metric that needs distortion (loglog n) to embed into L1.  Refutation of [Goemans Linial’97, ARV’04] conjectures. • [KS’09, RS’09] Similar gaps for SDP + Sherali-Adams LP. Negative type metric that is L1 embeddable locally but not globally.

Open Problems • Integrality gaps for the Lasserre SDP Relaxation? Lasserre Relaxation could potentially disprove UGC. • Sparsest Cut (NEG versus L1 Metrics) : [ARV’04, AroraLeeNaor’05] O(√log n). [LeeNaor’06, CheegerKleiner’06, CKNaor’09]. Ω(logc n), c = ½?

Gap Amplification Prove UGC in two steps (?): • Prove “mild” hardness, i.e. GapUG (1-ε’ , 1-ε’’ ) is hard. • Amplify gap via parallel repetition to GapUG (1-ε , δ). Note however that even proving “mild” hardness is a huge challenge.

Strong Parallel Repetition ? OPT(G) = 1-ε. [Raz’98] OPT(Gm) ≤ (1-ε32 )m/log k . 2P1R Games [Holenstein’07] OPT(Gm) ≤ (1-ε3 )m/log k . 2P1R Games [Rao’08] OPT(Gm) ≤ (1-ε2)m . Projection Games (UG). GapUG (1-ε , δ ) is NP-hard iff GapUG (1-ε , 1 - √ε C(ε) ) is NP-hard where C(ε) –> ∞ as ε –> 0. [Raz’08] The rate (1-ε2)m cannot be improved further.

On the Unique Games Conjecture