An algorithmic perspective on Unique Games

1. An algorithmic perspective onUnique Games Moses Charikar Joint work with Konstantin Makarychev Yury Makarychev Princeton University

2. 8 ( ) d 2 1 3 3 1 7 + ´ x x m o 1 4 > > > ( ) d 1 6 4 1 7 + < ´ x x m o 3 2 > : : : > > ( ) d : 5 3 9 1 7 + ´ x x m o 1 9 Example • Linear equations mod p, two var’s per equation. Maximize # of satisfied constraints.

3. Unique Games

4. Unique Games Permutations k labels

5. Unique Games Goal: Satisfy as many constraints as possible.

6. 2 colors: Max Cut Two colors: Red and Blue Maximize the number of pairs of adjacent vertices colored with distinct colors

7. Greedy Algorithm

8. Unique Games Conjecture • Unique Games Conjecture[Khot’02] Given a Unique Games instance where 1-fraction of constraints is satisfiable, it is NP-hard to satisfy even fraction of all constraints (for every constant positive  and  and sufficiently large k). • Used to prove (optimal ?) hardness of approximation results for several problems • seem difficult to obtain by standard complexity assumptions.

9. Hardness Results Assuming UGC with UGC without UGC

10. Algorithmic Motivation • Semidefinite programming techniques very useful for binary constraint satisfaction problems • Seems difficult to extend techniques for problems over larger domains • Unique Games is a good test case

11. ~ = 2 1 5 ( ) k O 1 ¡ " ( ) k O 1 ¡ " p 3 ( ) k O 1 ¡ p " ( ) k O 1 ¡ " Approximation Algorithms Assume 1- fraction is satisfiable,k ~ log n. • Random Assignment: 1/k. • [Andersson, Engebretsen, Hastad ’01], slightly better than random for lin. eq. • [Khot ’02], SDP based algorithm, • [Trevisan ’05], SDP based algorithm, • [Gupta, Talwar ’06], LP based, [GT]

12. = = k l k 1 1 o g = k 1 1 0 = k 1 Comparison For what  can we satisfy constant fraction of constraints ? Assume k ~ log n.

13. ³ ´ p l k O 1 ¡ " o g " ¡ ¡ ¢ k ­ ¡ 2 " Our results • Given an instance where 1-fraction of constraints is satisfiable, 1st algor. satisfies • The 2nd algorithm satisfies:

14. p ( ) l k O 1 ¡ " o g " ¡ k ¡ 2 " Near Optimality • Khot, Kindler, Mossel and O'Donnell showed that even a slight improvement of our results refutes the UGC.

15. p h l k G i t t > ( ¢ v e n a g u o g " ¡ k ¡ 2 " p [ ] h l k ? P i t > ¢ w a s r g v o g p ( ) l k O 1 ¡ " o g Matching upper and lower bounds ? g Gaussian random vector v u u · v = 1 

16. Roadmap SDP based algorithm for unique games Approaches to disproving UGC

17. Ã ! k [ ] h i 8 8 k 6 V 1 i j i j 0 2 2 u u u = = X X i j 2 ; ; ; j j i ¡ m n u v ( ) i i ¼ k 2 u v X i ( ) 1 2 E j j 2 8 V = 1 u v 2 u u ; = i i 1 = 2 ( ) [ ] h i j j 8 k E i 0 · · 2 2 u v u v u i j i ; ; Semidefinite Program

18. Ã ! k [ ] h i 8 8 k 6 V 1 i j i j 0 2 2 u u u = = X X i j 2 ; ; ; j j i ¡ m n u v ( ) i i ¼ k 2 u v X i ( ) 1 2 E j j 2 8 V = 1 u v 2 u u ; = i i 1 = 2 ( ) [ ] h i j j 8 k E i 0 · · 2 2 u v u v u i j i ; ; SDP Interpretation Pr[(u,v) constraint not satisfied] Pr[xu = i] Pr[xu = i and xv=j]

19. u v v u k k 2 2 v 1 u 1 Intuition For each vertex, we have an orthogonal system of vectors. For adjacent vertices the vectors are close. Our goal is to pick one vector for each vertex. Green vectors correspond to vertex u. Red vectors correspond to vertex v.

20. h i Z g u = i i ; 2 j j = k 1 u = i Algorithm: first attempt • Pick value for xu based on ui • Pick a random Gaussian vector g. • Project g on ui: . • Pick i with largest projection Zi • Works in uniform case:

21. ~ u i = j j ~ u u u = i i i Non-uniform Case • Long vectors will be chosen with disproportionately high probability. • Normalize all vectors: • To ensure Pr[xu = i ]|ui|2, project on several Gaussians; #projections is proportional to |ui|2.

22. 2 j j k ¢ r u » i u i h i f Z ~ 1 · · g u o r r r = i i r r u ; ; i ; Multiple Projections • Let be the number of projections. • Pick independent Gaussian random vectors g1, …, gk. • Let . • Picki with largest projection Zi,r

23. p ( ) l l k O 1 ¡ " o g n o g p p p ( ( ( ( ( ) ) ) ) ) l l l l k O O O O O 1 1 1 1 1 ¡ ¡ ¡ ¡ ¡ " " " " o o o g g o " g g n n n Max Cut vs. Unique Games [GVY ‘93] [GT ‘06] [GW ‘95] [CMM ‘06] [C’MM ‘06] [ACMM ’05]

24. But UGC is just a conjecture … • UGC seems to predict limitations of SDPs correctly • UGC based hardness for many problems matching best SDP based approximation • UGC inspired constructions of gap examples for SDPs • Disproof of Goemans-Linial conjecturel22 metrics do not embed into l1with constant distortion. [KV ’05]

25. Is UGC true ? • Points to limitations of current techniques • Focuses attention on common hard core of several important optimization problems • Motivates development of new techniques

26. Approaches to disproving UGC • Focus on possibly easier problems • Max Cut: • OPT = 1-, beat 1-1/2[GW ‘94] • Max k-CSP: • constraints are conjunctions of k literals • maximize #satisfied constraints • Beat k/2k[ST ‘06][CMM ‘07] • Distinguish between 1/k and 1/2k satisfiable

27. ¹ ´ x x i i ¡ ¹ ^ ^ ^ ^ ^ ^ ¢ ¢ ¢ ¢ ¢ ¢ x x x x x x i i i i i i ¡ k k 1 1 2 2 Max k-CSP • Constraints are conjunctions • Maximize number of constraints satisfied • Let us denote • Then

28. T 1 = C 1 2 ( ) T + + + ¢ ¢ ¢ z z z = C i i i ^ ^ ^ k 1 2 2 ¢ ¢ ¢ k z x x x x i i i i i k 1 2 Integer Relaxation • Introduce a {-1, 1}-variable for each Boolean variable . -1 encodes false, 1 encodes true. • For each constraint C consider the term • If the constraint is satisfied, then .

29. 2 ( ) + + + ¢ ¢ ¢ z z z X i i i k 1 2 m a x 2 k f g C i i = k 1 ; : : : ; ¡ z z = i i ¡ f g 1 § 2 z i Integer Relaxation Note: this integer program is a relaxation. Can be solved using algorithm by Nesterov.

30. p = ± k ( 2 ( ) = = ± 1 2 t + r u e w p z i ; . . 1 x = i 2 ( ) = f l ± ( ) 1 2 T ¡ + + + a s e w p z ¢ ¢ ¢ i z z z = C i i i ; . . k 1 2 2 k z x k i i ' [ ] ¯ d P C T i t r s a s e C k 2 Algorithm • Solve the integer program, find . • Let . • For each i, let • Flip the values of all with prob. 1/2. • We show:

31. 1 1 2 [ ] [ ( ) ] E T E + + + ¢ ¢ ¢ z z z = = C i i i k 1 2 2 k k Bottleneck • Objective function cannot distinguish between random instances and those where OPT=1/k

32. Approaches to disproving UGC • Lifting procedures for SDPs • Lovasz-Schrijver, Sherali-Adams, Lasserre • Simulate products of k variables • Can we use them ? • Relaxation for k-CSP using sum of quartic terms ? • Max Cut • OPT = 1-, beat 1-1/2 using f(1/) rounds of lifting ?

33. Moment matrices • SDP solution gives covariance matrix M • There exist normal random variables with covariances Mij • Basis for SDP rounding algorithms • There exist {+1,-1} random variables with covariances Mij/log n • Is something similar possible for higher order moment matrices ?

34. Conclusion and Future Goals Precise tradeoff of parameters in Unique Games Conjecture Algorithms require geometric insights: Is geometry intrinsic ? Prove or disprove the Unique Games Conjecture