Approximability & Sums of Squares

1. Approximability& Sums of Squares Ryan O’Donnell Carnegie Mellon

2. Basic Optimization Problems Minimum-Balanced-Separator: Given G=(V,E), partition V into 2 parts,each of size at least n/3, minimize # of edges crossing partition.

3. Basic Optimization Problems Minimum-Balanced-Separator: Given G=(V,E), partition V into 2 parts,each of size at least n/3, minimize # of edges crossing partition. Minimum-Vertex-Cover: Given G=(V,E), choose the smallestsubset S ⊆ V such that each edge touches S.

4. Both are NP-hard poly(n) time n-vbl 3SATformula F O(n)-vtxgraph G , β ⇒ F satisfiable Min-BS(G) = β ⇒ F unsatisfiable Min-BS(G) > β Distinguishing requires* at least 2Ω(n) time. Distinguishing requires* at least 2Ω(n) time. ⇒

5. Approximate Optimization “C-approximation algorithm” Stronger Guaranteed to find a solution with value at most C times the minimum. “C-certification algorithm” • Output form: “I certify the minimum is ≥α”. • Must always be correct. • Guaranteed that α≥ (true minimum) / C.

6. Minimum Balanced-Separator Is there a 1.01-approximationalgorithm running in O(n) time? DON’TKNOW Is there a 10000-certificationalgorithm running in 2n.99time? DON’TKNOW [AMS’11]: Cannot* 1.0000000000000001-certify in poly(n) time.

8. Minimum Vertex-Cover Can 2-approximate in linear time. Cannot* 1.17-certify even in 2n.99999 time. Cannot* 1.36-certify even in 2n.000001 time. Can you 1.5-certify in polynomial time? DON’TKNOW

9. How could you show that you can’t 1.5-certify Min-VC in poly time? poly(n) time n-vbl 3SATformula F O(n)-vtxgraph G , β ⇒ F satisfiable Min-VC(G) ≤β ⇒ F unsatisfiable Min-VC(G) > 1.5β DON’TKNOWHOW This would show 1.5-certifying Min-VCrequires* superpolynomial time.

10. How could you show that you can’t 1.5-certify Min-VC in poly time? give evidence that Show that known powerful poly-timeoptimization techniques fail to do it.

11. Prehistory: Linear programming can’t 1.999999-certify Min-VC. [GK’95]: Semidefinite programming can’t 1.999999-certify Min-VC. [ABL’02]: Lovász-Schrijverd Super-LP can’t1.999999-certify Min-VC. nO(d) time [GMPT’07]: Lovász-Schrijverd Super-SDPcan’t1.999999-certify Min-VC. [BCGM’10]: Sherali-Adamsd Super-Duper-SDPcan’t1.999999-certify Min-VC. +++

12. For Min-Balanced-Separator, a similar situation: [KS’09], [RS’09]: Sherali-Adamsd Super-Duper-SDPcan’t10000-certify Min-Bal-Sep.

13. Prehistory: Linear programming can’t 1.999999-certify Min-VC. • I.e., there are graphs G on n vertices such that: • Min-VC(G) ≥ .999999n • LP(G) = “I certify Min-VC(G) ≥ .500001n” LP certif. alg. for Min-VC outputs α, where α = minimize: ∑v∈V Xv [0,1] subject to: Xv ∈ {0,1} for all v∈V Xu + Xv≥ 1 for all (u,v)∈E

14. [BCGM’10]: Sherali-Adamsd Super-Duper-SDPcan’t1.999999-certify Min-VC. • I.e., there are graphs G on n vertices such that: • Min-VC(G) ≥ .999999n • SAd(G) = “I certify Min-VC(G) ≥ .500001n” Specifically, this is true for “Frankl-Rödl graphs” [FR’87]: V = {0,1}m, E = {(x,y) : ∆(x,y)=.999m}

15. [KS’09], [RS’09]: Sherali-Adamsd Super-Duper-SDPcan’t10000-certify Min-Bal-Sep. • I.e., there are graphs G on n vertices such that: • Min-BS(G) ≥β • SAk(G) = “I certify Min-BS(G) ≥”. Specifically, this is true for “Khot-Vishnoi graphs” [KV’05].

16. These are tough instances. We, the mathematicians, can analyze their opt. But our strongest poly-time algorithms cannot.

17. Actually…

18. There is one more algorithm…

19. It’s even stronger, but hard to analyze…

20. The “Lasserred Super-Duper-Ultra-SDP”…

21. Also known as… SOSd nO(d) time

22. Our Results [OZ’13]: SOS4 is a C-certification algorithm (for some small C, maybe 5) for Min-BS on Khot-Vishnoi graphs. SOSd is also pretty good for Max-Cut on Khot-Vishnoi graphs. [KOTZ’13]: SOSd is essentially a 1-certif. alg. for Min-VC on all but the ‘hardest’ Frankl-Rödl graphs.

23. So your whole result is thatone particular algorithmdoes well on one particular instance?

24. An Old Joke Q: Why did the complexity theorist work on algorithms? A: To get lower bounds on his lower bounds. SOSd is a dozen years old, but hard to analyze. The Dream: it’s great certification alg. not justfor these known hard graphs, but for all graphs.

25. Our Inspiration: STOC’12 paper of Barak, Brandão, Harrow, Kelner, Steurer, and Zhou. • Showed SOS4 is good certification alg.on known hard instances of “Unique-Games”. • Somewhat demystified analysis of SOSd.

26. So what is SOSd?

27. “Min-Balanced-Separator(G) >α” ⇔ “ has no real solutions”

28. infeasibility certificate: identity −1 = Q0 + Q1P1 + Q2P2+ ••• +QmPm where each Qi is a “sum of squares”: Qi = Ri12 + ••• + Rik2

29. Positivstellensatz Subject to some mild technical conditions,every infeasible system has such a certificate. Caveat: Qi’s might need to have high degree. SOSd algorithm:[Shor’87,Lasserre’00,Parrilo’00] If there existsan infeasibility certificate where all the Qi’s have degree ≤ d, finds it in time nO(d).

30. E.g.: SOSd for Min-VC(G) “Min-VC(G) > α” ⇔ infeasible Xv2 = Xv for all v∈V,Xu+Xv≥ 1 for all (u,v)∈E, ∑v Xv≤α ⇐ existence of sum-of-squares Q’s such that −1 = Q0 + Q1(α−∑ Xv) + ∑ Quv (Xu+Xv−1) + ••• Find largest α such that degree-d Q’s exist.

31. Our Results [OZ’13]: SOS4 is a C-certification algorithm (for some small C, maybe 5) for Min-BS on Khot-Vishnoi graphs. I.e., for Khot-Vishnoi graphs G, there are degree-4 SOS Q’s certifying “Min-Bal-Sep(G) > α” for some α > (true Min-Bal-Sep) / C.

32. One Slide How-To Thm: Min-VC in this graph is ≥ .999nProof: … vertex isoperimetry…… inductive argument… “Check out these polynomials.” Thm: Min-BS in this graph is ≥ blahProof: … hypercontractivity… “Check out these polynomials.”

33. Tiny Taste A bit of the analysis for Max-Cut: Lemma: Let a,b,c ∈ {−1,1}. If a ≠ c then either a ≠ b or b ≠ c. Formalization with polynomials: SOS Proof:

34. Open Problems Can you give an SOS proof of… • Vertex Isoperimetric Theorem in {0,1}n: If A, B ⊆ {0,1}n, |A|,|B| ≥ .1·2n, then ∃x∈A,y∈B with ∆(x,y) ≤ • Central Limit Theorem

35. Thanks! +