M. Krivelevich, D. Vilenchik SODA 2006

1. M. Krivelevich, D. VilenchikSODA 2006 Solving Random Satisfiable 3CNF Formulas in Expected Polynomial Time

2. Lecture Outline • What is expected polynomial time and some motivation • The planted SAT distribution and related work • Description of our algorithm • Outline of the analysis • Open problems

3. Why Consider Prob. Models ? • Many interesting problems are known to be NP-hard • Hardness results only show that there existhard instances • Should not discourage us from trying to designheuristics that work well for “almost all” instances • For rigorous analysis - define “almost all” in meaningful way • One possibility - use probabilistic models such as Gn,p

4. Expected Polynomial Time • D - a distribution on the inputs • Algorithm works whp over D, if it succeeds whp when instance sampled according to D • Such algorithm may fail completely on some instances • E.g. Greedy Coloring Algorithm: • Fix the vertices in some arbitrary order • For every vertex, assign minimal possible color

5. Expected Polynomial Time • Greedy uses whp at most n/logn colors for Gn,½ [GM75] • (Gn,½) ~ n/2logn whp Therefore, • Greedy yields whp 2-approximation of(G) for G2Gn,½ However, • Let G=Kn/2,n/2 minus some perfect matching • Greedy uses n/2 colors - order vertices according to matching • (G)=2 greedy fails completely

6. Expected Polynomial Time Cont. Alternatively, demand success for all instances while keeping an overallaveragepolynomialtime Formally … Def. Algorithm A with running time tA(I) on I runs in expected polynomial time over distribution D if PrD[I]¢tA(I) is polynomial in n

7. Expected Polynomial Time Cont. • To achieve this – separate “easy” instances (can be handled in polynomial time) from “hard” ones (rare, but may require super-polynomial time) • Requires a betterunderstanding of the probability space • Encourages efficient, natural and more robust algorithms

8. What’s Next ? • What is expected polynomial time and some motivation • The planted SAT distribution and related work. • Description of our algorithm. • Outline of the analysis. • Open problems.

9. 3SAT - Definition literal clause 3CNF form: (x1Ç x2 Ç ¬x5)Æ(x3Ǭx4Ǭx1) Æ (x1Ç x2Çx6) Æ… Partial truth assignment: • 3SAT = {all satisfiable 3CNF formulas}. • 3SAT is NP-complete [Cook71].

10. Different SAT Distributions • (Arguably) most natural distribution - Pn,p • Include every possible clause w.p. p=p(n) • Let  = expected number of clauses / n, • Satisfiability shows sharp threshold behavior [Fri99] • < 3.42, almost all instances are satisfiable[KKL02] • > 4.5, almost all are unsatisfiable[KKS+01] • Our focus is =d, d a sufficiently large constant Analog of Gn,p

11. Different SAT Distributions • Pn,p not interesting at such ratios (for satisfiability algorithms) Alternatively … • Consider distributions over satisfiable instances • One possibility, PSATn,p where PSATn,p (I) = Pn,p(I | I is sat.) • PSATn,p is hard to sample (experimentally) • PSATn,p seems hard to tackle rigorously (no efficient algorithm known for =o(logn))

12. Different SAT Distributions • Planted SAT can serve as intermediate step towards PSATn,p • It is interesting and well studied on its own right • It is the analog of Planted k-Coloring[BS95], [AK97], Planted Clique[AKS98], [FK00] • It is a random distribution over satisfiable3CNF formulas with arbitrarily large clauses/variables ratio • Can be efficiently sampled

13. The Planted 3SAT Distribution • Generating an instance: • Randomly pick a truth assignment  • Include every clause satisfied by  w.p. p=d/n2 E.g. (x1Ç x2Ç ¬x5)Æ(x3Ǭx4Çx1)Æ(¬x1Ç x2Ç x6)Æ…

14. Planted Distributions: Related Work • [KP92] - greedy variables assignment, p≥d/n (Implicitly) works in expected polynomial time • [AK97] – spectral technique for coloring sparse planted 3-colorable graphs (np=d) • [BSBG02] – majority vote suffices for p≥d¢logn/n2 • [Fla03] – techniques similar to [AK97], solves whp planted 3SAT, p≥d/n2

15. Related Work Cont. • [CO04] – SDP basedexpected polynomial time algorithm for (semi-random) planted k-colorable graphs, np≥d¢k¢logn • [Böt05] –SDP basedexpected polynomial time algorithm for planted k-colorable graphs, np≥d¢k2

16. What’s Next ? • What is expected polynomial time and some motivation • The planted SAT distribution and related work • Description of our algorithm • Outline of the analysis • Open problems

17. Our Results • An algorithm that decides 3SAT • Expected polynomial running timeover planted 3SAT, p=d/n2 • Result extends to any constant k(in which case d=d0k) • First work to address the issue of expected poly. time algorithms for satisfiable SAT distributions.

18. Algorithm: General Outline Most expected poly. time heuristics discard the solution and exhaustivelysearch for a correct one correct means coincides with the planted solution The algorithm proceeds in 2 steps: • Find a partial correct solution containing a large fraction of variables (always poly time) • a. Try to complete the partial solution to a satisfying assignment b. If not possible, gradually fix the partial solution until step 2.a ends up successfully (steps a+b run in expected poly. time) Typically, all but a small constant, e-(d), fraction

19. Algorithm: Basic Ingredients The Majority Vote: (x1Çx2Ǭx3)Æ(x4Ç x2Ǭx1)Æ(¬x1Ç x2Ç x4)Æ(x3Ǭx2Ç x4) F T T T

20. Basic Ingredients Cont. The Unassignment Procedure: • If C = (x Ç :y Ç z)!(T Ç F Ç F), then x supports C w.r.t  • Note: all three variables are assigned by  E.g. unassignment with threshold t =1 (x1Çx2Ǭx3)Æ(x4Çx2Ǭx1)Æ(¬x1Çx2Ç ¬x4)Æ(x3Ǭx1Ǭx4) (T Ç F Ç F) Æ (T Ç F Ç F) Æ (F Ç F Ç F ) Æ( T Ç F Ç F ) * * * * * * * * * * * * • Unassignment stops when all remaining variables d support at least t clauses

21. Basic Ingredients Cont. If every component is of size O(logn), the procedure is polynomial. The Exhaustive Search: • Given 3CNF formula I, define its induced graph GI=(V,E): • V = {x1, x2, …, xn} - the set of variables • (xi,xj)2E if 9 clause C containing both (polarity disregarded) • Given I, find the connected components in GI • Search every component separately for a satisfying assignment

22. Basic Ingredients: Motivation Wrongly assigned by the Majority. We call such variable wrong variable. • Assume input sampled according to planted 3SAT • Suppose (x)=T • In every clause, x appears w.p. 4/7¢ 3/n, :x w.p. 3/7¢ 3/n Therefore, • Majority Vote approximates closely whp • Suppose a wrongly assigned variable survives unassignment (TÇ F Ç F) But we also expect the majority to wrongly assign some variables whp (small fraction) Must be another wrong variable surviving the unassignment F T

23. Motivation Cont. • W - the set of wrong variables surviving unassignment • There exist at least t¢|W | clauses, each containing at least 2 variables from W • We call such Wdense • If |W | is small, this is analogous to small subgraph withatypically high average degree • This happens with small probability in random graphs, Gn,p each clause was counted once, as the support is unique.

24. Algorithm: General Outline Majority Vote + Unassignment The algorithm basically proceeds in 2 steps: • Find a partial correct solution containing a large fraction of the variables • a. Try to complete the partial solution to a satisfying assignment b. If not possible, gradually fix the partial solution until step 2.a ends up successfully. Exhaustive Search Make sure algorithm always succeeds.

25. Putting Everything Together d/2 is the expected support Algorithm SAT(I): • MAJ ÃMajority Vote of I. • Carry unassignment with threshold 0.999d/2 w.r.t MAJ. • Let  be the partial assignment. • Let U be the set of unassigned variables. • Construct G=(U,E). • For all subsets Y µ V\U, |Y|=0..|V\U|, and for all possible assignments Y of Y: • Fix  according to Y. • Using exhaustive search on G(U,E) try to complete  to a satisfying assignment. • If success, return the assignment. completeness soundness Y is the fixing set of variables

26. What’s Next ? • What is expected polynomial time and some motivation. • The planted SAT distribution and related work. • Description of our algorithm. • Outline of the analysis. • Open problems.

27. Analyzing the Running Time Algorithm SAT(I): • MAJ ÃMajority Vote of I. • Carry unassignment with threshold 0.999d/2 w.r.t MAJ. • Let  be the partial assignment. • Let U be the set of unassigned variables. • Construct G=(U,E). • For all subsets Y µ V\U, and for all possible assignments Y of Y: • Fix  according to Y. • Using exhaustive search on G(U,E) try to complete  to a satisfying assignment. • If success, return the assignment. Expected to perform O(1) times Expected running time O(n1+) Always polynomial. In fact expected linear time

28. Analysis Outline Typically (for Planted 3SAT), the following happens: • Distance between MAJ and the planted assignment is e-(d)n • Almost all correct variables, (1-e-(d) )n,surviveunassignment • Only correct variables survive the unassignment • G=(U,E) breaks down to O(logn)-size connected components Therefore, • Exhaustive search is successful and polynomial similar arguments to Gn,p, np<1 “Density” arguments

29. Analysis Outline • What can go wrong, preventing successful execution ? • Wrong variables survived the unassignment: • The partial assignment induces a (FÇFÇF) clause • Formula induced by unassigned variables is not satisfiable • Y0- the set of fixingvariables with which the algorithm ends • Typically, Y0=;

30. Analysis Outline Cont. Key observation: if Y0; then: • The Majority Vote is wrong for at least |Y0| variables • Y0 is a dense set of variables • For “large” |Y0|, (1) happens with small probability • For “small” |Y0|, (2) happens with small probability • It remains to carry out the exact calculations Suppose x 2 Y0! Otherwise, the algorithm would have ended with a smaller set Y’  Y0. x survives the unassignment ! x supports ~d/2 clauses ! x y (TÇ F Ç F) F T ! y2Y0, otherwise, algorithm can not end

31. A Taste of Rigorous Analysis The following properties hold whp for Planted 3SAT: • Let 0=e-d/C0 • FMAJ - the set of variables on which MAJ and  disagree • Claim:fory¸0n, Pr[|FMAJ|¸y] · e-yd/C1 • For JµV, F(J) is the set of clauses in I containing at least 2 variables from J • Claim:Pr[9J, |J|·a0n, |F(J)|¸|J|d/3]· e-|J|log(n/|J|)d/12 • Properties proved using standard probabilistic techniques (union bound, Chernoff)

32. A Taste of Rigorous Analysis The expected number of fixing iterations is at most:

33. A Taste of Rigorous Analysis

34. Open Problems • [FV04] show a k-opt based heuristic solving whp Planted 3SAT, p=d/n2 • Change k-opt version to run in expected polynomial time • Challenge: no explicit distinction between wrong and correct variables • Simplify [Böt05], e.g. replacingSDP approximation with simpler and stronger procedure (similar to Majority Vote) • Design an efficient algorithm for random (not planted) satisfiable formulas, p=d/n2