1 / 56

Paul Beame University of Washington

Satisfiability and Unsatisfiability: Proof Complexity and Algorithms. Paul Beame University of Washington. Outline. Intro to proof complexity Proof complexity and complete SAT algs Review Resolution/DPLL connection Application to random k-SAT Proof complexity and DPLL extensions

devi
Download Presentation

Paul Beame University of Washington

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Satisfiability and Unsatisfiability: Proof Complexity and Algorithms Paul Beame University of Washington

  2. Outline • Intro to proof complexity • Proof complexity and complete SAT algs • Review Resolution/DPLL connection • Application to random k-SAT • Proof complexity and DPLL extensions • Clause learning • Caching • Proof system survey

  3. Proof Complexity • Study of the number of symbols required for proofs of unsatisfiability (or tautology) in propositional logic • Does not address algorithmic issue • How would you find short proofs if they existed? • Existence of short proofs for every unsatisfiable formula is equivalent to NP = co-NP (and is implied by P=NP) • Generally believed that such proofs don’t exist

  4. Propositional proof systems • A propositional proof system is a polynomial time algorithmV s.t. for all formulas F • Fis unsatisfiable there exists a stringP s.t. V accepts input(P,F) • Note: •  direction is usually called soundness •  direction is usually called completeness • Proof complexity is size ofPas a function of the size ofF

  5. Sample propositional proof systems • Truth tables • proof is a fully filled out truth table • easy to verify that it is filled out correctly and all truth assignments yield T • Frege systems • inference rules:e.g. modus ponens A,(A B) |B • axioms: e.g. excluded middle| (A  A) • axioms & inference rules are schemas • can make consistent substitution of arbitrary formulas for variables in schema • e.g. excluded middle yields ((xy) (xy))

  6. F5 F1 F2 F7 F6 F3 F8 F10 F11 F4 F9 F12 F13 The graph of a proof Axioms/inputs are sources Inference rule associated with each node Sink labelled by tautology (orLfor refutation)

  7. Resolution • Start with clauses of CNF formula F • Resolution rule • Given (A  x), (B  x) derive (A  B) • The empty clause is derivable F is unsatisfiable • Proof size = # of clauses used

  8. Restricted forms of Resolution • Resolution • In general, graph of inferences is a directed acyclic graph • Tree Resolution • Graph of inferences forms a binary tree • i.e., if you want to use a clause more than once you need to re-derive it • Regular Resolution • Once a variable is resolved out of a clause you can’t add it back later • More general than Tree Resolution

  9. Satisfiability Algorithms • Incomplete Algorithms • Local search GSAT [Selman,Levesque,Mitchell 92] Walksat [Kautz,Selman 96] • Belief Propagation SP [Braunstein, Mezard, Zecchina 02] • Complete Algorithms • Backtracking search DPLL [Davis,Putnam 60][Davis,Logeman,Loveland 62] DPLL + clause learning GRASP, SATO, zchaff, berkmin • etc

  10. Proof Complexity and SAT Algorithms • Proof Complexity • Unsatisfiable formulas • Nondeterministic algorithms • SAT Algorithms • Satisfiable formulas • Deterministic or randomized algorithms

  11. Proof complexity and complete satisfiability algorithms • In general • The transcript of the run of a complete satisfiability algorithm on an unsatisfiable formula is a proof of its unsatisfiability • For backtracking style algorithms • Even runs on satisfiable formulas yield proofs of unsatisfiability of related formulas

  12. Backtracking search/DPLL DPLL(F) whileFcontains a 1-clause x F  F|x=1 if F is empty return 1 if F has an empty clause return 0 else selecta literal x to branch on return DPLL(F|x=1) DPLL(F|x=0) Residual formula

  13. Resolution and DPLL • DPLL tree with any variable selection rule on an unsatisfiable formula F generates a Resolution refutation of F • # of clauses  running time • Note: Regular resolution corresponds to a similar DAG (read-once branching program) rather than a tree but general resolution has no such analogue

  14. a a a b b b b b b c 3 d 3 c c b b d d 1 2 4 5 ab c d b ac a d DPLL Refutation Clauses 1.ab c 2.ab 3.c 4.a d 5.d b

  15. a:L b: a b: a c: ab 3 d:a b 3 b b 1 4 5 2 ab c d b ac a d DPLL Refutation= Tree-like Resolution Proof Clauses 1.ab c 2.ac 3.b 4.a d 5.a b

  16. Random k-CNF formulas • Randomly choose m clauses over n variables independently each of size k • Each size k clause is equally likely • Key parameter r=m/n, the ratio of clauses to variables: • Threshold value rk* of r • Below, almost certainly satisfiable • Above, almost certainly unsatisfiable • Hardest problems near threshold

  17. probability satisfiable 1 0 4.267 DPLL on random 3-CNF* Proof complexity shows 2Q(n/r)time is required for unsatisfiable formulas for r  r3* [B,Karp,Saks,Pitassi 98] [Ben-Sasson 02] # of DPLL backtracks What about satisfiable formulas below threshold? ratio of clauses to variables [Mitchell,Selman,Levesque 92] r * n = 50 variables

  18. Some simple select choices for DPLL algorithms • UC: Unit Clause/Ordered DLL • Choose variables in a fixed order • Always set True first • UCwm: Unit Clause with majority • Choose variables in a fixed order • Apply a majority vote among 3-clauses for assigning each value • GUC: Generalized Unit Clause • Choose a variable v in a shortest clause C • Set v to satisfy C

  19. Parameters of residual formulas follow trajectories 1 [Chao,Franco 88] [Frieze,Suen 95] [Achlioptas00] 2-clause ratio [Achlioptas,Sorkin 00] UC [Kaporis et al 03] GUC 2/3 8/3 3.52 0 3-clause ratio

  20. Satisfiability for mixed random formulas: proven properties [Achlioptas et al 96] 1 [Kaporis et al 03] ? ? [Dubois 01] ? ? ? ? UNSAT 2-clause ratio ? ? ? ? SAT ? ? 3.52 2/3 2.28 4.501 3-clause ratio

  21. Resolution proof complexity of mixed random formulas Theorem: [Achlioptas, B, Molloy 01]For any constants r21 and r3 0 random formulas with r2n2-clauses andr3n 3-clauses need exponential-size resolution proofs almost surely Extends [Chvatal-Szemeredi 88]

  22. Long-running DPLL Executions and Proof Complexity Residual formula at each node is a mix of 2- and 3-clauses Residual formula at is unsatisfiable 2rn Every resolution Algorithm’s proof of unsatisfiability is exponentially long

  23. Trajectory on 3-CNF UC Algorithm Trajectory 1 Provably UNSAT & Hard 2-clause ratio Provably SAT & Easy 0 4.51 3.81 3.52 4.267 3-clause ratio

  24. Exponential lower bounds for 3-CNF formulas below ratio 4.267 Corollary For almost all 3-CNF formulas, above ratio • 3.81 UC takes exponential time • 3.83 UCwm takes exponential time • 4.01 GUC takes exponential time

  25. Exponential lower bound below the proven k-CNF threshold • [Achlioptas, Peres 02]rk*  2k ln 2 – (k+4)/2 • [Achlioptas, B, Molloy 04] For k  4, for almost all k-CNF formulas UC takes exponential time above ratio c 2k/k Note These formulas have huge numbers of satisfying assignments (more than 2 (1-) n out of a possible 2n) but still are hard

  26. Open Problem • Closing gap for unsatisfiability of mixed formulas would yield sharp threshold behavior for each algorithm A • Below rAalgorithm runs in linear time • Above rA algorithm requires exponential time • Conjecture: no polynomial time selection heuristicwill work up to the threshold • rA  r3* for all algorithms A • Backtracking algorithms for other random problems with phase transitions? • e.g. k-colorability on random graphs G(n,r/n) • Unsatisfiable phase exp(cn/rak)[B, Culberson, Mitchell, Moore 03]

  27. Outline • Intro to proof complexity • Proof complexity and complete SAT algs • Review Resolution/DPLL connection • Application to random k-SAT • Proof Complexity and DPLL extensions • Clause learning • Caching • Proof system survey

  28. Clause Learning • At each backtrack point in DPLL search add new clauses to the input formula that correspond to known causes of failure of the search • Critical to the good behavior of the best current algorithms • What is the power of clause learning? • Intuitively: it makes DPLL trees DAG-like • regular resolution?  general resolution?

  29. x1 Conflict Graph y x2 false y x3 1-UIP scheme (t) p q a b t Decision scheme (p  q   b) Known Clauses (p  q  a) ( a   b   t) (t  x1) (t  x2) (t  x3) (x1 x2  x3  y) (x2 y) Current decisions p false q false b  true

  30. Proof complexity and clause learning • Clause learning is no stronger than Resolution • Implication graph yields resolution derivation of each learned clause • There are examples where clause learning works in polynomial time but any Regular Resolution proof is exponential [Sabharwal, B, Kautz 04] • Clause learning can sometimes be much stronger than Regular Resolution

  31. Proof complexity and clause learning • Separation is true for any proper subsystem R of Resolution for which refuting F|x is no harder than F • Clause learning does not have this property unless it is as powerful as resolution • Idea: Take formula F easy for Resolution but hard for R and create new formula F’ that is F plus clauses for (CpC) for each C in Resolution proof of F where each pC is a new propositional variable • Setting each pCto true yields original F but branching on sequence of pC allows sequence of clauses C to be learned

  32. Still Open • Can clause learning do everything regular resolution can?* • Is clause learning as powerful as general resolution? * Note: Alan Van Gelder has pointed out that if one returns learned clauses up the tree and resolves them while backtracking then clause learning does efficiently simulate regular resolution

  33. Proof Complexity and Formula Caching • Formula Caching: Memoization • Cache residual formulas that have been already found to be unsatisfiable • Do not make recursive call if the current residual formula is already cached

  34. DPLL(F) if F is empty report satisfiable and halt if F contains the empty clause L backtrack else select a literal x DPLL(F|x) DPLL(F|x) DPLL

  35. FC(F,L) if F is empty report satisfiable and halt if F contains the empty clause Lor FL return else choose a literal x FC(F|x,L) FC(F|x,L) add F to L Basic Formula Caching Start with FC(F,)

  36. Proof Complexity of Formula Caching [B, Impagliazzo, Pitassi, Segerlind 03] • Basic Formula Caching can exponentially improve DPLL • Basic Formula Caching is not as powerful as regular resolution • but some of its natural extensions are • These natural extensions can be exponentially more powerful than general resolution or even Res(k) for any k! • Res(k) is like resolution but uses k-DNFs instead of clauses

  37. Extensions of Formula Caching • Don’t just check membership in the cache • F is trivially unsatisfiable given G iff every clause of G contains some clause of F • Record the reasons why residual formulas are known to be unsatisfiable and check the reasons rather than the formulas

  38. FC*(F,L) if F is empty report satisfiable and halt if F is trivially unsatisfiable given L add F to L return else choose a literal x FC*(F|x,L) FC*(F|x,L) add F to L More General Formula Caching

  39. FCreason(F,L) if F is empty report satisfiable and halt if F is trivially unsatisfiable based on GL add F to L return(G) else choose a literal x GFCreason(F|x,L) HFCreason(F|x,L) J(xG) (xH) add F,J to L return(J) Adding Reasons

  40. Combining Clause Learning and Formula Caching • Cachet system used to count # of satisfying assignments [Sang, Bacchus, B., Kautz, Pitassi 04] • Built on zchaff’s clause learning • Uses decomposition of residual formula into components • Caches components rather than the whole formula • Subtle interactions

  41. Outline • Intro to proof complexity • Proof complexity and complete SAT algs • Resolution/DPLL review • Application to random k-SAT • DPLL extensions • Clause learning • Caching • Proof system survey

  42. Proof Systems Hierarchy ZFC Ext-Frege Frege LS AC0-Frege Cutting Planes PCR Res(k) Polynomial Calculus Resolution DPLL Nullstellensatz Truth Tables

  43. AC0-Frege • Constant-depth Frege (AC0-Frege) • Like Frege but formulas in proof only have constant # of alternations between  and  • Resolution is depth-1 Frege

  44. Some Known Hard Problems for Resolution • Exact/Modular Counting • Pigeonhole principle: n+1 pigeons can’t fit into n pigeonholes without getting cosy with each other • Parity Principle: e.g. can’t pair up an odd number of people • Tseitin Tautologies • These problems are also hard for AC0-Frege • Random Unsatisfiable • k-CNF • Graph k-colorability • Existence of k-cliques in graphs

  45. Cutting Planes/PseudoBoolean • Introduced to relate integer and linear programming [Gomory 59, Chvatal 73]: • Objects are linear integer inequalities • Clause(x1x2 x3)becomes inequality x1+(1-x2)+x31 • Addinequalitiesxi 0and 1-xi  0 • Goal: derive0  1

  46. a1x1 + ... + anxn A b1x1 + ... + bnxn B (a1+b1)x1+...+(an+bn)xn A+B a1x1 + ... + anxn A ca1x1 + ... + canxn cA ca1x+ ... + canxn B a1x1 + ... + anxnB/c Cutting Planes rules • addition: • multiplication by positive integer: • Division by positive integer:

  47. Limitations of Cutting Planes • At least as good as resolution • Can also handle • pigeonhole principle • parity principle • But can’t handle counting when it is a derived property • Clique-Coloring • A graph can’t have a k-clique and be k-1-colorable • Tseitin tautologies still open

  48. Proof Systems Hierarchy ZFC Ext-Frege Frege LS AC0-Frege Cutting Planes PCR Res(k) Polynomial Calculus Resolution DPLL Nullstellensatz Truth Tables

  49. Hilbert’s Nullstellensatz • System of polynomials Q1(x1,…,xn)=0,…,Qm(x1,…,xn)=0 over field Khas no solution in any extension field ofKthere exist polynomialsP1(x1,…,xn),…,Pm(x1,…,xn)inK[x1,…,xn] s.t.

  50. Nullstellensatz proof system C • Clause (x1x2 x3)becomes equation (1-x1)x2(1-x3)=0 • Add equations xi2-xi =0for each variable • Guarantees only 0-1 solutions • A proof is polynomials P1,…, Pm+nproving unsatisfiability: i.e. such that QC

More Related