Heuristics for Efficient SAT Solving

Heuristics forEfficient SAT Solving As implemented in GRASP, Chaff and GSAT.

Why SAT? • Fundamental problem from theoretical point of view • Cook theorem, 1971: the first NP-complete problem. • Numerous applications: • Solving any NP problem... • Verification: Model Checking, theorem-proving, ... • AI: Planning, automated deduction, ... • Design and analysis: CAD, VLSI • Physics: statistical mechanics (models for spin-glass material)

SAT made some progress…

The SAT competitions

Agenda • Modeling problems in Propositional Logic • SAT basics • Decision heuristics • Non-chronological Backtracking • Learning with Conflict Clauses • SAT and resolution • More techniques: decision heuristics, deduction. • Stochastic SAT solvers: the GSAT approach

CNF-SAT • Conjunctive Normal Form: Conjunction of disjunction of literals. Example:(:x1Ç:x2) Æ (x2Ç x4Ç: x1) Æ ... • Experience shows that CNF-SAT solving is faster than solving a general propositional formula. • There exists a polynomial transformation due to Tseitin (1970) of a general propositional formula  to CNF, with addition of || variables.

(CNF) SAT basic definitions: literals • A literal is a variable or its negation. • Var(l) is the variable associated with a literal l. • A literal is called negative if it is a negated variable, and positive otherwise.

SAT basic definitions: literals • If var(l) is unassigned, then l is unresolved. • Otherwise, l is satisfied by an assignment  if (var(l)) = 1 and l is positive, or (var(l)) = 0 and l is negative, and unsatisfied otherwise.

SAT basic definitions: clauses • The state of an n-long clause C under a partial assignment  is: • Satisfied if at least one of C’s literals is satisfied, • Conflicting if all of C’s literals are unsatisfied, • Unit if n-1 literals in C are unsatisfied and 1 literal is unresolved, and • Unresolved otherwise.

SAT basic definitions: clauses • Example

SAT basic definitions: the unit clause rule • The unit clause rule: in a unit clause the unresolved literal must be satisfied.

Basic Backtracking Search • Organize the search in the form of a decision tree • Each node corresponds to a decision • Depth of the node in the decision tree is called the decision level • Notation:x=v@dxis assigned v2 {0,1} at decision level d

x1 x1 = 0@1 x2 x2 = 0@2 Backtracking Search in Action 1 = (x2  x3) 2 = (x1  x4) 3 = (x2  x4) x1 = 1@1  x4 = 0@1  x2 = 0@1  x3 = 1@1  x3 = 1@2 {(x1,1), (x2,0), (x3,1) , (x4,0)} {(x1,0), (x2,0), (x3,1)} No backtrack in this example, regardless of the decision!

x1 x1 = 0@1 x1 = 1@1 x2 x2 = 0@2  x3 = 1@2 {(x1,0), (x2,0), (x3,1)} Backtracking Search in Action Add a clause 1 = (x2  x3) 2 = (x1  x4) 3 = (x2  x4) 4 = (x1  x2  x3)  x4 = 0@1  x2 = 0@1  x3 = 1@1 conflict

A Basic SAT algorithm(DPLL-based) Choose the next variable and value. Return False if all variables are assigned While (true) { if (!Decide()) return (SAT); while (!Deduce()) if (!Resolve_Conflict()) return (UNSAT); } Apply unit clause rule. Return False if reached a conflict Backtrack until no conflict. Return False if impossible

Decision heuristicsDLIS (Dynamic Largest Individual Sum) • Maintain a counter for each literal: in how many unresolved clauses it appears ? • Decide on the literal with the largest counter. • Requires O(#literals) queries for each decision.

Decision heuristicsJeroslow-Wang method Compute for every clause w and every literal l: • J(l) := • Choose a variable l that maximizes J(l). • This gives an exponentially higher weight to literals in shorter clauses.

Decision heuristicsMOM (Maximum Occurrence of clauses of Minimum size). • Let f*(x) be the # of unresolved smallest clauses containing x. Choose x that maximizes: ((f*(x) + f*(!x)) * 2k + f*(x) * f*(!x) • k is chosen heuristically. • The idea: • Give preference to satisfying small clauses. • Among those, give preference to balanced variables (e.g. f*(x) = 3, f*(!x) = 3 is better than f*(x) = 1, f*(!x) = 5).

Pause... • We will see other (more advanced) decision Heuristics soon. • These heuristics are integrated with a mechanism called Learning with Conflict-Clauses, which we will learns next.

4 x2=1@6 x5=1@6 1 4 3 6 x4=1@6  conflict 6 3 2 5 2 5 x6=1@6 x3=1@6 Implication graphs and learning Current truth assignment: {x9=0@1 ,x10=0@3, x11=0@3, x12=1@2, x13=1@2} Current decision assignment: {x1=1@6} x10=0@3 1 = (x1  x2) 2 = (x1  x3  x9) 3 = (x2  x3  x4) 4 = (x4  x5  x10) 5 = (x4  x6  x11) 6 = (x5   x6) 7 = (x1  x7  x12) 8 = (x1 x8) 9 = (x7  x8   x13) x1=1@6 x9=0@1 x11=0@3 We learn the conflict clause10 : (: x1Ç x9Ç x11Ç x10) and backtrack to the highest (deepest) dec. level in this clause (6).

x13=1@2 x8=1@6 9 8 9 ’ 9 7 x7=1@6 7 x12=1@2 Implication graph, flipped assignment 1 = (x1  x2) 2 = (x1  x3  x9) 3 = (x2  x3  x4) 4 = (x4  x5  x10) 5 = (x4  x6  x11) 6 = (x5  x6) 7 = (x1  x7  x12) 8 = (x1 x8) 9 = (x7  x8   x13) 10 : (: x1Ç x9Ç x11Ç x10) x9=0@1 10 x10=0@3 x1=0@6 10 10 x11=0@3 Due to theconflict clause We learn the conflict clause11 : (:x13Ç x9Ç x10Ç x11 Ç :x12) and backtrack to the highest (deepest) dec. level in this clause (3).

Non-chronological backtracking Which assignments caused the conflicts ? x9= 0@1 x10= 0@3 x11= 0@3 x12= 1@2 x13= 1@2 Backtrack to decision level 3 3 Decision level 4 5 These assignments Are sufficient for Causing a conflict. x1 6  ’ Non-chronological backtracking

Non-chronological backtracking (option #1) • So the rule is: backtrack to the largest decision level in the conflict clause. • Q: What if the flipped assignment works ? A: continue to the next decision level, leaving the current one without a decision variable. • Backtracking back to this level will lead to another conflict and further backtracking.

Non-chronological Backtracking x1 = 0 x2 = 0 x3 = 1 x3 = 0 x4 = 0 x6 = 0 ... x5 = 0 x5 = 1 x7 = 1 x9 = 1 x9 = 0

4 x2=1@6 x5=1@6 1 4 2 3 6 1 x4=1@6  conflict 6 3 2 5 2 5 x6=1@6 x3=1@6 3 More Conflict Clauses • Def: A Conflict Clause is any clause implied by the formula • Let L be a set of literals labeling nodes that form a cut in the implication graph, separating the conflict node from the roots. • Claim: Çl2L:l is a Conflict Clause. 1. (x10Ç:x1Ç x9Ç x11) x10=0@3 2. (x10Ç:x4Ç x11) 3. (x10Ç:x2Ç:x3Ç x11) x1=1@6   x9=0@1 x11=0@3 Skip alternative learning

Conflict clauses • How many clauses should we add ? • If not all, then which ones ? • Shorter ones ? • Check their influence on the backtracking level ? • The most “influential” ? • The answer requires two definitions: • Asserting clauses • Unique Implication points (UIP’s)

Asserting clauses • Def: An Asserting Clause is a Conflict Clause with a single literal from the current decision level. Backtracking (to the right level) makes it a Unit clause. • Modern solvers only consider Asserting Clauses.

6  conflict 6 Unique Implication Points (UIP’s) • Def: A Unique Implication Point (UIP) is an internal node in the Implication Graph that all paths from the decision to the conflict node go through it. • The First-UIP is the closest UIP to the conflict. • The method of choice: an asserting clause that includes the first UIP. In this case (x10Ç:x4Çx11). x10=0@3 4 1 4 UIP 3 UIP x4=1@6 3 2 5 2 5 x11=0@3

Previous method: backtrack to highest decision level in conflict clause (and erase it). A better method (empirically): backtrack to the second highest decision level in the clause, without erasing it. The asserted literal is implied at that level. In our example: (x10Ç:x4Çx11) Previously we backtracked to decision level 6. Now we backtrack to decision level 3. x4 = 0@3 is implied. Conflict-driven backtracking (option #2) 3 6 3

Conflict-driven Backtracking x1 = 0 x2 = 0 x5 = 1 x3 = 1 x7 = 1 x9 = 1 x4 = 0 x3 = 1 x6 = 0 ... x5 = 0 x9 = 0

Conflict-Driven Backtracking • So the rule is: backtrack to the second highest decision level dl, but do not erase it. • If the conflict clause has a single literal, backtrack to decision level 0. • Q: It seems to waste work, since it erases assignments in decision levels higher than dl, unrelated to the conflict. • A: indeed. But allows the SAT solver to redirect itself with the new information.

Progress of a SAT solver work invested in refutingx=1 (some of it seems wasted) C x=1 Refutation of x=1 C5 C2 Decision Level C1 C4 BCP C3 Decision Time Conflict

Conflict clauses and Resolution The Binary-resolution is a sound inference rule: Example:

Conflict clauses and resolution • Consider the following example: • Conflict clause: c5: (x2 Ç:x4Çx10)

Conflict clauses and resolution • Conflict clause: c5: (x2 Ç:x4Çx10) • Resolution order: x4,x5,x6,x7 • T1 = Res(c4,c3,x7) = (:x5Ç:x6) • T2 = Res(T1, c2, x6) = (:x4Ç:x5 Ç X10 ) • T3 = Res(T2,c1,x5) = (x2Ç:x4Ç x10 )

Finding the conflict clause: cl is asserting the first UIP Applied to our example:

4 1 4 3 3 2 5 6 2 5  conflict 6 9 10 8 9 ’ conflict 10 9 10 7 7 The Resolution-Graph keeps track of the “inference relation” Resolution Graph 1 2 3 10 4 11 5 7 6 8 9

The resolution graph What is it good for ? Example: for computing an Unsatisfiable core [Picture Borrowed from Zhang, Malik SAT’03]

Empty clause Inferred clauses Original clauses Resolution graph: example learning L: Unsatisfiable core

Decision heuristicsVSIDS (Variable State Independent Decaying Sum) 1. Each variable in each polarity has a counter initialized to 0. 2. When a clause is added, the counters are updated. 3. The unassigned variable with the highest counter is chosen. 4. Periodically, all the counters are divided by a constant. (Implemented in Chaff)

Decision heuristicsVSIDS (cont’d) • Chaff holds a list of unassigned variables sorted by the counter value. • Updates are needed only when adding conflict clauses. • Thus - decision is made in constant time.

Decision heuristicsVSIDS (cont’d) VSIDS is a ‘quasi-static’ strategy: - static because it doesn’t depend on current assignment - dynamic because it gradually changes. Variables that appear in recent conflicts have higher priority. This strategy is a conflict-driven decision strategy. “..employing this strategy dramatically (i.e. an order of magnitude) improved performance ... “

Decision Heuristics - Berkmin • Keep conflict clauses in a stack • Choose the first unresolved clause in the stack • If there is no such clause, use VSIDS • Choose from this clause a variable + value according to some scoring (e.g. VSIDS) • This gives absolute priority to conflicts.

Berkmin heuristic tail-first conflict clause

More engineering aspects of SAT solvers Observation: More than 90% of the time SAT solvers perform Deduction(). Deduction() allocates new implied variables and conflicts. How can this be done efficiently ?

Heuristics for Efficient SAT Solving