SAT Problem Definition KR with SAT Tractable Subclasses DPLL Search Algorithm

1. SAT Problem Definition KR with SAT Tractable Subclasses DPLL Search Algorithm Slides by: Florent Madelaine Roberto Sebastiani Edmund Clarke Sharad Malik Toby Walsh Thomas Stützle Kostas Stergiou KNOWLEDGE REPRESENTATION & REASONING - SAT

2. Material of lectures on SAT • SAT definitions • Tractable subclasses • Horn-SAT • 2-SAT • CNF • Algorithms for SAT • DPLL-based • Basic chronological backtracking algorithm • Branching heuristics • Look-ahead (propagation) • Backjumping and learning • Local Search • GSAT • WalkSAT • Other enhancements • Application of SAT • Planning as satisfiability • Hardware verification KNOWLEDGE REPRESENTATION & REASONING - SAT

3. Local Search for SAT • Despite the success of modern DPLL-based solvers on real problems there are still very large (and hard) instances that are out of their reach • Some random SAT problems really are hard! • Chaff’s and the other complete solvers’ tricks don’t work so well here • Especially conflict-directed backjumping and learning • Local search methods are very successful on a number of such instances • they can quickly find models, if models exist • …but cannot prove insolubility if no models exists KNOWLEDGE REPRESENTATION & REASONING - SAT

4. k-SAT • Subclass of the SAT problem • Exactly k variables in each clause (P Q)  (Q  R)  (R P) is in 2-SAT • 3-SAT is NP-complete • SAT can be reduced to 3-SAT • 2-SAT is in P • Exists linear time algorithm KNOWLEDGE REPRESENTATION & REASONING - SAT

5. Experiments with 3-SAT • Where are the hard 3-SAT problems? • Sample randomly generated 3-SAT • Fix number of clauses, l • Number of variables, n • By definition, each clause has 3 variables • Generate all possible clauses with uniform probability KNOWLEDGE REPRESENTATION & REASONING - SAT

6. Experiments with random 3-SAT • Which are the hard instances? • around l/n = 4.3 What happens with larger problems? Why are some dots red and others blue? This is a so-called “phase transition” KNOWLEDGE REPRESENTATION & REASONING - SAT

7. Varying problem size, n Complexity peak appears to be largely invariant of algorithm complete algorithms like DPLL Incomplete methods like local search What’s so special about 4.3? Experiments with random 3-SAT KNOWLEDGE REPRESENTATION & REASONING - SAT

8. Experiments with random 3-SAT • Complexity peak coincides with satisfiability transition • l/n < 4.3 problems under-constrained and SAT • l/n > 4.3 problems over-constrained and UNSAT • l/n=4.3, problems on “knife-edge” between SAT and UNSAT KNOWLEDGE REPRESENTATION & REASONING - SAT

9. Phase Transitions • Similar “phase transitions” for other NP-hard problems • job shop scheduling • traveling salesperson • exam timetabling • constraint satisfaction problems • Was a “hot research topic” until recently: • predict hardness of a given instance & use hardness to control search strategy • but real problems with structure are much different than random ones… KNOWLEDGE REPRESENTATION & REASONING - SAT

10. Local Search Methods • Can handle “big” random SAT problems • Can go up about 10x bigger than systematic solvers!  • Rather smaller than structured problems (espec. if ratio  4.26) • Also handle big structured SAT problems • But lose here to best systematic solvers  • Try hard to find a good solution • Very useful for approximating MAX-SAT • the optimization version of SAT • Not intended to find all solutions • Not intended to show that there are no solutions (UNSAT) KNOWLEDGE REPRESENTATION & REASONING - SAT

11. Local vs. Complete Search on Hard Random Instances DPLL “Hard” means on the phase transition KNOWLEDGE REPRESENTATION & REASONING - SAT

12. Problem Formulation for Complete Search • INPUT: someinitialstateand a set ofgoalstates • BASIC ACTIONS: explore asuccessorof a visitedstate. • PATH COSTS: sum of thestepcosts • SOLUTION: apathfrom the initial state to one ofthe goal states through the state space. • OPTIMAL SOLUTION: a solution oflowestcost • QUESTION: Find a (optimal) solution. KNOWLEDGE REPRESENTATION & REASONING - SAT

13. Problem Formulation for Local Search • INPUT: someinitialstate • BASIC ACTIONS:movefrom the current state to asuccessor state. • EVALUATION FUNCTION: gives thescoreofastate • SOLUTION: state with the highest score. KNOWLEDGE REPRESENTATION & REASONING - SAT

14. Local Search Techniques • Typicallyignorelargeportionsofthesearchspaceand are based oncomplete initial formulation of the problem • deterministic: • (steepest) Hill Climbing (greedy local search) • Local beam search • Tabu search • stochastic: • Stochastic Hill Climbing • Simulatedannealing • Genetic algorithms • Ant colonyoptimisation • They have one important advantage: theyworkonlargeinstancesbecause they only keep track of few states. KNOWLEDGE REPRESENTATION & REASONING - SAT

15. Steepest Hill Climbing starts from initial state while there is a better successor do move to the best successor end while • This method has one important drawback: it may return a bad solutionbecause it can get stuck on a local optimumand is very dependent of the initial state • a local optimum is a state where there is no successor better than the current state, but there are other states that are better KNOWLEDGE REPRESENTATION & REASONING - SAT

16. Problems with Hill Climbing • Foothills / Local Optimal: No neighbor is better, but not at global optimum. • (Maze: may have to move AWAY from goal to find (best) solution) • Plateaus: All neighbors look the same. • (8-puzzle: perhaps no action will change # of tiles out of place) • Ridge: going up only in a narrow direction. • Suppose no change going South, or going East, but big win going SE • Ignorance of the peak: Am I done? KNOWLEDGE REPRESENTATION & REASONING - SAT

17. Improving the Hill Climbing Procedure • Using random restarts • when stuck start again from another initial state • By allowing sideways move • when stuck move to successor states worth the same as the current one • By allowing some limited memory • avoid states that will get you stuck (tabu search) • By enlarging the size of the neighbourhood • change successor function to generate more states • By allowing transition to worse states • when stuck allow moves to successor states worse than the current one KNOWLEDGE REPRESENTATION & REASONING - SAT

18. Local Search for SAT • Choose a random initial state I • i.e. a random assignment of 0 or 1 to all variables • If Iis not a model, repeatedly choose a variablepand change its value in I(flip the variable) • flip(I,p) determines the successor state The flipped variables are chosen using heuristics or randomly, orboth • the evaluation function is usually the number of satisfied clauses in a state • most algorithms use random restart if no solution has been found after a fixed number of search steps • Several instantiations of this general framework have been proposed KNOWLEDGE REPRESENTATION & REASONING - SAT

19. Definitions • Positive/Negative/Net Gain • Given candidate variable assignment T for CNF formula F, let B0 be the total # of clauses that are currently unsatisfied in F. Let T’ be the state of F if variable V is flipped. Let B1 be the total # of clauses which would be unsatisfied in T’. • The net gain of V is B0-B1. The negative gain of V is the # of clauses which are satisfied in T but unsatisfied in T’. The positive gain of V is the # of clauses which are unsatisfied in T but satisfied in T’. • Variable Age • The age of a variable is the # of flips since it was last flipped. KNOWLEDGE REPRESENTATION & REASONING - SAT

20. GSAT & GWSAT • The GSAT Algorithm (Selman, Mitchell, Levesque, 1992) • search initialisation: randomly chosen assignment • in each search step, flip the variable which givesthe maximal increase in the number ofsatisfied claused (maximum net gain) • ties are broken randomly • if no model found after maxStepssteps, restart from another randomly chosen assignment • HSAT (Gent and Walsh, 1993) – Same as GSAT, but break ties in favor of maximum age variable. • The GWSAT Algorithm (Selman, Kautz, Cohen, 1994) • search initialisation: randomly chosen assignment • Random Walk step: randomly choose a currently unsatisfiedclause and a literal in that clause, flip the corresponding variableto satisfy this clause • GWSAT steps: choose probabilistically between a GSAT stepand a Random Walk step with ‘walk probability’ (noise) π KNOWLEDGE REPRESENTATION & REASONING - SAT

21. WalkSAT • TheWalkSAT algorithm family • search initialisation: randomly chosen assignment • search step • randomly select a currently unsatisfied clause • select a literal from this clause according to a heuristic h • if no model found after maxStepssteps, restart from randomlychosen assignment • Original Walksat (Selman, Kautz, Cohen 1994) • Pick random unsatisfied clause BC. • If any variable in BC has negative gain of 0, randomly select one. • Otherwise, with probability p, select random variable from BC to flip, and with probability (1-p), select variable in BC with minimal negative gain (break ties randomly). KNOWLEDGE REPRESENTATION & REASONING - SAT

22. WalkSAT • Beating a correct, optimized implementation of Walksat is difficult. • Many variants of “walksat” appear in literature • Many of them are incorrect interpretations/ implementations of [Selman, Kautz, Cohen] Walksat. • Most of them perform significantly worse. • None of them performs better. • Positive gain: #clauses that were unsatisfied and now become satisfied • Negative gain: #clauses that were satisfied and now become unsatisfied KNOWLEDGE REPRESENTATION & REASONING - SAT

23. Variations of WalkSAT • SKC: Select variable such that minimal number of currentlysatisfied clauses become unsatisfied by flipping;if ‘zero-damage’ possible, always go for it; otherwise,with probability wp variable is randomly selected. • Tabu: Select variable that maximises increase in total numberof satisfied clauses when flipped; use constant length tabu-listfor flipped variables and random tie-breaking. • Tabu search prevents returning quickly to same state. • To implement: Keep fixed length queue (tabu list). • Add most recent step to queue; drop oldest step. • Never make step that's on current tabu list. • Example: • without tabu: • flip v1, v2, v4, v2, v10, v11, v1, v10, v3, ... • with tabu (length 4)—possible sequence: • flip v1, v2, v4, v10, v11, v1, v3, ... KNOWLEDGE REPRESENTATION & REASONING - SAT

24. SAT Local Search: Novelty Family • Novelty [McAllester, Selman, Kautz 1997] • Pick random unsatisfied clause BC. • Select variable v in BC with maximal net gain, unless v has the minimal age in BC. • In the latter case, select v with probability (1-p); otherwise, flip v2 with 2nd highest net gain. • Novelty+ [Hoos and Stutzle 2000] – Same as Novelty, but after BC is selected, with probability pw, select random variable in BC; otherwise continue with Novelty • R-Novelty [McAllester, Selman, Kautz 1997] and R-Novelty+ [Hoos and Stutzle 2000]– similar to Novelty/Novelty+, but more complex. KNOWLEDGE REPRESENTATION & REASONING - SAT

25. (the “novelty” part) Adaptive Novelty+ (current winner) • with probability 1% (the “+” part) • Choose randomly among all variables that appear in at least one unsatisfied clause • else be greedier (other 99%) • Randomly choose an unsatisfied clause C • Flipping any variable in C will at least fix C • Choose the most-improving variable in C … except … • with probability p, ignore most recently flipped variable in C • The “adaptive” part: • If we improved # of satisfied clauses, decrease p slightly • If we haven’t got an improvement for a while, increase p • If we’ve been searching too long, restart the whole algorithm KNOWLEDGE REPRESENTATION & REASONING - SAT

26. Structure of the standard SAT locals search variable selection heuristics • Score variables with respect to gain metric • Walksat uses negative gain, • GSAT and Novelty use net gain. • Restricted set of candidate variables • GSAT – any var in formula; • Walksat/Novelty – pick variable from a single random unsatisfied clause • Ranking of variables with respect to scoring metric • Greedy choices considered by all heuristics. Novelty variants also consider 2nd best variable. • Variable Age – prevent cycles and force exploration • Novelty variants, HSAT, Walksat+tabu KNOWLEDGE REPRESENTATION & REASONING - SAT

27. Some observations on the variable selection heuristics • Difficult to determine a priori how effective any given heuristic is. • Many heuristics look similar to Walksat, but performance varies significantly (c.f. [McAllester, Selman, Kautz 1997] • Empirical evaluation necessary to evaluate complex heuristics. Previous efforts to discover new heuristics involved significant experimentation. • Humans are skilled at identifying primitives, but find it difficult/time-consuming to combine the primitives into effective composite heuristics. KNOWLEDGE REPRESENTATION & REASONING - SAT

28. Simulated Annealing (popular general local search technique – often very effective, rather slow) • Simulated Annealing is an SLS method that tries to avoid localoptima by accepting probabilistically moves to worse solutions. • Simulated Annealing was one of the first SLS methods • now a "mature" SLS method • many applications available (ca. 1,000 papers) • (strong) convergence results • simple to implement • inspired by an analogy to physical annealing KNOWLEDGE REPRESENTATION & REASONING - SAT

29. Simulated Annealing description • Given a solution s generate some neighboring solution s’ N(s) • by picking a variable at random and flipping it • if f(s’)  f(s) then accept s’ • if f(s’) > f(s) then a probabilistic yes/no decision is made • if outcome is yes, then s’ replaces s • if outcome is no, then s is kept • probabilisitic decision depends on • the difference f(s’) - f(s) • a control parameter T, called temperature KNOWLEDGE REPRESENTATION & REASONING - SAT

30. Simulated Annealing description • Pick a variable at random If flipping it improves assignment: do it. Else flip anyway with probability p = e-/T where •  = f(s’) - f(s) (i.e. damage to score) • What is p for =0? For large ? • T = “temperature” • What is p as T tends to infinity? • Higher T = more random, non-greedy exploration • As we run, decrease T from high temperature to near 0. KNOWLEDGE REPRESENTATION & REASONING - SAT

31. Simulated Annealing Algorithm KNOWLEDGE REPRESENTATION & REASONING - SAT

32. Simulated Annealing general issues • generation of neighboring solutions • often: generate a random neighboring solution s’ N(s) • possibly better: systematically generate neighboringsolutions => at least we are sure to sample the whole neighbourhood if no move is accepted • acceptance criterion • often used:Metropolis acceptance criterion • If f(s’)  f(s) then accept s’ • if f(s’) > f(s) then accept it with a probability of e-/T , where  = f(s’) - f(s) KNOWLEDGE REPRESENTATION & REASONING - SAT

33. Simulated Annealing cooling schedule • open questions • how to define the control parameter? • how to define the (inner and outer) loop criteria? • cooling schedule • initial temperature T0 (Example: base it on some statistics about cost values, acceptance ratios etc.) • temperature function—how to change the temperature (Example: geometric cooling, Tn+1 = aTn, n=0,1,… 0<a<1) • number of steps at each temperature (inner loopcriterion) (Example: multiple of the neighbourhood size) • termination criterion (outer loop criterion) (Example: no improvement of sbest for a number of temperature values and acceptancerate below some critical value) KNOWLEDGE REPRESENTATION & REASONING - SAT

34. Simulated Annealing theoretical results • if SA can be run long enough • with an infinite number of temperature values • andfor each temperature value with an infinite number ofsteps one can be sure to be at an optimal solution at the end • however, it is not clear what end means • in addition, when run at a single temperature level longenough • we can be sure to find the optimum solution • hence, an optimal solution can be found withoutannealing • one only needs to store the best solution found andreturn it at the end • BUT: we need n ->  to guarantee optimality KNOWLEDGE REPRESENTATION & REASONING - SAT

35. Simulated Annealing theoretical results • From theoretical proofs we can also conclude that • better solutions are becoming more likely • this gives evidence that after • a sufficient number of temperature values • a sufficient number of steps at each temperature chances are high to have seen a good solution • however, it is unclear what sufficient means KNOWLEDGE REPRESENTATION & REASONING - SAT

36. Evolutionary algorithms (another popular general technique) • Many local searches at once • Consider 20 random initial assignments • Each one gets to try flipping 3 different variables(“reproduction with mutation”) • Now we have 60 assignments • Keep the best 20 (“natural selection”), and continue KNOWLEDGE REPRESENTATION & REASONING - SAT

37. 1 0 1 0 1 1 1 Parent 1 1 1 0 0 0 1 1 Parent 2 1 0 10 0 1 1 Child 1 Mutation Child 2 1 1 00 1 10 Reproduction (another popular general technique – at least for evolutionary algorithms) Derive each new assignment by somehow combining two old assignments, not just modifying one (“reproduction” or “crossover”) Good idea? KNOWLEDGE REPRESENTATION & REASONING - SAT

38. Dynamic Local Search • Dynamic local search is a collective term for a number ofapproaches that try to escape local optima by iterativelymodifying the evaluation function value of solutions. • different concept for escaping local optima • several variants available • promising results KNOWLEDGE REPRESENTATION & REASONING - SAT

39. Dynamic Local Search • guide the local search by a dynamic evaluation function • evaluation function h(s) composed of • cost function f(s) • penalty function • penalty function is adapted at computation time to guide thelocal search • penalties are associated to solution features • related approaches: • long term strategies in tabu search • noising method • usage of time-varying penalty functions for (strongly)constrained problems • etc. KNOWLEDGE REPRESENTATION & REASONING - SAT

40. Dynamic Local Search issues • timing of penalty modifications • at every local search step • only when trapped in a local optimum • long term strategies for weight decay • strength of penalty modifications • additive vs. multiplicative penalty modifications • amount of penalty modifications • focus of penalty modifications • choice of solution attributes to be punished KNOWLEDGE REPRESENTATION & REASONING - SAT

41. Example: Guided Local Search (GLS) • guided local search (GLS) • modifies penalties when trapped in local optima • variants exist that use occasional penalty decay • uses additive penalty modifications • chooses few among the many solution components forpunishment KNOWLEDGE REPRESENTATION & REASONING - SAT

42. GLS - details KNOWLEDGE REPRESENTATION & REASONING - SAT

43. GLS - details KNOWLEDGE REPRESENTATION & REASONING - SAT

44. GLS for SAT • uses the GSAT architecture for local search • uses in addition a special tie-breaking criterion that favorsflipping a variable that was flipped the longest time ago • if inxconsecutive iterations no improved solution is found,then modify penalties • solution attributes are clauses • when trapped in a local optimum, add penalties to clauses ofmaximum utility • which clauses are these? KNOWLEDGE REPRESENTATION & REASONING - SAT

45. GRASP • Greedy Randomized Adaptive Search Procedures (GRASP) is anSLS method that tries to construct a large variety of good initialsolutions for a local search algorithm. • predecessors: semi-greedy heuristics • tries to combine the advantages of random and greedysolution construction KNOWLEDGE REPRESENTATION & REASONING - SAT

46. Greedy construction heuristics • Iteratively construct solutions by choosing at eachconstruction step one solution component • solution components are rated according to a greedyfunction • the best ranked solutions component is added to thecurrent partial solution • examples: Kruskal’s algorithms for minimum spanningtrees, greedy heuristic for the TSP, • advantage: generate good quality solutions; local searchruns fast and finds typically better solutions than fromrandom initial solutions • disadvantage: do not generate many different solutions;difficulty of iterating KNOWLEDGE REPRESENTATION & REASONING - SAT

47. Random vs. greedy construction • random construction • high solution quality variance • low solution quality • greedy construction • good quality • low (no) variance • goal: exploit advantages of both KNOWLEDGE REPRESENTATION & REASONING - SAT

48. Semi-greedy heuristics • add at each step not necessarily the highest rated solution component • repeat until a full solution is constructed: • rate solution components according to a greedy function • put high rated solution components into a restrictedcandidate list (RCL) • choose one element of the RCL randomly and add it tothe partial solution • adaptive element: greedy function depends on the partialsolution constructed so far KNOWLEDGE REPRESENTATION & REASONING - SAT

49. Generation of the RCL KNOWLEDGE REPRESENTATION & REASONING - SAT

50. GRASP • GRASP tries to capture advantages of random and greedysolution construction • iterate through • randomized solution construction exploiting a greedyprobabilistic bias to construct feasible solutions • apply local search to improve over the constructedsolution • keep track of the best solution found so far and return it atthe end KNOWLEDGE REPRESENTATION & REASONING - SAT