1 / 34

Random 3-SAT

Random 3-SAT. Random 3-SAT sample uniformly from space of all possible 3-clauses n variables, l clauses Which are the hard instances? around l/n = 4.3. slide thanks to Henry Kautz. Random 3-SAT. Varying problem size, n Complexity peak appears to be largely invariant of algorithm

adlai
Download Presentation

Random 3-SAT

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Random 3-SAT • Random 3-SAT • sample uniformly from space of all possible 3-clauses • n variables, l clauses • Which are the hard instances? • around l/n = 4.3 600.325/425 Declarative Methods - J. Eisner slide thanks to Henry Kautz

  2. Random 3-SAT • Varying problem size, n • Complexity peak appears to be largely invariant of algorithm • backtracking algorithms like Davis-Putnam • local search procedures like GSAT • What’s so special about 4.3? 600.325/425 Declarative Methods - J. Eisner slide thanks to Henry Kautz

  3. Random 3-SAT • Complexity peak coincides with solubility transition • l/n < 4.3 problems under-constrained and SAT • l/n > 4.3 problems over-constrained and UNSAT • l/n=4.3, problems on “knife-edge” between SAT and UNSAT 600.325/425 Declarative Methods - J. Eisner slide thanks to Henry Kautz

  4. “Order parameter” for 3SAT [Mitchell, Selman, Levesque AAAI-92] • b = #clauses / # variables • This predicts • satisfiability • hardness of finding a model 600.325/425 Declarative Methods - J. Eisner slide thanks to Tuomas Sandholm

  5. 600.325/425 Declarative Methods - J. Eisner slide thanks to Tuomas Sandholm

  6. Generality of the order parameter b • The results seem quite general across model finding algorithms • Other constraint satisfaction problems have order parameters as well 600.325/425 Declarative Methods - J. Eisner slide thanks to Tuomas Sandholm

  7. …but the complexity peak does not occur under all ways of generating the 3SAT instances 600.325/425 Declarative Methods - J. Eisner slide thanks to Tuomas Sandholm

  8. Iterative refinement algorithms for SAT 600.325/425 Declarative Methods - J. Eisner slide thanks to Tuomas Sandholm

  9. Avg. total flips 50 variables, 215 3SAT clauses 2000 1600 1200 800 400 max-climbs 100 200 GSAT [Selman, Levesque, Mitchell AAAI-92](= a local search algorithm for model finding) Incomplete (unless restart a lot) Greediness is not essential as long as climbs and sideways moves are preferred over downward moves. 600.325/425 Declarative Methods - J. Eisner slide thanks to Tuomas Sandholm

  10. BREAKOUT algorithm [Morris AAAI-93] Initialize all variables Pi randomly UNTIL currently state is a solution IF current state is not a local minimum THEN make any local change that reduces the total cost (i.e. flip one Pi) ELSE increase weights of all unsatisfied clause by one Incomplete, but very efficient on large (easy) satisfiable problems. Reason for incompleteness: the cost increase of the current local optimum spills to other solutions because they share unsatisfied clauses. 600.325/425 Declarative Methods - J. Eisner slide thanks to Tuomas Sandholm

  11. Real-World Phase Transition Phenomena • Many NP-hard problem distributions show phase transitions - • job shop scheduling problems • TSP instances from TSPLib • exam timetables @ Edinburgh • Boolean circuit synthesis • Latin squares (alias sports scheduling) • Hot research topic: predicting hardness of a given instance, & using hardness to control search strategy (Horvitz, Kautz, Ruan 2001-3) 600.325/425 Declarative Methods - J. Eisner slide thanks to Henry Kautz

  12. Local search for SAT • Repair based methods • Instead of building up a solution, take complete assignment and flip variable to satisfy “more” clauses • Cannot prove UNSATisfiability • Often will solve MAX-SAT problem 600.325/425 Declarative Methods - J. Eisner

  13. Papadimitrou’s procedure • T:= random truth assignment • Repeat until T satisfies all clauses • v := variable in UNSAT clause • T := T with v’s value flipped Semi-decision procedure Solves 2-SAT in expected O(n^2) time 600.325/425 Declarative Methods - J. Eisner

  14. GSAT [Selman, Levesque, Mitchell AAAI 92] • Repeat MAX-TRIES times or until clauses satisfied • T:= random truth assignment • Repeat MAX-FLIPS times or until clauses satisfied • v := variable which flipping maximizes number of SAT clauses • T := T with v’s value flipped Adds restarts and greediness Sideways flips important (large plateaus to explore) 600.325/425 Declarative Methods - J. Eisner

  15. WalkSAT [Selman, Kautz, Cohen AAAI 94] • Repeat MAX-TRIES times or until clauses satisfied • T:= random truth assignment • Repeat MAX-FLIPS times or until clauses satisfied • c := unsat clause chosen at random • v:= var in c chosen either greedily or at random • T := T with v’s value flipped Focuses on UNSAT clauses 600.325/425 Declarative Methods - J. Eisner

  16. Novelty [McAllester, Selman, Kautz AAAI 97] • Repeat MAX-TRIES times or until clauses satisfied • T:= random truth assignment • Repeat MAX-FLIPS times or until clauses satisfied • c := unsat clause chosen at random • v:= var in c chosen greedily • If v was last var in clause flipped and rand(1)<p then v:= var in c with 2nd highest score • T := T with v’s value flipped Encourages diversity (like Tabu search) 600.325/425 Declarative Methods - J. Eisner

  17. Other local search methods • Simulated annealing • Tabu search • Genetic algorithms • Hybrid methods • Combine best features of systematic methods (eg unit propagation) and local search (eg quick recovery from failure) 600.325/425 Declarative Methods - J. Eisner

  18. Exotic methods • Quantum computing • DNA computing • Ant computing • … 600.325/425 Declarative Methods - J. Eisner

  19. Beyond the propositional • Linear 0/1 inequalities • Quantified Boolean satisfiability • Stochastic satisfiability • Modal satisfiability • … 600.325/425 Declarative Methods - J. Eisner

  20. Linear 0/1 inequalities • Constraints of the form: • Sum ai . Xi >= c • SAT can easily be expressed as linear 0/1 problem • X v -Y v Z => X + (1-Y) + Z >= 1 • … 600.325/425 Declarative Methods - J. Eisner

  21. Linear 0/1 inequalities • Often good for describing problems involving arithmetic • Counting constraints • Objective functions • Solution methods • Complete Davis-Putnam like methods • Local search methods like WalkSAT(PB) 600.325/425 Declarative Methods - J. Eisner

  22. Quantified Boolean formulae • Propositional SAT can be expressed as: • Exists P, Q, R . (P v Q) & (-Q v R) & … • Can add universal quantifiers: • Forall P . Exists Q, R . (P v Q) & (-Q v R) & .. • Useful in formal methods, conditional planning … 600.325/425 Declarative Methods - J. Eisner

  23. Quantified Boolean formulae • SAT of QBF is PSPACE complete • Can be seen as game • Existential quantifiers trying to make it SAT • Universal quantifiers trying to make it UNSAT • Solution methods • Extensions of the DPLL procedure • Local search methods just starting to appear 600.325/425 Declarative Methods - J. Eisner

  24. Stochastic satisfiability • Randomized quantifier in addition to the existential • With prob p, Q is True • With prob (1-p), Q is False • Can we satisfy formula in some fraction of possible worlds? • Useful for modelling uncertainty present in real world (eg planning under uncertainty) 600.325/425 Declarative Methods - J. Eisner

  25. Randomized Algorithms • A randomized algorithm is defined as an algorithm where at least one decision is based on a random choice. 600.325/425 Declarative Methods - J. Eisner slide thanks to Russ Greiner and Dekang Lin

  26. Monte Carlo and Las Vegas There are two kinds of randomized algorithms: • Las Vegas: A Las Vegas algorithm always produces the correct answer, but its runtime for each input is a random variable whose expectation is bounded. • Monte Carlo: A Monte Carlo algorithm runs for a fixed number of steps for each input and produces an answer that is correct with a bounded probability • One sided • Two sided 600.325/425 Declarative Methods - J. Eisner slide thanks to Russ Greiner and Dekang Lin

  27. Local Search Summary • Surprisingly efficient search technique • Wide range of applications • Formal properties elusive • Intuitive explanation: • Search spaces are too large for systematic search anyway. . . • Area will most likely continue to thrive 600.325/425 Declarative Methods - J. Eisner slide thanks to Russ Greiner and Dekang Lin

  28. Pure WalkSat • PureWalkSat( formula ) Guess initial assignment While unsatisfied do Select unsatisfied clause c = ±Xi v ±Xj v ±Xk Select variable v in unsatisfied clause c Flip v 600.325/425 Declarative Methods - J. Eisner slide thanks to Russ Greiner and Dekang Lin

  29. Example: 600.325/425 Declarative Methods - J. Eisner slide thanks to Russ Greiner and Dekang Lin

  30. Mixing Random Walk with Greedy Local Search • Usual issues: • Termination conditions • Multiple restarts • Value of p determined empirically, by finding best setting for problem class 600.325/425 Declarative Methods - J. Eisner slide thanks to Russ Greiner and Dekang Lin

  31. Finding the best value of p • WalkSat[p]: • W/prob p, flip var in unsatisfied clause • W/prob 1-p, make a greedy flip to minimize # of unsatisfied clauses • Q: What value for p? • Let: Q[p, c] be quality of using WalkSat[p] on problem c. • Q[p, c] = Time to return answer, or = 1 if WalkSat[p] return (correct) answer within 5 minutes and 0 otherwise, or = . . . perhaps some combination of both . . . • Then, find p that maximize the average performance of WalkSat[p] on a set of challenge problems. 600.325/425 Declarative Methods - J. Eisner slide thanks to Russ Greiner and Dekang Lin

  32. Experimental Results: Hard Random 3CNF • Time in seconds • Effectiveness: prob. that random initial assignment leads to a solution. • Complete methods, such as DP, up to 400 variables • Mixed Walk better than Simulated Annealing • better than Basic GSAT • better than Davis-Putnam 600.325/425 Declarative Methods - J. Eisner slide thanks to Russ Greiner and Dekang Lin

  33. Overcoming Local Optimum and Plateau • Random restarts • Simulated annealing • Mixed-in random walk • Tabu search (prevent repeated states) • Others (Genetic algorithms/programming, . . . ) 600.325/425 Declarative Methods - J. Eisner slide thanks to Russ Greiner and Dekang Lin

  34. Other Techniques • random restarts: restart at new random state after pre-defined # of local steps. • [Done by GSAT] • tabu: prevent returning quickly to same state. • Implement: Keep fixed length queue (tabu list). • Add most recent step to queue; drop oldest step. • Never make step that's on current tabu list. • Example: • without tabu: • flip v1, v2, v4, v2, v10, v11, v1, v10, v3, ... • with tabu (length 5)—possible sequence: • flip v1, v2, v4, v10, v11, v1, v3, ... • Tabu very powerful; competitive w/ simulated annealing or random walk (depending on the domain) 600.325/425 Declarative Methods - J. Eisner slide thanks to Russ Greiner and Dekang Lin

More Related