Presented by Yunho Kim Provable Software Lab, KAIST

1. Heavy-Tailed Phenomena in Satisfiability and Constraint Satisfaction Problemsby Carla P. Gomes, Bart Selman, NunoCrato and henryKautz Presented by Yunho Kim Provable Software Lab, KAIST

2. Introduction Search procedures and problem domains Cost distributions of backtrack search Consequences for Algorithm Design Conclusion Contents Heavy-Tailed Phenomena in Satisfiability and Constraint Satisfaction Problems Yunho Kim, Provable Software Lab, KAIST

3. The DPLL algorithm is a complete algorithm for deciding the satisfiability of propositional logic formulas • It is guaranteed that eventually either the DPLL algorithm finds a satisfying model or proves the formula is unsatisfiable Introduction(1/4) The iterative version of DPLL algorithm 1 status = preprocess(); 2 if (status!=UNKNOWN) return status; 3 while(1){ 4 decide_next_branch(); 5 while(1){ 6 status = deduce(); 7 if (status == CONFLICT){ 8 blevel = analyze_conflict(); 9 if (blevel == 0) 10 return UNSAT; 11 else backtrack(blevel); 12 } 13 else if (status == SAT) 14 return SAT; 15 else break; 16 } 17 } Heavy-Tailed Phenomena in Satisfiability and Constraint Satisfaction Problems Yunho Kim, Provable Software Lab, KAIST

4. At each step a heuristic is used to select the next branch variable • A branch heuristic scores each variable in some manner and select the highest one • Randomization can be used for tie-breaking • If several choices are ranked equally, choose among them at random • All variables that receive scores within H-percent of the highest score are considered equally good • H is a heuristic equivalence parameter • Empirically it is known that randomized branching heuristics is effective on hard instances Introduction(2/4) Heavy-Tailed Phenomena in Satisfiability and Constraint Satisfaction Problems Yunho Kim, Provable Software Lab, KAIST

5. Problem instance: quasigroup completion problem (N = 11, 30% pre-assignments) Introduction(3/4) The sample mean of the number of backtracks does diverge Sample mean (number of backtracks) Number of runs Heavy-Tailed Phenomena in Satisfiability and Constraint Satisfaction Problems Yunho Kim, Provable Software Lab, KAIST

6. The authors have observed the erratic behavior of the mean and the variance of the search cost distributions on a same instance • The mean cost calculated over an increasing number of runs, on the same satisfiable problem instance, of a randomized backtrack search procedure does diverge. • The authors have not found unsatisfiable instances with heavy-tailed behavior Introduction(4/4) Heavy-Tailed Phenomena in Satisfiability and Constraint Satisfaction Problems Yunho Kim, Provable Software Lab, KAIST

7. Introduction Search procedures and problem domains Cost distributions of backtrack search Consequences for Algorithm Design Conclusion Contents Heavy-Tailed Phenomena in Satisfiability and Constraint Satisfaction Problems Yunho Kim, Provable Software Lab, KAIST

8. The authors modified two state-of-the-art SAT solvers(at that time), Satz and Relsat • Both solvers hire similar occurrences-based decision heuristics • Satz employs chronological backtracking while Relsat uses non-chronological backtracking Search Procedures Heavy-Tailed Phenomena in Satisfiability and Constraint Satisfaction Problems Yunho Kim, Provable Software Lab, KAIST

9. A quasigroupis an ordered pair(Q, ¢), where Q is a set and (¢) is a binary operation on Q such that the equations a¢x = b and y¢a = b are uniquely solvable for every pair of elements a, b in Q • The order N of the quasigroup is the cardinality of the set Q • The multiplication table of a finite quasigroup is a Latin square • An N£N table filled with n different symbols in such a way that each symbol occurs exactly once in each row and exactly once in each column Quasigroup Completion Problem(1/2) Order 4 quasigroup Order 10 quasigroup Heavy-Tailed Phenomena in Satisfiability and Constraint Satisfaction Problems Yunho Kim, Provable Software Lab, KAIST

10. The quasigroup completion problem • determining whether the remaining entries of the partial Latin square can be filled in such a way that we obtain a complete Latin square • The quasigroup completion problem is NP-complete Quasigroup Completion Problem(2/2) 32% pre-assigned Heavy-Tailed Phenomena in Satisfiability and Constraint Satisfaction Problems Yunho Kim, Provable Software Lab, KAIST

11. Timetabling, planning and instances in the Dimacs Challenge benchmark are also considered Timetabling problem is to determine whether there exists a feasible schedule that consider a set of pairing and distribution constraints Planning is to find a sequence of actions that transform an initial state to a goal state Other Problems Heavy-Tailed Phenomena in Satisfiability and Constraint Satisfaction Problems Yunho Kim, Provable Software Lab, KAIST

12. Introduction Search procedures and problem domains Cost distributions of backtrack search Consequences for Algorithm Design Conclusion Contents Heavy-Tailed Phenomena in Satisfiability and Constraint Satisfaction Problems Yunho Kim, Provable Software Lab, KAIST

13. Data was produced by running the randomized backtrack search procedure 10,000 times on the same instance Even though 50% of the runs solve the instance in 1 backtrack or less, after 100,000 backtracks 0.5%of the runs were still not completed Cumulative Distribution(1/2) Completion of quasigroup Cumulative fraction of successful runs Number of backtracks Heavy-Tailed Phenomena in Satisfiability and Constraint Satisfaction Problems Yunho Kim, Provable Software Lab, KAIST

14. A solution is found in 1,000 backtracks or less in 80% of runs However, 5% of the runs do not result in a solution even after 1,000,000 backtracks Cumulative Distribution(2/2) Timetabling Cumulative fraction of successful runs Number of backtracks Heavy-Tailed Phenomena in Satisfiability and Constraint Satisfaction Problems Yunho Kim, Provable Software Lab, KAIST

15. Power Law Decay Exponential Decay Standard Distribution (finite mean & variance) Heavy-Tailed Distributions(1/3) Standard Exponential decay e.g. Normal: P{X>x} ~ Ce-x2 for some C > 0 Heavy-Tailed Power law decay e.g. Pareto-Levy: P{X>x} ~ Cx-® where for some 0 < ® < 2 and C > 0 Heavy-Tailed Phenomena in Satisfiability and Constraint Satisfaction Problems Yunho Kim, Provable Software Lab, KAIST

16. We consider distributions that asymptotically have “heavy tails”, namely, where for some 0 < ® < 2 and C > 0 • The ® is referred to as the index of stability of the distribution • The lower the index, the heavier the tail • Heavy-tailed distributions have finite/infinite mean and infinite variance Heavy-Tailed Distributions(2/3) P{X>x} ~ Cx-® Heavy-Tailed Phenomena in Satisfiability and Constraint Satisfaction Problems Yunho Kim, Provable Software Lab, KAIST

17. Comparison of tail probabilities P{X > c} • Cauchy distribution is the heavy-tailed distribution which has ® = 1.0 • Levy distribution is the heavy-tailed-distribution which has ® = 0.5 Heavy-Tailed Distributions(3/3) Heavy-Tailed Phenomena in Satisfiability and Constraint Satisfaction Problems Yunho Kim, Provable Software Lab, KAIST

18. Log-log plot of the cost distribution of the satisfiable completion of quasigroups instances 1-F(x) = P{X > x} ~ Cx-® Visual Check(1/3) Completion of quasigroups Log(1-F(x)) ~ -®Log(x) + C’ Log(1-F(x)) N = 15, 40% pre-assignments N = 15, 30% pre-assignments N = 11, 30% pre-assignments Log number of backtracks Heavy-Tailed Phenomena in Satisfiability and Constraint Satisfaction Problems Yunho Kim, Provable Software Lab, KAIST

19. Log-log plot of the cost distribution of the satisfiable timetabling instance 1-F(x) = P{X > x} ~ Cx-® Visual Check(2/3) Completion of timetabling Log(1-F(x)) Log number of backtracks Heavy-Tailed Phenomena in Satisfiability and Constraint Satisfaction Problems Yunho Kim, Provable Software Lab, KAIST

20. Log-log plot of the cost distribution of the satisfiable logistics planning from two different SAT solvers 1-F(x) = P{X > x} ~ Cx-® Visual Check(3/3) Logistics planning Log(1-F(x)) Satz Relsat Log number of backtracks Heavy-Tailed Phenomena in Satisfiability and Constraint Satisfaction Problems Yunho Kim, Provable Software Lab, KAIST

21. Calculated maximum likelihood estimates of ® using Hill estimator • k is sample size • Since ®· 1, mean and variance are infinite Estimation of ® Heavy-Tailed Phenomena in Satisfiability and Constraint Satisfaction Problems Yunho Kim, Provable Software Lab, KAIST

22. Log-log plot of the cost distribution of unsatisfiable completion of quasigroupsinstnces 1-F(x) = P{X > x} ~ Cx-® Unsatisfiable Instance Completion of quasigroups Log(1-F(x)) Log number of backtracks Heavy-Tailed Phenomena in Satisfiability and Constraint Satisfaction Problems Yunho Kim, Provable Software Lab, KAIST

23. Introduction Search procedures and problem domains Cost distributions of backtrack search Consequences for Algorithm Design Conclusion Contents Heavy-Tailed Phenomena in Satisfiability and Constraint Satisfaction Problems Yunho Kim, Provable Software Lab, KAIST

24. Restart after a fixed number of backtracks prevent a solver from entering pitfall Restarts Effect of restarts on a quasigroup instance N = 20, 5% pre-assignments No restarts Log(1-F(x)) Without restarts and given a total of 300 backtracks, 70% of runs failed With restarts, only 0.01% of runs failed With restarts Total number of backtracks Heavy-Tailed Phenomena in Satisfiability and Constraint Satisfaction Problems Yunho Kim, Provable Software Lab, KAIST

25. Randomized rapid restarts(RRR) show better performance than deterministic Restarts Heavy-Tailed Phenomena in Satisfiability and Constraint Satisfaction Problems Yunho Kim, Provable Software Lab, KAIST

26. Introduction Search procedures and problem domains Cost distributions of backtrack search Consequences for Algorithm Design Conclusion Contents Heavy-Tailed Phenomena in Satisfiability and Constraint Satisfaction Problems Yunho Kim, Provable Software Lab, KAIST

27. The authors show the suitability of heavy-tailed distributions in modeling the runtime behavior of DPLL SAT solver with the random decision heuristic • Restarts can exploit the mass of probability on the left of the cost distributions Conclusion Heavy-Tailed Phenomena in Satisfiability and Constraint Satisfaction Problems Yunho Kim, Provable Software Lab, KAIST

28. Heavy-Tailed Phenomena in Satisfiability and Constraint Satisfaction Problems by Carla P. Gomes, Bart Selman, NunoCrato and Henry Kautz in Journal of Automated Reasoning 24: 67-100, 2000 Reference Heavy-Tailed Phenomena in Satisfiability and Constraint Satisfaction Problems Yunho Kim, Provable Software Lab, KAIST

29. Probability density function of Lévy distribution • Lévy have infinite mean and variance Lévy Distribution(1/2) Heavy-Tailed Phenomena in Satisfiability and Constraint Satisfaction Problems Yunho Kim, Provable Software Lab, KAIST

30. Cumulative distribution function of Lévy distribution Lévy Distribution(2/2) Heavy-Tailed Phenomena in Satisfiability and Constraint Satisfaction Problems Yunho Kim, Provable Software Lab, KAIST