Unifying SAT-based and Graph-based Planning

1. Unifying SAT-based and Graph-based Planning • Henry Kautz • AT&T Labs • Bart Selman • Cornell University IJCAI-99

2. SATPLAN (Kautz & Selman 1996) instantiated propositional clauses instantiate axiom schemas problem description length mapping SAT engine(s) interpret satisfying model plan

3. SAT Algorithms • Systematic Search • DP (Davis Putnam Logemann Loveland)backtrack search + unit propagation • satz (Chu Min Li) - variable selection by forward checking: max unit props • relsat (Bayardo) - dependency directed backtracking: add new clauses at dead-ends • Local Search • Walksat(Selman, Kautz & Cohen)local search + noise to escape minima

4. Critically-constrained Logistics Planning Problems

5. Tradeoffs of SAT Approach • Advantages • Can trade space for time by avoiding variable binding during search • Domain modeling can substitute for algorithm development • New high powered SAT algorithms can take advantage of implicit structure of encoded problems • Disadvantages • Instantiated formulas huge, much redundancy • Good domain models can be hard to develop - automatic STRIPS translations disappointing • No way to explicitly leverage structure

6. SATPLAN & Graphplan: Disjunctive Planners • Graphplan (Blum & Furst 1995) Set new paradigm for planning • Like SATPLAN... • Two phases: instantiation of propositional structure, followed by search • Unlike SATPLAN... • Interleaves instantiation and pruning of plan graph • Employs specialized search engine • Neither approach best for all domains or all instances • Graphplan - better instantiation • SATPLAN - better search • IJCAI Challenge in Bridging Plan Synthesis Paradigms(Kambhampati 1997)

7. Blackbox Reachability Analysis Plan Graph STRIPS Translator CNF Simplifier General Stochastic / Systematic SAT engines Solution CNF

8. Staged Inference Polytime domain specific inference Abstract problem specification Domain specific model Polytime general inference General language encoding Encoding scheme Full general inference (NP complete) Combinatorial CORE Solution

9. Intuition • Many real-world problems not tractable, but are nearly so • polytime inference takes advance of special kinds of structure • structure may be visible at the level of a domain specific representation, or only after the problem is encoded • small number of practical methods for combinatorial core

10. Component 1: Reachability Analysis • Graphplan instantiates in a forward direction, pruning unreachable nodes • conflicting actions are mutex • if all actions that add two facts are mutex, the facts are mutex • if the preconditions for an action are mutex, the action is unreachable • Reachability analysis in unfolded Petri Nets (K. McMillian 1992)

11. The Plan Graph Facts Facts Actions Actions Facts Facts ... ... ... ... mutually exclusive preconditions add effects delete effects

12. Component 2: Translation Pre1 Act1 Fact Pre2 Act2 • Fact  Act1  Act2 • Act1  Pre1  Pre2 • ¬Act1  ¬Act2 Backward-chaining axioms force groundedness Prevents underconstrained variables from taking on arbitrary values

13. Mutex Algorithm as Resolution • Each mutex computation equivalent to a series of resolutions • one resolvant always negative binary clause • K actions add P (1 clause) • K actions add Q (1 clause) • all P adders mutex Q adders (K2 clauses) • Inferring (~P v ~Q) requires 4K2 resolutions

14. Improved Encodings • Translations of Logistics.a: • STRIPS  Axiom Schemas  SAT (Medic system, Weld et. al 1997) • 3,510 variables, 16,168 clauses • 24 hours to solve • STRIPS  Plan Graph  SAT • 2,709 variables, 27,522 clauses • 5 seconds to solve!

15. Component 3: Simplification • Generated wff can be further simplified by consistency propagation techniques • unit propagation: is Wff inconsistant by resolution against unit clauses? O(n) • failed literal rule: is Wff + { P } inconsistant by unit propagation? O(n2) • binary failed literal rule: is Wff + { P V Q } inconsistant by unit propagation? O(n3) • General limited inference complements domain specific limited inference (mutex) Reveals hidden local structure

16. General Limited Inference

17. Component 4: Improved Systematic SAT Solvers • Systematic search generally best for wffs derived from STRIPS operators • Wffs not as “flat” - long chains of unit propagations • Problem: • Solution time for backtrack search highly variable as problem instance varied • “easier” problems may take orders of magnitude longer to solve than “harder” ones!

18. Unpredictability of Systematic Search

19. Randomized Restarts • Heavy tailed distribution of running times • Solution: randomize the systematic solver • Add noise to the heuristic branching (variable choice) function • Cutoff and restart search after a fixed number of backtracks • In practice: rapid restarts with low cutoff can dramatically improve performance • (Gomes 1996, Gomes, Kautz, and Selman 1997, 1998)

20. Increased Predictability

21. Summary of Results

22. Observations • SAT engines can outperform direct search of plan graph • when problems critically constrained • bottleneck is extraction (not reachability) • when graphplan/IPP heuristics for non-optimal planning (e.g. RIFO) not applicable • Solution time using best randomized systematic SAT algorithm virtually identical for BlackBox and SATPLAN wffs • although SATPLAN wffs included much extra explicit domain knowledge - invariants, etc. • Scaling of BlackBox/satz-rand closely matches scaling of SATPLAN/walksat (~ 4x)

23. Applicability • When is the BlackBox approach not a good idea? • when domain too large for propositional planning approaches • when long sequential plans are needed • when solution time dominated by reachability analysis (plan-graph generation), not extraction • when optimal or near optimal planning not necessary

24. Efficiency of Translation Approach • Translation usually not a bottleneck • wff grows linearly in size of plan graph • modified translation reduces explicit mutex clauses by 75% • new compact representations of plan graph will challenge this approach! (Koehler, Fox & Long, Smith & Weld...) • Loss of cached information acceptable on hardest problems • Graphplan caches info when searching “too short” graphs, use to speed up search of expanded graph • For critically constrained problems, nearly all effort goes into searching last (or next to last) size problem

25. Next Steps... • 1. Domain-specific Control Knowledge • Encode state invariants & heuristics axiomatically • Trucks always in one location • Don’t move a package from a destination location Dramatic speedup possible (Kautz & Selman 1998) • For non-admissible control knowledge, tradeoff between speed / solution quality (Huang, Selman, Kautz AAAI-99) • Temporal logic specification used to generate axioms and/or prune plan graph • Using control knowledge from TLPlan (Bacchus 1996), can find better parallel plans • Current work: inductive learning of control knowledge

26. Comparison between Blackbox and TLPlan(Parallel Plan Length)

27. Next Steps... • 2. Beyond SAT: Planning with Resources and Optimization Criteria • SAT special case of 0/1 integer linear programming • ILPPlan(Kautz & Walser AAAI-99)Model extended STRIPS in AMPL, solve with • Branch and bound • Local search WSAT(OIP) • Current work: IP translator for BlackBox (Nau et al 1999) - better encodings for B&B solvers (Weld et al 1999) - new SAT+LP engine

28. Next Steps... • 3. Planning with Incomplete & Uncertain Information • The “Holy Grail” • SAT-encoding approaches • Contingent planning via QBF (Rintanen 1999) • C-MAXPLAN, ZANDER (Littman & Majercik 1999)Probabilistic planning via stochastic SAT state of the art performance on (small, hard) POMDP problems • Extensions to Graphplan • contingent plans (Weld, Anderson, Smith 1998) • probabilistic plans (Blum & Langford 1998) • GOAL: a universal BlackBox

29. Big Picture Polytime domain specific inference Abstract problem specification Domain specific model Polytime general inference General language encoding Encoding scheme Full general inference (NP complete) Combinatorial CORE Solution