State-Space Planning

1. State-Space Planning Dr. Héctor Muñoz-Avila • Sources: • Ch. 3 • Appendix A • Slides from Dana Nau’s lecture

2. Reminder: Some Graph Search Algorithms (I) 20 G = (V,E) G Z 10 16 Edges: set of pairs of vertices (v,v’) 9 B 6 F C 8 24 4 A 6 E Vertices D 20 • Breadth-first search: Use a queue (FIFO policy) to control order of visits • State-firsts search: Use an stack (FILA policy) to control order of visits • Best-first search: minimize an objective function f(): e.g., distance. Djikstra • Greedy search: select best local f() and “never look back” • Hill-climbing search:likegreedy search but every node is a solution • Iterative deepening: exhaustively explore up to depth i

3. State-Space Planning C A • Search is performed in the state space • State space • V: set of states • E: set of transitions (s, (s,a)) • But: computed graph is a subset of state space • Remember the number of states for DWR states? • Search modes: • Forward • Backward B C A B C B A B A C B A C B A B C C B A B A C A C A A B C B C C B A A B C

4. Nondeterministic Search • The oracle definition: given a choice between possible transitions, there is a device, called the oracle, that always chooses the transition that leads to a state satisfying the goals ... … ... Oracle chooses this transition because it leads to a favorable state “current state” ... • Alternative definition (book): parallel computers

5. goals+(g)  s and goals–(g)  s =  satisfied Forward Search take c3 … take c2 move r1 …

6. Soundness and Completeness • A planning algorithm A can be sound, complete, and admissible • Soundness: • Deterministic: Given a problem and A returns P  failure then P is a solution for the problem • Nondeterministic: Every plan returned by A is asolution • Completeness: • Deterministic: Given a solvable problem then A returns P  failure • Nondeterministic: Given a solvable problem then A returns a solution • Admissible: If there is a measure of optimality, and a solvable problem is given, then A returns an optimal solution • Result: Forward-search is sound and complete • Proof: …

7. Reminder: Plans and Solutions • Plan: any sequence of actions  = a1, a2, …, an such thateach ai is a ground instance of an operator in O • The plan is a solution for P=(O,s0,g)if it is executable and achieves g • i.e., if there are states s0, s1, …, snsuch that • (s0,a1) = s1 • (s1,a2) = s2 • … • (sn–1,an) = sn • sn satisfies g Example of a deterministic variant of Forward-search that is not sound? Dana Nau: Lecture slides for Automated PlanningLicensed under the Creative Commons Attribution-NonCommercial-ShareAlike License: http://creativecommons.org/licenses/by-nc-sa/2.0/

8. Deterministic Implementations s1 a1 s4 a4 • Some deterministic implementationsof forward search: • breadth-first search • depth-first search • best-first search (e.g., A*) • greedy search • Breadth-first and best-first search are sound and complete • But they usually aren’t practical because they require too much memory • Memory requirement is exponential in the length of the solution • In practice, more likely to use depth-first search or greedy search • Worst-case memory requirement is linear in the length of the solution • In general, sound but not complete • But classical planning has only finitely many states • Thus, can make depth-first search complete by doing loop-checking s0 sg s2 a2 s5 … a5 a3 s3 So why they are typically not used? How can we make it complete? Dana Nau: Lecture slides for Automated PlanningLicensed under the Creative Commons Attribution-NonCommercial-ShareAlike License: http://creativecommons.org/licenses/by-nc-sa/2.0/

9. Another domain: The Blocks World • Infinitely wide table, finite number of children’s blocks • Ignore where a block is located on the table • A block can sit on the table or on another block • Want to move blocks from one configuration to another • e.g., initial state goal • Can be expressed as a special case of DWR • But the usual formulation is simpler a d b c c e a b

10. Classical Representation: Symbols • Constant symbols: • The blocks: a, b, c, d, e • Predicates: • ontable(x) - block x is on the table • on(x,y) - block x is on block y • clear(x) - block x has nothing on it • holding(x) - the robot hand is holding block x • handempty - the robot hand isn’t holding anything d c e a b Dana Nau: Lecture slides for Automated PlanningLicensed under the Creative Commons Attribution-NonCommercial-ShareAlike License: http://creativecommons.org/licenses/by-nc-sa/2.0/

11. unstack(x,y) Precond: on(x,y), clear(x), handempty Effects: ~on(x,y), ~clear(x), ~handempty, holding(x), clear(y) stack(x,y) Precond: holding(x), clear(y) Effects: ~holding(x), ~clear(y), on(x,y), clear(x), handempty pickup(x) Precond: ontable(x), clear(x), handempty Effects: ~ontable(x), ~clear(x), ~handempty, holding(x) putdown(x) Precond: holding(x) Effects: ~holding(x), ontable(x), clear(x), handempty Classical Operators c a b c a b c a b c b a c a b Dana Nau: Lecture slides for Automated PlanningLicensed under the Creative Commons Attribution-NonCommercial-ShareAlike License: http://creativecommons.org/licenses/by-nc-sa/2.0/

12. Branching Factor of Forward Search a1 • Forward search can have a very large branching factor • E.g., many applicable actions that don’t progress toward goal • Why this is bad: • Deterministic implementations can waste time trying lots of irrelevant actions • Need a good heuristic function and/or pruning procedure • Chad and Hai are going to explain how to do this a2 a3 a3 … a50 a1 a2 goal initial state

13. Backward Search • For forward search, we started at the initial state and computed state transitions • new state = (s,a) • For backward search, we start at the goal and compute inverse state transitions • new set of subgoals = –1(g,a) • To define -1(g,a), must first define relevance: • An action a is relevant for a goal g if • a makes at least one of g’sliterals true • g effects(a) ≠  • a does not make any of g’s literals false • g+ effects–(a) =  and g– effects+(a) =  How do we write this formally? Dana Nau: Lecture slides for Automated PlanningLicensed under the Creative Commons Attribution-NonCommercial-ShareAlike License: http://creativecommons.org/licenses/by-nc-sa/2.0/

14. Inverse State Transitions • If a is relevant for g, then • –1(g,a) = (g – effects+(a))  precond(a) • Otherwise –1(g,a) is undefined • Example: suppose that • g = {on(b1,b2), on(b2,b3)} • a = stack(b1,b2) • What is –1(g,a)? • What is –1(g,a) = {s : (s,a) satisfies g}?

15. g1 a1 How about using –1(g,a) instead of –1(g,a)? a4 g4 g2 g0 a2 s0 a5 g5 a3 g3 Dana Nau: Lecture slides for Automated PlanningLicensed under the Creative Commons Attribution-NonCommercial-ShareAlike License: http://creativecommons.org/licenses/by-nc-sa/2.0/

16. Efficiency of Backward Search a1 a2 • Backward search can also have a very large branching factor • E.g., many relevant actions that don’t regress toward the initial state • As before, deterministic implementations can waste lots of time trying all of them a3 a3 … a50 a1 a2 goal initial state Compared with forward search? Dana Nau: Lecture slides for Automated PlanningLicensed under the Creative Commons Attribution-NonCommercial-ShareAlike License: http://creativecommons.org/licenses/by-nc-sa/2.0/

17. Lifting p(a,a) • Can reduce the branching factor of backward search if we partially instantiate the operators • this is called lifting foo(a,a) p(a,b) foo(x,y) precond: p(x,y) effects: q(x) foo(a,b) p(a,c) q(a) foo(a,c) … foo(a,y) q(a) p(a,y) Dana Nau: Lecture slides for Automated PlanningLicensed under the Creative Commons Attribution-NonCommercial-ShareAlike License: http://creativecommons.org/licenses/by-nc-sa/2.0/

18. Lifted Backward Search • More complicated than Backward-search • Have to keep track of what substitutions were performed • But it has a much smaller branching factor Dana Nau: Lecture slides for Automated PlanningLicensed under the Creative Commons Attribution-NonCommercial-ShareAlike License: http://creativecommons.org/licenses/by-nc-sa/2.0/

19. The Search Space is Still Too Large • Lifted-backward-search generates a smaller search space thanBackward-search, but it still can be quite large • Suppose a, b, and c are independent, d must precede all of them, and d cannot be executed • We’ll try all possible orderings of a, b, and c before realizing there is no solution • More about this in Chapter 5 (Plan-Space Planning) a d b c b d a b d a goal b a c d b d c a c d b Dana Nau: Lecture slides for Automated PlanningLicensed under the Creative Commons Attribution-NonCommercial-ShareAlike License: http://creativecommons.org/licenses/by-nc-sa/2.0/

20. STRIPS • π  the empty plan • do a modified backward search from g • instead of -1(s,a), each new set of subgoals is just precond(a) • whenever you find an action that’s executable in the current state, then go forward on the current search path as far as possible, executing actions and appending them to π • repeat until all goals are satisfied π = a6, a4 s = ((s0,a6),a4) g6 satisfied in s0 g1 a1 a4 a6 g4 g2 g a2 g3 a5 g5 a3 a3 g3 current search path Dana Nau: Lecture slides for Automated PlanningLicensed under the Creative Commons Attribution-NonCommercial-ShareAlike License: http://creativecommons.org/licenses/by-nc-sa/2.0/

21. The Sussman Anomaly Initial state goal • On this problem, STRIPS can’t produce an irredundant solution a b c c a b Dana Nau: Lecture slides for Automated PlanningLicensed under the Creative Commons Attribution-NonCommercial-ShareAlike License: http://creativecommons.org/licenses/by-nc-sa/2.0/

22. The Register Assignment Problem • State-variable formulation: Initial state: {value(r1)=3, value(r2)=5, value(r3)=0} Goal: {value(r1)=5, value(r2)=3} Operator: assign(r,v,r’,v’) precond: value(r)=v, value(r’)=v’ effects: value(r)=v’ • STRIPS cannot solve this problem at all Dana Nau: Lecture slides for Automated PlanningLicensed under the Creative Commons Attribution-NonCommercial-ShareAlike License: http://creativecommons.org/licenses/by-nc-sa/2.0/

23. How to Fix? • Several ways: • Do something other than state-space search • e.g., Chapters 5–8 • Use forward or backward state-space search, with domain-specific knowledge to prune the search space • Can solve both problems quite easily this way • Example: block stacking using forward search Dana Nau: Lecture slides for Automated PlanningLicensed under the Creative Commons Attribution-NonCommercial-ShareAlike License: http://creativecommons.org/licenses/by-nc-sa/2.0/

24. Domain-Specific Knowledge • A blocks-world planning problem P = (O,s0,g) is solvableif s0 and g satisfy some simple consistency conditions • g should not mention any blocks not mentioned in s0 • a block cannot be on two other blocks at once • etc. • Can check these in time O(n log n) • If P is solvable, can easily construct a solution of length O(2m), where m is the number of blocks • Move all blocks to the table, then build up stacks from the bottom • Can do this in time O(n) • With additional domain-specific knowledge can do even better …

25. initial state goal a d b e c c a b d Additional Domain-Specific Knowledge • A block x needs to be moved if any of the following is true: • s contains ontable(x) and g contains on(x,y) • s contains on(x,y) and g contains ontable(x) • s contains on(x,y) and g contains on(x,z) for some y≠z • s contains on(x,y)and y needs to be moved Axioms Dana Nau: Lecture slides for Automated PlanningLicensed under the Creative Commons Attribution-NonCommercial-ShareAlike License: http://creativecommons.org/licenses/by-nc-sa/2.0/

26. initial state goal a d b e c c a b d Domain-Specific Algorithm loop if there is a clear block x such that x needs to be moved and x can be moved to a place where it won’t need to be moved then move x to that place else if there is a clear block x such that x needs to be moved then move x to the table else if the goal is satisfied then return the plan else return failure repeat Dana Nau: Lecture slides for Automated PlanningLicensed under the Creative Commons Attribution-NonCommercial-ShareAlike License: http://creativecommons.org/licenses/by-nc-sa/2.0/

27. Easily Solves the Sussman Anomaly loop if there is a clear block x such that x needs to be moved and x can be moved to a place where it won’t need to be moved then move x to that place else if there is a clear block x such that x needs to be moved then move x to the table else if the goal is satisfied then return the plan else return failure repeat initial state goal a c b a b c

28. Properties • The block-stacking algorithm: • Sound, complete, guaranteed to terminate • Runs in time O(n3) • Can be modified to run in time O(n) • Often finds optimal (shortest) solutions • But sometimes only near-optimal (Exercise 4.22 in the book) • PLAN LENGTH for the blocks world is NP-complete Dana Nau: Lecture slides for Automated PlanningLicensed under the Creative Commons Attribution-NonCommercial-ShareAlike License: http://creativecommons.org/licenses/by-nc-sa/2.0/

29. An Actual Application: FEAR. • F.E.A.R.: First Encounter Assault Recon • Here is a video clip • Behavior of opposing NPCs is modeled through STRIPS operators • Attack • Take cover • … • Why? • Otherwise you would need too build an FSM for all possible behaviours in advance

30. Patrol • Preconditions: No Monster • Effects: patrolled • Fight • Preconditions: Monster in sight • Effects: No Monster FSM: Monster In Sight Planning Operators Patrol Fight No Monster A resulting plan: Monster in sight No Monster patrolled Fight Patrol FSM vs AI Planning Neither is more powerful than the other one

31. Patrol • Preconditions: No Monster • Effects: patrolled • Fight • Preconditions: Monster in sight • Effects: No Monster … reasoning knowledge Many potential plans: Planning Operators Fight Fight Fight Fight Fight Patrol Patrol Patrol Patrol Patrol … But Planning Gives More Flexibility • “Separates implementation from data” --- Orkin If conditions in the state change making the current plan unfeasible: replan!