Partial Order Planning Based on slides by: Carmel Domshlak

1. Partial Order PlanningBased on slides by:Carmel Domshlak Automated Planning and Decision Making Prof. Ronen Brafman Automated Planning and Decision Making 2007 1

2. Good article for this topic: • An Introduction to Least Commitment Planning – AI Magazine - Dan Weld Automated Planning and Decision Making 2

3. State Space Search • So far we have considered planning as search in state space • Forward - build a plan in the same order that it is executed • Backward - build a plan in the reverse order of its execution C C B A B A Automated Planning and Decision Making 3

4. State Space Search • Potential problem:Spending lots of time on trying the same set of actions in different orderings before realizing that there is no solution (with this set) • Key observation: In these algorithms, when we choose what to do, we also choose when to do it Automated Planning and Decision Making 4

5. Searching in the Space of Plans • In 1974, Earl Sacerdoti built a planner, called NOAH, that considered planning as search through plan space • Search states (nodes) = partially specified plans • Transitions (edges) = plan refinement operations • Initial state = null plan • Goal states = valid plans for the problems Move A to B Move B to CMove A to B Move C to TMove B to CMove A to B Move A to TMove A to B Automated Planning and Decision Making 5

6. State Space vs. Plan Space Search through plan space -- then what is a plan? • Answer I: Totally ordered sequences of actions (or meta-actions) • But then search through state space is isomorphic to search through plan space! • So what is the point of introducing “search through plan space”?? • Answer II: Opens the road to more interesting plan representations and more interesting operators on plans, in particular, partially ordered sequence of actions Automated Planning and Decision Making 6

7. Least Commitment Planning • Think how you might solve a planning problem of going for a vacation in Italy • Need to purchase plane tickets • Need to buy the “Lonely Planet” guide to Italy BUT there is no need to decide (yet) which purchase should be done first Least Commitment Planning • Represent plans in a flexible way that enables deferring decisions • At the planning phase, only the essential ordering decisions are recorded Automated Planning and Decision Making 7

8. Partial-Order Plans • Given a Strips task Π = (P,A, I,G) we search through a space of hypothetical partial-order plans • A plan (= search node) is a triplet: <A, O, L> in which • A is a set of actions from A, possibly with (labeled) repetitions • O is a set of ordering constraints over A • L is a set of causal links (a bit later) • Example: A = {a1, a2, a3}, O = {a1 < a3, a2 < a3} • Observe: Planner must (eventually) ensure the consistency of O. Automated Planning and Decision Making 8

9. Causal Links A key aspect of least commitment planning: keep track of past decisions and the reasons for those decisions • If you purchase plane tickets early to board the plane, make sure they’re with you when you get to the airport • If another goal causes you to drop the tickets (e.g., having your hands free to open the taxi door), then you should be sure to pick them up again. • A good way to reason about and ensure non-interference between different actions introduced into the plan is to record dependencies between actions explicitly • Causal links ap q ac records our decision to use ap to produce the precondition q of ac Automated Planning and Decision Making 9

10. Threats • Causal links are used to detect when a newly introduced action interferes with past decisions. • Such an action is called a threat • Suppose that • ap q acis a causal link in L (of some plan<A, O, L>) • at is yet another action in A • We say that atthreatens ap q acif • O U{ap < at < ac} is consistent, and • q ∈ del(at) Automated Planning and Decision Making 10

11. Eliminating Threats • When a plan contains a threat, then it is possible that the plan would not work as anticipated. • Which means what? • Solution: identify threats and take evasive countermeasures • promotion by O U= {at > ac} • demotion by O U= {at < ap} • . . . Automated Planning and Decision Making 11

12. *start* *end* Planning Problems as Null Plans Uniformity is the key to simplicity • Can use the same structure to represent both the planning problem and complete plans • Planning problem as a null plan<A, O, L> where • A = {a0, a1}, O = {a0 < a1}, L = {} • pre(a0) = {}, del(a0) = {}, add(a0) = I • pre(a1) = G, del(a1) = {}, add(a1) = {} (on c a) (clear b) (clear c) (on a table) (on b table) (on a b) (on b c) Automated Planning and Decision Making 12

13. The POP AlgorithmSchematic description Regressive algorithm that searches plan space • Starts with the null plan • Makes non-deterministic plan refinement choices until • all preconditions of all actions in the plan have been supported by causal links, and • all threats against any causal link have been removed Automated Planning and Decision Making 13

14. The POP AlgorithmInput and Output • Recursive calls to POP with POP(<A, O, L>, agenda, A) where • <A, O, L> is a plan structure • agenda is a list of “open goals” that need to be supported by causal links • A is the action set of our Strips problem • Initial call is with • null plan <{a0, a∞}, {a0 < a∞}, {}> • agenda = {(g, a∞) | g ∈ pre(a∞) ≡ G} If <A, O, L> is outputted by POP, then any total ordering of actions A consistent with O is a valid plan for our problem. Automated Planning and Decision Making 14

15. The POP Algorithm POP(<A, O, L>, agenda, A) • Termination:if agenda = Ø then return <A, O, L> • Goal selection:choose (q, aneed ) ∈ agenda • Action selection… Automated Planning and Decision Making 15

16. The POP Algorithm • Action selection: • choose action aadd (either from A, or from A) such that • q ∈ add(aadd), and • O U {aadd < aneed} is consistent • if no such action then return FALSE • otherwise • L U= {aadd qaneed} and O U= {aadd < aneed} • if aadd is a new action instance thenA U= {aadd}, and O U= {a0 < aadd < a1} • Update goal set: • agenda \= {(q, aneed)} • if aadd was a new action instance then agenda U= {(r , aadd) | r ∈ pre(aadd)} Automated Planning and Decision Making 16

17. The POP Algorithm POP(<A, O, L>, agenda, A) • Termination: if agenda = Ø then return <A, O, L> • Goal selection: choose (q, aneed ) ∈ agenda • Action selection: choose and process aadd . . . • Update goal set: add preconditions of aadd to the agenda ... • Causal link protection:foreach causal link {apq ac} ∈L, and at that threatens it • choose either O U= {at > ac}, or O U= {at < ap} • if neither constraint is consistent, then return FALSE • Recursive invocation: POP(<A, O, L>, agenda, A) Automated Planning and Decision Making 17

18. The POP AlgorithmIn one slide ... POP(<A, O, L>, agenda, A) • Termination:if agenda = Ø then return <A, O, L> • Goal selection:choose (q, aneed ) ∈ agenda • Action selection: • choose action aadd (either from A, or from A) such that • q ∈ add(aadd), and • O U {aadd < aneed} is consistent • if no such action then return FALSE • otherwise • L U= {aadd qaneed} and O U= {aadd < aneed} • if aadd is a new action instance then A U= {aadd}, and O U= {a0 < aadd < a1} • Update goal set: • agenda \= {(q, aneed)} • if aadd was a new action instance then agenda U= {(r , aadd) | r ∈ pre(aadd)} • Causal link protection:foreach causal link {apq ac} ∈L, and at that threatens it • choose either O U= {at > ac}, or O U= {at < ap} • if neither constraint is consistent, then return FALSE • Recursive invocation: POP(<A, O, L>, agenda, A) Automated Planning and Decision Making 18

19. Choice Points Three choice points • Goal selection • Action selection • Causal link protection How crucial these choices are? • Affect soundness? • Affect completeness? • Affect efficiency? Automated Planning and Decision Making 19

20. Example - Step 1 Initial call to POP with • Null Plan (see the right figure) • agenda = {(onAB, a∞) , (onBC, a∞)} First choice is goal selection • Affects efficiency, but not completeness! C A B Automated Planning and Decision Making 20

21. Example - Step 2 Suppose (onBC, a∞) is selected (i.e., aneed = a∞) • Need to choose an action aadd that will provide onBC • This is a real non-deterministic choice! Suppose that an oracle suggests making aadd a new instance of the action move-B-from-Table-to-C • a causal link aadd onBCa∞ is added to L • agenda is properly updated (how exactly?) • no threats to resolve . . . recursive call Automated Planning and Decision Making 21

22. *start* *end* (move b from table to c) (on c a) (clear b) (clear c) (on a table) (on b table) (clear b) (clear c) (on b table) (clear table) ¬(on b table) ¬(clear c) (on b c) (on a b) (on b c) Example - Step 2 Suppose that an oracle suggests making aadd a new instance of the action move-B-from-Table-to-C • a causal link aadd onBCa∞ is added to L • agenda is properly updated (how exactly?) • no threats to resolve . . . recursive call Automated Planning and Decision Making 22

23. *start* *end* (move b from table to c) (on c a) (clear b) (clear c) (on a table) (on b table) (clear b) (clear c) (on b table) (clear table) ¬(on b table) ¬(clear c) (on b c) (on a b) (on b c) Example - Step 3 Suppose (clearB,move-B-from-Table-to-C) is selected • Oracle suggests to reuse an existing action instance a0 • add a causal link a0 clearBmove-B-from-Table-to-C • agenda is properly updated (how exactly?) • no threats to resolve . . . recursive call Automated Planning and Decision Making 23

24. *start* *end* (move a from table to b) (move b from table to c) (on c a) (clear b) (clear c) (on a table) (on b table) (clear b) (clear a) (on a table) (on b table) (clear b) (clear c) (clear table) ¬(on b table) ¬(clear c) (on b c) (on a b) (clear table) ¬(on a table) ¬(clear b) (on a b) (on b c) Example - Step 4a • Suppose (onAB, a∞) is selected • Oracle suggests making aadd be a new instance of the action move-A-from-Table-to-B, and we do that ... • ... BUT this time we have a threat! • move-A-from-Table-to-B and move-B-from-Table-to-C have no constraints on their relative ordering • move-A-from-Table-to-B deletes clearB that is required by move-B-from-Table-to-C Automated Planning and Decision Making 24

25. *start* *end* (move a from table to b) (move b from table to c) (on c a) (clear b) (clear c) (on a table) (on b table) (on b table) (clear b) (clear c) (clear b) (clear a) (on a table) (clear table) ¬(on b table) ¬(clear c) (on b c) (on a b) (clear table) ¬(on a table) ¬(clear b) (on a b) (on b c) Example - Step 4b Try to protect the causal link a0clearB move-B-from-Table-to-C • In general, there are two options — promotion and demotion — and this is a true non-deterministic choice! • In our example, demotion is inconsistent (why?), but promotion is OK Automated Planning and Decision Making 25

26. *start* *end* (move b from table to b) (move b from table to c) (on c a) (clear b) (clear c) (on a table) (on b table) (on b table) (clear b) (clear c) (clear b) (clear a) (on a table) (clear table) ¬(on b table) ¬(clear c) (on b c) (on a b) (clear table) ¬(on a table) ¬(clear b) (on a b) (on b c) Example - Next steps What is now on the agenda? ... in A? ... in L? ... In O? Next steps follow the same lines of reasoning … Automated Planning and Decision Making 26

27. *start* *end* (move a from table to b) (move b from table to c) (move c from a to table) Example - Next steps Eventually POP returns Is it a correct partial order plan? (on c a) (clear b) (clear c) (on a table) (on b table) (on c a) (clear c) (clear a) (on c table) (on c a) (clear b) (clear a) (on a table) (clear b) (clear c) (on b table) (on a b) (clear table) ¬(on a table) (clear table) ¬(on b table) ¬(clear b) ¬(clear c) (on b c) (on a b) (on b c) Automated Planning and Decision Making 27

28. Advantages • Natural extension to planning with partially instantiated actions • ... add action instance move(A,x,B) • ... postpone unifying ?x with a concrete object until necessary • Natural extensions to more complex action formalisms • ... action durations • ... delayed effects • ... • Least commitment may lead to shorter search times • Mainly due to smaller branching factor • Another way of viewing it: constraint-based planning Automated Planning and Decision Making 28

29. Disadvantages • Significantly more complex algorithm • ... higher per-node cost • Hard to determine what is true in a state • ... harder to devise informed heuristics (for all three types of choices) • ... how to prune infinitely long paths?? Automated Planning and Decision Making 29

30. Literature 1. D. Weld, An Introduction to Least Commitment Planning, AI Magazine, 1994. 2. J.S. Penberthy and D. Weld, UCPOP: A sound, complete, partial order planner for ADL, KR, 1992. 3. ... for many more, see, e.g., bibliography in (1) Automated Planning and Decision Making 30