Continuous Planning

1. Continuous Planning 15-887 Planning, Execution, & Learning Term Project Trey Smith May 2, 2000

2. Talk Outline • Planning outside classical paradigm (recap on Martha Pollack’s talk) • A continuous planner (CASPER) • Multi-resolution planning • Concept overview • A home-brew probabilistic resource representation • Dana Nau’s Simple Hierarchical Ordered Planner (SHOP) • My MRP implementation

3. Outside Classical Planning • Ways in which real world planning violates the classical planning paradigm: • Varying degrees of goal satisfaction • Multiple agents • Actions have temporal extent • Incomplete knowledge • Probabilistic world state, action effects • Dynamic environment (exogenous events) • Dynamic goals

4. Continuous Planning • Avoid batch-style sense-plan-act setup • While executing, continuously modify plans on the basis of • New state information • New goals • Interleave planning and execution because • Devoting less time to planning leaves more time for acting • Early execution gains information to help later planning • Well, how about if we have two processors?(or maybe just periods during which we have nothing to do but think anyway)

5. Continuous Planning • Blurs line between planning and execution. Scaling from less capable to more capable executive: • Execute a sequence • Execute a conditional sequence with explicit branch points to handle failures • Same as above, but with opportunistic side-plans • Same as above, but with state projection and resource conflict detection on the fly • Same as above, but with plan repair ability ?

6. CASPER • Continuous Activity Scheduling, Planning, Execution, and Replanning • ST4 Champollion mission scenario • On-going, iterative repair strategy, plans and replans in response to • State conflicts (e.g. equipment failure) • Resource conflicts (e.g. oversubscription of memory) • Activity updates (e.g., drilling finishing late) • New goals • JPL, Steve Chien; iterative repair is an old idea

7. CASPER • Mars scenario: central, static planner distributes plans to continuous planners on multiple rovers • Improved response time (RAX took 4 hours for a 3 day plan). Implies • Models don’t need to be as detailed • Fewer scenarios require explicit recovery in executive procedures

8. CASPER • Weakness of current implementation: not very good at noticing new opportunities, since repair is based on conflict detection • Claim: Distinction between planner and executive is now just a matter of horizon length • How does it address this trade-off? • Need for expressive procedural control • Difficulty of reasoning about an expressive language

9. Abstraction Time Multi-resolution Planning • Plan at higher detail for critical times • Near term, because • We need to execute it now • We have more information about it • Crisis points (e.g. planetary fly-by) • Times when resources tend to be oversubscribed • Trying to guarantee the downward solution property: there may not be a suitable abstract plan • With less data, may be forced to do contingency planning

10. Probabilistic Resource Representation • Amount of resource remaining, or usage of a particular action, is a random variable X • The randomness may come from true uncertainty, or from bounded rationality. • Representation of X as intervals with probability assignments { ([t00..t01], p0), ([t10..t11], p1) } • Says “proportion pi of the probability mass lies inside interval [ti0..ti1]”. • Same interval may be specified more than once, and intervals may overlap. • General rule is, for pairs (T1, p1), …, (Tn,pn) P(X in T1 U … U Tn) > p1 + … + pn • Allows +infinity and -infinity values for interval endpoints.

11. Probabilistic Resource Representation A • Can easily add intervals • Generalize to adding the entire random variables • With n intervals, get a lower bound on P(X > a) in O(n) time • Adding k random variables with q intervals each gives n = q^k B A+B A B A-B X={(A,0.5)} Y={(B,0.2), (C,0.3)} X+Y={(A+B,0.1), (A+C,0.15)}

12. Probabilistic Resource Representation • Comparison to other representations • Probabilistic aspect (c.f. C-Buridan, Weaver) • Possibilistic aspect (c.f. Matt Mason’s talk) • Related to Goodwin & Simmons work on bounding intervals for abstract plans • Advantages • Easy to come up with this kind of possibilistic, probabilistic mix (as opposed to having to specify a distribution) • Probabilistic OR, arithmetic operations are easy (though the complexity can get bad if not careful… interval merging?) • I don’t have a good way of representing possibilistic OR • Summary: not sure when this approach would be more useful than existing techniques: just an idea

13. SHOP • Simple Hierarchical Ordered Planner (Dana Nau) • Served my purposes very well • Licensed under GPL • Entire planner about 500 lines of LISP • Well documented, lots of examples • Algorithm too restrictive for full-bore multi-resolution planning

14. SHOP algorithm • Very much in the STRIPS vein • Select first task • Choose a variable binding which unifies it with an operator (primitive) or method (abstract) • If we chose a method, select an expansion based on the current state • Makes it trivial to maintain state • SHOP allows arbitrary LISP expressions to modify state Track state here

15. Consultant Domain • Fly-by-night programmer chooses what work to do each day, has active night life that requires lots of cash • Could design a web site or hack a bank • Web site • Porn sites don’t yield much money • On-line auction sites yield more but require sub-contracting to finish on time, so you need front money • Bank hacking • Potentially more lucrative, but no guaranteed payoff • Some banks riskier (but more profitable) than others • Goal statement: live for a few days, end up with money at least a with probability p.

16. Multi-resolution Aspect • I added a feature to SHOP which does DF iterative deepening on the resolution parameter • Preconditions of some method expansions depend on the value of resolution. • I set up the domain to have the downward solution property: for instance, if you don’t choose which bank to hack (at low resolution), the result of the action is as risky as any bank and as low-paying as any bank. • As a result, there may be no plans at low resolution, forcing you to try planning at high resolution.

17. Some results • The planner can come up with 3-day plans like: • Lax constraints on final cash (works at low-res) • do-web-site, do-web-site, do-web-site • Tighter constraints on final cash • do-porn-site, do-porn-site, do-auction-site • Note that we didn’t need to plan everything in detail, but did • Require more cash, but risk allowed (small p) • do-web-site, do-web-site, hack-bank • Timing example (CPU time, seconds): • Multi-res on: easy=0.09 hard=0.29 • Multi-res off: easy=0.19 hard=0.20