Reasoning about Multiple Plans in Dynamic Multi-agent Environments

1. Reasoning about Multiple Plans in Dynamic Multi-agent Environments Jonathan Gratch Information Sciences Institute University of Southern California October, 1998

2. Problem • This workshop focuses on ambitious domains • Integrate planning, execution, and repair • Integrate individual and multi-agent planning • Integrate collaborative and adversarial planning • Integrate organizational constraints: authority, autonomy… • Question: How to integrate all these capabilities in single system

3. Solution • How do we attain flexibility • Link multiple reasoning modules? • Collaborative box and adversarial box and execution box ... • CPEF (Myers) and MPA (Wilkins&Myers) of this flavor • Unified Theory of Cognition? e.g. SOAR • Take single relatively simple reasoning system • Demonstrate it supports necessary flexibility • Argue that relatively simpler planner can support this flexibility • Integrate planning, execution, and repair, collaborative and adversarial reasoning, organizational issues

4. Unified Theory? • SNLP McAllester and Rosenblitt • Actually Classical Planning • Specifically IPEM (Ambros-Ingerson), X11 (Golden) Integrates Planning, Execution, and Repair • How far can we push technology • Designed for single agent planning • can’t reason about other agents plans • no notion of collaboration collaboration • Can it support flexibility without significant modification?

5. Plan Management • “There’s more to life than making plans” - Pollack • Must reason about plans - plan management • intentions, commitments, ... • Get flexibility by adding plan management to classical planner • But how? Connection to classical planning little studied • Plan management as search control • Plan management places constraints on reasoning process • It can be viewed / implemented as a search control theory • Ground semantics of e.g. “intentions” in terms of the primitive decisions of general purpose planner

6. Approach • Take classical planner • Knows nothing of issues like collaboration, intentions, … • Doesn’t support reasoning of plans of others • Make simple change that • Allows reasoning about multiple plans • Allows different planning “stances” (collaborative, adversarial) • Allows easy encoding of meta concepts like joint intentions • Build layer on top of planner • Implements a set of control primitives • Implement plan management in terms of these primitives

7. Caveats • Deliberative planning • Projecting effects of future action • World changes on order of minutes, not seconds • Implementation interfaces with RAP-like subsystem • Address flexibility related to reasoning about plans • Broad but shallow • Integrate lots of capabilities • Not much subtlety • Focus on efficiency, pragmatics • Focused on domain of military command and control • Company and battalion-level helicopter commanders • Intention is to produce general model BUT… • Collaboration restricted to hierarchical organizations

8. First, Planning • Planner based on IPEM and X11 • Provides integrated model of planning, execution and repair Planner Plan Network Actions Constraints Simple establishment Step Addition Decomposition Conflict Resolution initiation / termination / fail retraction Current World Description Environment

9. Example: Planning Stances • Assuming I could represent plans of other, how should I use them in my planning. Depends... Initial State Watch(me,movie) Drive(me,movie) at(car,movie) at(car,home) Happy(me) at(me,movie) at(me,movie) at(me,home) My Plan

10. Example Wife’s Plan Drive(wife,work) Work(wife) at(wife,work) at(wife,work) at(wife,home) Make$$(wife) at(car,work) at(car,home) Initial State Watch(me,movie) Drive(me,movie) at(car,movie) at(car,home) Happy(me) at(me,movie) at(me,movie) at(me,home) My Plan

11. Planning stances • Rude: use other’s plans to serve my own goals • get to the car first • Strategy: resolve threats you introduce into my plan ignore threats I introduce into yours • Deferential: use other’s plan to defer to their actions • let them have the dumb car • Strategy: resolve threats I introduce into your plan work around threats you introduce into mine • Fair: use other’s plan to reach mutual solution • I’ll take a cab this time • Strategy: treat threats equally regardless of who’s plans it is

12. Helpful: use other’s plans to offer help • I’ll bring home some milk duds • Strategy: resolve threats in your plan that I didn’t cause • Authoritative: generate plans for other agents • You WILL drive me to work • Strategy: derive plans for my goals that other’s will execute • Adversarial: use other’s plan against them • Get to the car first, even if I don’t really want to go to work • Strategy: introduce threats into other’s plans

13. Example: Plan management • Plan management well studied in military • Military decision making process (MDMP) MDMP Mission Analysis Generate Plan Analyze Plan Transmit Plan Execute Plan Situation Interrupts Brief Subordinates Merge Subplans Repair Plan Transmit Changes

14. Approach • Support planning stances and plan management • Claim: • Different stances are really different search control strategies • Many stances arise from mixing key “control primitives” • Represent multiple plans • Associate control primitives w/ different plans • Primitives interact to control the planning process

15. Representing Multiple Plans • Redefine “plan” • Distinguish between plan and “plan network” • A plan is some subset of the plan network • Some set of tasks and constraints • Plans may overlap or contain subplans • Plan modification • determined by planning actions • establishment adds causal link to plan containing the open precondition • Plan creation • New plans created under domain specific control • e.g. when receiving a communication, insert the tasks into the network and create new plan structure

16. Example Wife’s Plan Drive(wife,work) Work(wife) at(wife,work) at(wife,work) at(wife,home) Make$$(wife) at(car,work) at(car,home) Initial State Watch(me,movie) Drive(me,movie) at(car,movie) at(car,home) Happy(me) at(me,movie) at(me,movie) at(me,home) My Plan

17. Control Primitives • Associate primitives with plans • These interact to determine search control • E.G. Modifiability • If a plan is unmodifiable, the planner cannot add or retract constraints to the plan in response to flaws • e.g. If an unmodifiable plan has an open precondition flaw, this cannot be resolved via simple establishment / step addition • Unmodifiability is a component of modeling authority • my bosses plans should be unmodifiable by me

18. Example Revisited unmodifiable Drive(wife,work) Retract at(wife,work) at(wife,home) at(car,work) at(car,home) Planner Retract Initial State Drive(me,movie) at(car,movie) at(car,home) at(me,movie) at(me,home) My Plan

19. Control Primitives • Plan executability • can I initiate tasks in this plan • Threat resolution • if an decision in one plan introduces a threat in another, is this ok? • If ok, Filter out actions by planner to resolve these threats • Role assignments • assume tasks have an “agent” variable • which agents can the planner bind to that variable • if I’m helpful I can add myself to tasks in your plan • if I’m authoritative I can add you to tasks in my plan • can be represented via codesignation constraints

20. Control Primitives • Commitment • Plans can be either conjectured or comitted • Two conjectured plans can’t create flaws in each other • Allows us to consider alternative courses of action • Allows us to represent proposed changes to other plan • Related to representing intentions

21. Dynamic Example • Most plan management involves complex reasoning • e.g., joint intentions (Cohen&Levesque) • Do not modify plan unless achieved, failed, irrelevant • Communicate to others if achieved, failed, irrelevant • Can we avoid introducing new reasoning system • Goal, push general purpose planner as far as possible • Represent plan management as just another plan • Declaratively represent properties of plans • Control primitives under control of plan management plan

22. Deliberate Example • Military decision making process (MDMP) • Requires reasoning about when to change phases Generate Plan Create a plan free of flaws Don’t execute it until it is commonly known Tell the plan to my collaborators Should modify it once it is transmited Transmit Plan Run the plan till completion assuming there are no unpredicted flaws Execute Plan

23. MDMP Plan • How does one decide which plans have which props? • Plan-predicates declaritively represent plan properties • Plan-predicates are “part of the world” as in Georgeff’s PRS Transmit Plan Generate Plan -Flawed(p) CommonlyKnown(p) Goal(g) Plan-for(g, p) -Modifiable(p) -Flawed(p) -Modifiable(p) -Executable(p) Repair Plan Flawed(p) -CommonlyKnown(p) -Flawed(p)

24. Approach Plan Management Domain Theory domain independent Plan Management Plan Plan Manager Base Domain Theory Base Plans general purpose Reasoner (Planner)

25. Tactical simulations for training • Battalion-level deep-strike missions • 1 battalion planning agent • 2 company planning agents • 10 helicopter execution agents • 3000 other friendly and enemy units • Participated in 2-day simulated exercise: STOW97 • Collaborative planning and execution in hierarchical organizations • develop plan, contract out details to subordinates, monitor execution and replan as needed

26. Related Work • Work on lots of pieces, nothing in classical planning framework • Joint Intentions/ Shared plans: • Focus on collaborative agents • intentions can be seen as a form of search contol • don’t represent authority. Don’t consider selfish / adversarial reasoning • Do have richer model of commitment and mutual belief • Decker&Lesser GPGP • Focus on collaborative scheduling • Don’t represent authority, adversarial reasoning. • Have focused more on coordination strategies • Barbuceanu • Has a model of obligation that could represent authority/deference • Martin&Barber - levels of autonomy approach

27. Issues • Provide a framework but little guidance of how to use Provides a platform for flexible reasoning but: • Use domain specific search control for: • balance planning and execution • respond to changes in the world • Use domain theory to specify: • intentional reasoning • how to maintain coordination To what extent do control primitives support existing theories • e.g. Have impoverished notion of mutual belief

28. Issues • Collaboration in hierarchical organizations • What is a primitive task? • Level of abstraction impacts replanning / coordination • Different levels have different domain theories • have to resolve ambiguities • have to resolve conflicting views • Plan execution involves plan recognition of subordinate activities