Landscape of Solution Approaches

1. Landscape of Solution Approaches • coordinated action Coordinating Executing Plans Heterogeneous P/S Agents Negotiation-based Approaches Self- Scheduling Systems Market Mechanisms Constraint-based Resource Mgnt. • independent jobs/goals • optimized team behavior • high dynamics

2. [1981–1991] The Distributed Vehicle Monitoring Problem • Acoustic vehicle tracking • grammar specifies vehicle's “signature” • varying signal strengths • uncorrelated noise • “ghost tracks” • Multiple agents with overlapping sensors • faulty sensors • Coordinate processing to terminate as quickly as possible

3. [1981–1991] The Distributed Vehicle Monitoring Problem • Making choices about what activity to do... • ...in what order • ...and at what time • Coordinate processing to terminate as quickly as possible • These choices do make a difference

4. Distributed Continual Planning via Local Plan Merging By combining together interacting local goals/plans of different agents, an agent constructs partial global goals and plans • To guide an agent in reordering its actions so as to exploit results from other agents and avoid resource contention • To provide results in a timely manner that could be helpful for the solution of other agents goals • To avoid redundant solution of goals except where desirable • To achieve a more accurate view of the global importance of it achieving a local goal

5. Partial Global Planning (Durfee & Lesser, 1991) Each agent constructs and maintains an intermediate level view of its likely plans that would occur over the near term. • Expected goal order • Time estimates for each goal • Goal importance + expected result quality • High level plan for locally solving each goal Use a meta-level organization to know who is responsible for what aspects of plan coordination—to whom to send this info • Reduce costs by transmitting only “best” goals/plans • update when changed

6. Recognizing More Global Goals • Each agent receives subset of other agents’ goals and plans • Subset leads to partial global view • Potential for different agents having different views • Compare goals of different agents’ plans: • use simplified domain knowledge, • find goals that could be part of a larger goal, • generate partial-global-goal.

7. Improving Coordination • PGP interleaves agents’ planned activities into a plan-activity-map: • Each activity has predicted start and end times, results • Plan-activity-map roughly predicts concurrent activities • Plan-activity-map = partial global SCHEDULE • Rate each activity based on expected costs and results, how it is affected by preceding acts, and how it affects succeeding acts • SCHEDULING PHASE: Iteratively reorder acts until sum of ratings does not improve • Hill-climbing, possibly non-optimal ordering

8. Planning Solution Integration • Identify in plan-activity-map (schedule) when each solution piece will be developed at some agent; • Find times and locations where results can be combined • Create solution-construction-graph • Permit integration redundancy to increase reliability • Graph improves communication decisions by only sending information when needed; • Graph improves flexibility (time-windows) for choosing plans to pursue. • Introduces expected interactions -- primitive form of commitment: current decisions based on assumed future activity; • change of plans causes retractions of assumptions.

9. Partial Global Planning (cont) • Map PGP back to local plans: Partial global plan commitments are internalized • Local plan executed • Cycle repeats as local plans change or new plans from other agents arrive. Always acting on local information means that there could be inconsistencies in global view, but these are tolerated

10. Key Assumptions of PGP • Agents can predict the intermediate-level goal structure that is the focus of their near-term work with some level of accuracy and without significant computation; • Agents can estimate how long it takes to achieve goals; • Agents generally follow the prescribed order for achieving goals; • Agents can recognize the major subproblem/goal interactions among agents using intermediate-level goals; • Agents can transmit intermediate-level goal structure without significant communication costs.

11. Generalized Partial Global Planning • Domain-independent, coordinated scheduling of agent actions • Action choice, order, and timing • Generalizes and extends Durfee’s PGP algorithm, and von Martial’s work on task relationships • Deadlines • Heterogeneous agent capabilities • Communicate less info than PGP, and at multiple levels of abstraction • Individual Coordination Mechanisms • Recognize certain task structure patterns • Re-write the agent’s HTN • Respond via instantiating a protocol for communicating commitments, non-local task structure information, and partial results. • Works in conjunction with agent’s local task scheduler to remove uncertainty [rather than PGP hill-climbing] • (DTC — Wagner; DTT — Garvey; DRU — Graham)

12. TÆMS Task Structure Representation • Representing the “interdependencies” that need to be managed in “complex” domains • worth-oriented (vs. state- or task-oriented) • time-oriented (synchronization, not just choreography) • distributed: no global view • uncertainty in action characteristics & outcomes

13. TÆMS Task Structure Representation • “Interdependency” = quantitative change in task characteristics when another task is executed • Quality • Cost • Duration (vs. deadline) • …could be others (precision, reliability, #deaths, etc.) • State-based semantics • Annotation for HTN style task networks

14. Actions/Executable Methods • Characteristic Vector • maximum possible cost, quality, duration [c0, q0, d0] • associated uncertainty • Execution Profile • start, suspend/resume, finish • Accumulation Function: Characteristics vs execution time • Quality Accumulation Function [QAF]

15. Tasks • Characteristic Accumulation Functions • Quality Accumulation Function [QAF] Q(t) = Max(Q (t),Q (t)) = 1 OR Q(t) = Min(Q (t),Q (t)) = 0 AND A B A B B A B A q = 1 q = 1 q = 1 q = 1 0 0 0 d = 1 0 d = 1 d = 1 d = 1 0 0 0 Q (t) = 0 0 Q (t) = 1 Q (t) = 0 Q (t) = 1 B A B A Exactly one Q(t) = 0 SUM Q(t) = Q (t)+Q (t)+Q (t) = 2 A B C B B A A C q = 1 q = 1 q = 1 q = 1 q = 1 0 0 0 0 0 d = 1 d = 1 d = 1 d = 1 d = 1 0 0 0 0 0 Q (t) = 1 Q (t) = 0 Q (t) = 1 Q (t) = 1 Q (t) = 1 B B A A C

16. Performance Measure • Utility function over characteristic vector • maximize quality • maximize quality - cost • minimize duration subject to Qactual > Qmin • etc.

17. TÆMS Representation Framework Develop framework to specify the task structure of any computational environment • Attempt to maximize performance measure • Represent structure at multiple levels of abstraction • Tasks • Executable methods • Methods have duration, max quality, QAF • Explicit, Quantitative representation of task interrelationships

18. Non-Local Effects & Coordination Relationships • NLE’s are defined when the execution of one method changes the duration or quality or cost of another • NLE’s give an environment its unique characteristics • A NLE may depend on the communication of information • A NLE between parts of a task structure known by different agents is called a coordination relationship

19. NLEs have quantitative defs See also: DARPA CTÆMS specification…

20. TÆMS Usage • TÆMS can be used for environment modeling, algorithm analysis, and simulation • UMass simulators: TÆMS2, MAS • DARPA COORDINATORS • Agents may use any internal representation; but if task structure is created dynamically must translate • However, can use TÆMS to build domain independent reasoning capability into an agent architecture that represents task structures internally • Planning, Scheduling, Coordination

21. Hospital Scheduling

22. Generalized Partial Global Planning (GPGP, Decker & Lesser, 1995) • Mechanisms to generalize PGP • updating non-local viewpoints • communicating results • handling redundancy of effort • resolve conflicts (hard constraints) • handle soft constraints (“optimize”) • Examines tradeoffs of using mechanisms according to • communication overhead • execution time • plan quality • missed deadlines

23. A B enables Do? Do Do before DL? Do before DL Do after EST? Do after EST

24. Mutual Objective Task Group Agent B Agent A Agent B’s View Agent A’s View Minimizing non-local information

25. Example: Coordination by Reservation Agent A TaskA Agent A’s Model of Agent B Act1 TaskB enables What is Act1’s Quality, Cost, Duration? Does Agent B even know I need TaskB?

26. Agent A Agent B TaskA CM1a CM1 TaskB Confirm Remote Ask What If? Propose Process Confirm Act2 Act1 TaskB Reply Reply 1. When can you finish TaskB? [GPGP Reservation CM Protocol] 2. Commit TaskB finish at time t1, quality 34, cost 6. 3. Agreed. 4. Here is TaskB’s result. Example: Coordination by Reservation

27. Implementation • Assume agent has local scheduling capability • Attempt to maximize utility (self, shared, whatever) by future action sequence • Problem is non-local effects make schedule more uncertain or simply unknown (I can’t start my task until Agent B does Task B) • Other assumptions needed for full range of mechanisms • Some way to do “what-if” schedule reasoning • Ability to make commitments to do, don’t, and do w.r.t earliest start times and deadlines • Ability to move code for action promotion/demotion

28. Plan file Incoming KQML/FIPA messages Incoming Message Queue Objectives Queue What-if? Task Queue Task Queue Agenda Queue Agent Initialization Dispatcher Planner GPGP Scheduler Executor Task Templates Hash Table Action Results Queue Pending Action Queue [concurrent] Domain Facts and Beliefs Action Modules Action Modules Action Modules Action Modules Outgoing KQML/FIPA messages Action Modules http://www.cis.udel.edu/~decaf DECAF Architecture