Planning and Coordination in A Multi-Agent Environment.

1. Planning and Coordination in A Multi-Agent Environment. Gökay Burak AKKUŞ 2003700717 cmpe530 Boğaziçi University

2. Agent • An agent is something that acts. • An agent is anything that can be viewed as perceiving its environment through sensors and acting upon that environment through actuators. Gökay Burak AKKUŞ

3. Planning • The task of coming up with a sequence of actions that will achieve a goal is called planning Gökay Burak AKKUŞ

4. Planning • Classical Planning • Current State →→ Desired Goal • Fully Observable • Deterministic • Finite • Static • Discrete • Non-Classical Planning • Hierarchical, Decision-Theoretic, Continual, Distributed Gökay Burak AKKUŞ

5. An agent plans on the basis of anincrementally learnt world model and reacts on the basis of incrementally learntvalues that indicate the usefulness of his potential actions. Gökay Burak AKKUŞ

6. Planning • Single-Agent Environments • The agent is alone • Multi-Agent Environments • Other agents in enviroment considered • The solution to a global problem emerges from the collective activities of independent agents. Gökay Burak AKKUŞ

7. Planner Figure 1 : Planner [7] Gökay Burak AKKUŞ

8. Multi-Agent Planning • Cooperation • Joint-goals & plans • Coordination • Multy-body planning(centralized planning agent) • Synchronization • Cooperation - Competition Gökay Burak AKKUŞ

9. Planning Agent • A Single Agent for • Planning & Coordination • Execution & monitoring • Re-Planning(when something goes wrong) in real-time responses • Adaptation to dynamic environment • Performance Prediction of agents • A single agent search for multi-agent problems. • Search for actions to be taken • Search in state space (Agent + Action) Gökay Burak AKKUŞ

10. Planning Agent • Distributed Planning • Cooperative Distributed Planning (CDP), takes the individual planning process and distributes it among a subset of an agent society such that the generation and execution of a plan requires the interaction of several specialized agents • Negotiated Distributed Planning (NDP), is based on the concept that agents should primarily be focused on the individual, but can use plans as a means of coordinating actions Gökay Burak AKKUŞ

11. agents should work together according to the following model: • Each agent works on large grained sub-problems • Agents should be able to generate partial results, even if these results are uncertain or theagent is missing information • Agents should communicate partial results to other agents asynchronously as the partialresults are developed • Agents should use partial results from other agents to help resolve their own uncertaintiesand guide individual solutions • Our planning agent helps in these, as it is the root of coordination hierarchy Gökay Burak AKKUŞ

12. Learning, Planning, Reacting Figure-2 : relationships between learning, planning, and reacting. Gökay Burak AKKUŞ

13. Some Definitions • Reactive Coordination vs. Planning coordination • The Knowledge needed: • Ag = {A1,...An} : Available Agents in Environment • Set of Actions that can be performed by an agent in an environment • Set of Actions that can be performed at a certain state • Set of agents that can carry out actions to reach a successor state (S → T) • Estimated usefullnes of actions • Validity, Satisfiability of Results, Partial results • Joint Learning:Learning is realized by the agents through adjusting the estimates oftheir actions’ usefulness. Gökay Burak AKKUŞ

14. Performance Measures • A performance measure embodies the criterion for success of an agent’s behavior. • Objective performance measure : • Typically one imposed by the designer. • As a general rule it is better to design performance measures according to what one actually wants in the environment, rather than according to how one thinks the agent should behave. Gökay Burak AKKUŞ

15. Performance Evaluation • PACE : Performance Analysis and Characterisation Environment • PACE provides a means of dynamically obtaining runtimeestimates for different applications on different resourcesthrough the performance information servicesframework. • multiple (and oftenconflicting) metrics must be considered Gökay Burak AKKUŞ

16. Metrics • Total execution time • Average advance time • Resource Utilization Rate • Load balancing level Gökay Burak AKKUŞ

17. What To Do? • A certain goal is defined, • Planning agent searches the problem domain for possible instances of agents working in that domain (agent society) • If problem involves more domains, tries to divide problem into sub-problems considering domain interactions • Planning agent reaches the execution history of agents it will deal with. • Planning agent evaluates the agent in [0..1] scale by means of • Goal Satisfaction, (d1) • Resource usage, (d2) • Problem usefulnes, (d3) • Coordination capabilities (d4) • ... • 0<= d1*w1+d2*w2+d3*w3+d4*w4+...+dn*wn <= 1 • w: specifies the impotance weight of a metric for the problem • Then, it generates a joint plan using the ranking of agents to employ appropriate agents on specific tasks, • Monitors the ongoing activities, communication between agents, and when needed Re-planning takes place Gökay Burak AKKUŞ

18. Evaluation • For a better planning prediction of next action of an agent, in terms of predefined metrics, is important. • The prediction of next action has two components: • Previous experiences about the agent, • Possible results that will be generated by the agent for the new problem • So, a model that can be used to predict future behavior of an agent must be generated. • Possible modelling options: • Extrapolation, Gökay Burak AKKUŞ

19. Main Goal • A model that can work both under perfect and imperfect monitoring needs to be developed for evaluation process of agents. • underlying knowledge structure, previous task experiences and solution generation capabilities over a feasible time period, will be used as weights of evaluation • coordination for timely samplings of goal satisfaction for multi-agent systems will be taken into account Gökay Burak AKKUŞ

20. References • [1] William Hansel Turkett, Jr. (turkett@cse.sc.edu), Robust Multiagent Plan Generation and Execution with Decision-Theoretic Planners Dissertation Proposal, Department of Computer Science and Engineering University of South Carolina Columbia, SC 29208, Spring 2003 • [2] Gerhard Weiß (weissg@in.tum.de ), An Architectural Framework for Integrated Multiagent Planning, Reacting, and Learning Institut für Informatik, Technische Universitat München D-80290 München, Germany • [3] Romen I. Brafman (brafman@cs.bgu.ac.il), Moshe Tennenholtz (moshe@robotics.stanford.edu), Learning to Coordinate Efficiently: A Model Based Approach, 2003 • [4] Michael Brenner (brenner@informatik.uni-freiburg.de), MAPL: a Framework for Multiagent Planning with Partially Ordered Temporal Plans, Institut für Informatik, Universitat Freiburg, 79110 Freiburg, Germany • [5] Junwei Cao, Subhash Saini,Stephen A. Jarvis, Daniel P. Spooner, Helene N. Lim Choi Keung, Graham R. Nudd High Performance Systems Group, University of Warwick, Coventry, UK, Performance Prediction and its use in Parallel and Distributed Computing Systems • [6] Nils J. Nilsson, Artificial Intelligence A New Sysnthesis, Stanford University, 1998 • [7] Jonathan Gratch, Reasoning about Multiple Plansin Dynamic Multi-agentEnvironments, Information Sciences Institute University of Southern California, October 1998. Gökay Burak AKKUŞ

21. Thanks... Questions ? Gökay Burak AKKUŞ