Multi-Agent Systems (Chapter 9) Adapted with permission from Adina Magda Florea adina@wpi

1. Multi-Agent Systems(Chapter 9)Adapted with permission from Adina Magda Floreaadina@wpi.edu Slides from Sobah Abbas Peterson

2. Benevolent vs.. self-interested agents • Benevolent: cooperative distributed systems. (CDPS) Simplifies the task enormously. • Self-interested- potential for conflict Slides from Sobah Abbas Peterson

3. Distributed problem solving • Group coherence - agents want to work together - cooperative agents • Competence - agents must find ways to work together - coordinate to cooperate • Task and result sharing - an agent has many tasks to do and asks other agents to do some of its tasks; then it should integrate the results • Distributed planning - the problem to be solved is to design and execute a plan in a distributed manner, by many agents Slides from Sobah Abbas Peterson 3

4. Distributed Problem Solving • Motivations: • Speed up through parallelization • Distribution of expertise • Distribution of Data, features change • Problem is inherently distributed • Distribution of Results • General Steps • Task decomposition • Task allocation • Exchange sub problem solutions • Task accomplishment • Results Synthesis (make whole) Slides from Sobah Abbas Peterson

5. Task Decomposition • Partitioning of a task into sub-tasks for possible allocation to another agent • Goal is to make sub-tasks independent Minimize coordination (so communication costs don’t outweigh gain) • Minimize shared data • Minimize share resources • Task decomposition is a hard problem and generally performed a priori by system designers. Slides from Sobah Abbas Peterson

6. Task Allocation • Homogenous Systems • Agents identical, allocation simple since each is equally qualified to work on sub-tasks • Heterogeneous Systems • Sub-task requirements - matched to agent skills • Potentially difficult problem (perfect match problem) Slides from Sobah Abbas Peterson

7. Which kind of system to build? • Homogenous systems are simpler • Only one kind of agent to build • Don’t have to consider agent skills when distributing sub-tasks • Homogenous systems considered unsuitable for complex problems • Low overall utilization of skills and resources Slides from Sobah Abbas Peterson

8. Agent Roles in Task Allocation • Agents can assume two roles • Servers: Agents capable of providing a service • Clients: Agents requiring a service • Agents can be both • I.e. An agent may use the services of other agents to complete a service is to providing to another agent • Task allocation systems must provide a way to match clients with servers Slides from Sobah Abbas Peterson

9. Centralized Allocation Systems • Centralized • 3rd party manages client-server matching • Hierarchical Subordination • Superior agents order subordinates to carry out task. • Typically a static, pre-defined agent organization • “Egalitarian” - all agents considered “equal” • Requires special “broker” or “trader” agents to manage client requests and server bids • Allows centralized allocation techniques Slides from Sobah Abbas Peterson

10. Egalitarian Allocation System RejectC C RequestA RequestA A B AcceptD AcceptD D Client Trader Servers Slides from Sobah Abbas Peterson

11. Distributed Allocation Systems • Each agent individually attempts to obtain required services • Acquaintance Network • Direct Allocation • Agents can only use the services of the agents it knows about • Potentially serious scalability issues • Delegated Allocation • Agents can ask other agents to use their acquaintances to find an agent capable of providing a particular service • Requires strongly connect acquaintance network • Both methods require accurate knowledge of agent skills • May use various “caching” strategies to maintain and age acquaintance information Slides from Sobah Abbas Peterson

12. Distributed Allocation Systems (cont) • Contract Net • “Market Place” approach • Clients issue description of tasks • Servers reply with bids • Client chooses the best bidder • Server affirms its commitment • Proven approach from other disciplines/simple • Well suited for dynamic environments • Concurrent and many-to-many nature of the protocol creates challenging race conditions Slides from Sobah Abbas Peterson

13. Task Allocation System Tradeoffs Distributed • Benefits • Coherence • Drawbacks • Bottleneck • Fault Intolerance Centralized Trader Acquaintance Contract Net • Benefits • No Bottleneck • Fault tolerance • Drawbacks • Coherence • Scalability • Latency • Benefits • Proven/Simple • Flexibility • Drawbacks • Message volume • Temporal & Spatial Ignorance Slides from Sobah Abbas Peterson

14. Types of Tasks • Independent • Tasks are self-contained • Can be performed in any order and concurrently • Interdependent • The solutions of some sub-tasks are required for the solution of other sub-tasks • Coordination possible if dependencies known before • Possible dependencies only become apparent at runtime • A Results Sharing mechanism is needed to solve these dependencies Slides from Sobah Abbas Peterson

15. Motivations for Results Sharing • Confidence: • Independent derivations affirm/challenge previous results leading to more confidence • Completeness: • Combination of partial results leads to a larger set of results • Precision: • Sharing of results allows for iterative refinement (agents come to see interface) • Timeliness: • Obvious performance benefits via parallel processing Slides from Sobah Abbas Peterson

16. A1 A2 A3 Result Sharing • Problem solving proceeds by agents cooperatively exchanging information as the solution is developed. • Results may be shared: • proactively - one agent sends another agent some information because it believes that the other will be interested in it. • reactively – an agent sends information to another in response to a request. Slides from Sobah Abbas Peterson

17. Result Sharing Benefits • Confidence (checking solutions) • Completeness/precision: share local views • Timeliness: may get results faster (even if agent could do it himself) Slides from Sobah Abbas Peterson

18. What about inconsistency? • Ignore it – but are you throwing away the true information (the part that doesn’t fit the expectation)? • Resolve it through negotiation • Degrade gracefully • progress opportunistically (not in strict predetermined order) • communicate high level results, not raw data • inconsistency resolved as you go (not at end) • no single solution route (if one is problematic, try another) Slides from Sobah Abbas Peterson

19. The Coordination Problem • Managing the interdependencies between the activities of agents. e.g. • You and I both want to leave the room. We independently walk towards the door, which can only fit one of us. I graciously permit you to leave first. Slides from Sobah Abbas Peterson

20. Coordination Techniques • Organisational Structures • Multi-agent Planning • Norms and social laws • Coordination Models based on human teamwork: • Joint commitments (Jennings) • Mutual Modelling Slides from Sobah Abbas Peterson

21. Organizational Structuring • Organizes agents into an organization • May be based on how the task was decomposed • Agents use knowledge of the organization to • Determine with whom to communicate • Prioritize tasks • Agents only need to know about the local organizational structure (coherence) • Choosing an organization structure can, itself, be a difficult problem! Slides from Sobah Abbas Peterson

22. Organizational Structuring Geographicallydistributed “cells” Slides from Sobah Abbas Peterson

23. Organizational Structures • A pattern of information and control relationships between individuals. • Responsible for shaping the types of interactions among the agents. • Aids coordination by specifying which actions an agent will undertake. • Organizational structures may be: • Functional (based on skills) • Spatial (based on physical location) • Temporal (based on time relationship) Slides from Sobah Abbas Peterson

24. Organizational Structure Models • A pattern for decision-making and communication among a set of agents who perform tasks in order to achieve goals. e.g. • Automobile industry • Has a set of goals: To produce different lines of cars • Has a set of agents to perform the tasks: designers, engineers, salesmen Slides from Sobah Abbas Peterson Reference: Malone 1987

25. Product Manager 2 Product Manager I Salesman Designer Engineer Salesman Designer Engineer Alternative Coordination Structures 1Product Hierarchy Slides from Sobah Abbas Peterson

26. Design Manager Sales Manager Engineering Manager Designers Salesmen Engineers Alternative Coordination Structures 2Functional Hierarchy Product Manager (several products) Slides from Sobah Abbas Peterson

27. Design Manager Sales Manager Engineering Manager Designers Salesmen Engineers Alternative Coordination Structures 3Centralised Market Product Manager 3 Product Manager 1 Product Manager 2 Functional Managers Slides from Sobah Abbas Peterson

28. Alternative Coordination Structures 4Decentralised Market Product Manager 3 Product Manager 1 Product Manager 2 Designers Salesmen Engineers Slides from Sobah Abbas Peterson

29. Comparison of Organization Structures – the Issues! Slides from Sobah Abbas Peterson

30. Organizational Structures - Critique • Useful when there are master/slave relationships in the MAS. • Control over the slaves actions – mitigates against benefits of DAI such as reliability, concurrency. • Presumes that atleast one agent has global overview – an unrealistic assumption in MAS. Slides from Sobah Abbas Peterson

31. Agenti Overlapping area Vehicle track Agentj Partial Global Planning (PGP) • A DAI testbed – Distributed Vehicle Monitoring Testbed (DVMT) – to successfully track a number of vehicles that pass within the range of a set of distributed sensors (agents). • Each agent monitors a dedicated area • There could be overlapping areas Slides from Sobah Abbas Peterson

32. Partial Global Planning (PGP) • Main principle: cooperating agents exchange information in order to reach common conclusions about the problem solving process. • Why is planning partial? • The system does not generate a plan for the entire problem. • Why is planning global? • Agents form non-local plans by exchanging local plans and cooperating to achieve a non-local view of problem solving. Slides from Sobah Abbas Peterson

33. Partial Global Planning (PGP) • Starts with the premise that tasks are inherently decomposed. • Assumes that an agent with a task to plan for might be unaware as to what tasks other agents might be planning for and how those tasks are related to its own. • No individual agent might be aware of the global tasks or states. • Purpose of coordination is to develop sufficient awareness. Slides from Sobah Abbas Peterson

34. Partial Global Planning (PGP) • PGP involves 3 iterated stages: • Each agent decides what its own goals are and generates short-term plans in order to achieve them. • Agents exchange information to determine where plans and goals interact. • Agents alter local plans in order to better coordinate their own activities. Slides from Sobah Abbas Peterson

35. Partial Global Planning (PGP) • Partial Global Plan: a cooperatively generated datastructure containing the actions and interactions of a group of agents. • Contains: • Objective – the larger goal of the system. • Activity map – what agents are actually doing and the results generated by the activities. • Solution construction graph – a representation of how the agents ought to interact in order to successfully generate a solution. Slides from Sobah Abbas Peterson

36. Partial Global Planning (PGP) • A DAI testbed – revisited. Agenti Overlapping area Vehicle track j i Agentj Slides from Sobah Abbas Peterson

37. Coordination Techniques • Organisational Structures • Multi-agent Planning • Norms and social laws • Coordination Models based on human teamwork: • Joint commitments (Jennings) • Mutual Modelling Slides from Sobah Abbas Peterson

38. Multi-agent Planning • Agents generate, exchange and synchronise explicit plans of actions to coordinate their joint activity. • They arrange apriori precisely which tasks each agent will take on. • Plans specify a sequence of actions for each agent. • It is a trade-off between specificity and reactive. Slides from Sobah Abbas Peterson

39. Multi-agent Planning • Two basic approaches: • Centralised – plans of individual agents analysed by a central coordinator to identify interactions. • Distributed – a group of agents cooperate to form a: • Centralized plan • Distributed plan Big difference between them! Slides from Sobah Abbas Peterson

40. Multi-agent Planning • Distributed Planning for centralised plans: • e.g. Air traffic control domain (Cammarata) • Aim: Enable each aircraft to maintain a flight plan that will maintain a safe distance with all aircrafts in its vicinity. • Each aircraft send a central coordinator information about its intended actions. The coordinator builds a plan which specifies all of the agents’ actions including the ones that they should take to avoid collision. Slides from Sobah Abbas Peterson

41. Multi-agent Planning • Distributed Planning for distributed plans: • Individual plans of agents, coordinated dynamically. • No individual with a complete view of all the agents’ actions. • More difficult to detect and resolve undesirable interactions. Slides from Sobah Abbas Peterson

42. 2 Distributed planning • What can be distributed: • The process of devising a plan is distributed among agents • Execution is distributed among agents Planning • State representation and plan representation • Search vs. planning • representation of changes to the world state • representation of and reasoning about the plan (steps/actions) Planning  Search Slides from Sobah Abbas Peterson 42

43. on(B,A) S1: move(B,T,A) on(B,T) clear(B) clear(A) movetotable(A,B) move(A,B,y) S2: move(A,B,E) clear(A) clear(E) on(A,B) ………….. …………. on(E,T) S3: movetotable(E,F) C Sinit A D B Sfinal B F A E D C E F 2.1 Centralized planning for distributed plans • Operators • move(b,x,y) move b from x to y  movetotable(b,x) Precond: on(b,x) clear(b) clear(y) Precond: on(b,x)  clear(b) Postcond: on(b,y) clear(x) Postcond: ob(b,T)  clear(x)  on(b,x) on(b,x)clear(y) work backward from each “on” goal I'm Tom Agent2 I'm Bill Agent1 on(A,B) on(C,D) on(E,F) on(B,T) on(D,T) on(F,T) on(B,A) on(F,D) on(A,E) on(D,C) on(E,T) on(C,T) 1. Given a goal description, a set of operators, and an initial state description generate a partial order plan Slides from Sobah Abbas Peterson 43

44. S1: move(B,T,A) To satisfy the preconditions, we have: S2: move(A,B,E) S2 < S1, S3 < S4 S3:movetotable(E,F) S6 < S4, S6 < S5 S4: move(F,T,D) Also S5: move(D,T,C) S2 threat to S3  S3 < S2 S6: movetotable(C,D) S4 threat to S5  S5 < S4 Then the partial ordering is: S3 < S2 < S1 S6 < S5 < S4 S3 < S4 Any total ordering that satisfies this partial ordering is a good plan for Agent1 2. Decompose the plan into sub problems so as to minimize order relations across plans 3. Insert synchronization 4. Allocate sub plans to agents Slides from Sobah Abbas Peterson 44

45. What if we have 2 agents? DECOMP1 Subplan1S3 < S2 < S1 Subplan2 S6 < S5 < S4 and S3 < S4 Agent1 S3 < send(clear(F)) < S2 < S1 Agent2 S6 < S5 < wait(clear(F)) < S4 Slides from Sobah Abbas Peterson

46. < < S3: movetotable(E,F) S2: move(A,B,E) S1: move(B,T,A) S6: movetotable(C,D) S5: move(D,T,C) S4: move(F,T,D) DECOMP2 Subplan1S3 < S5 < S4 Subplan2 S6 < S2 < S1 and S3 < S2 and S6 < S5 Agent1 S3 < send(don't_care(E)) < wait(clear(D)) < S5 < S4 Agent2 S6 < wait(don't_care(E)) < wait(clear(D)) < S2 < S1 • Obviously, DECOMP2 has more order relations among sub plans than DECOMP1 • Therefore, we choose DECOMP1 S3 < send(clear(F)) < S2 < S1 S6 < S5 < wait(clear(F)) < S4 But then back to DECOMP2 4. If failure to allocate sub plans then redo decomposition (2) If failure to allocate sub plans with any decomposition then redo generate plan (1) 5. Execute and monitor sub plans I know how to move only D, E, F I know how to move only A, B, C Slides from Sobah Abbas Peterson 46

47. 2.2 Distributed planning for centralized plans • generate separate plans, then merge • parallel result sharing • may involve negotiation Agent 1 - is specialized in doing movetotable(b,x) Agent 2 - is specialized in doing move(b,x,y) PAgent1 = { S3: movetotable(E,F) satisfies on(E,T) S6:movetotable(C,D) satisfies on(C,T) no ordering } PAgent 2 = { S1: move(B,T,A), S2: move(A,B,E) satisfies on(B,A)  on(A,E) S4: move(F,T,D), S5: move(D,T,C) satisfies on(F,D)  on(D,C) ordering S2 < S1 and S5 < S4 } • Merge PAgent1 with PAgent2 by checking preconditions and threats • S3 < S2, S6 < S5, S3 < S4, S2 < S1 and S5 < S4 • one agent executes (as is centralized) Slides from Sobah Abbas Peterson 47

48. C Si A D B Sf B F A E D C E D A F E C F B • The problem is decomposed , given to specialize • similar to task sharing • may involve backtracking Agent 1 - knows only how to deal with 2-block stacks Agent 2 - knows only how to deal with 3-block stacks Slides from Sobah Abbas Peterson 48

49. 2.3 Distributed planning for distributed plans a) Plan merging • How much effort on coordinating issues? • Agents formulate local plans to satisfy their goals • Local plans are exchanged • Local plans are combined analyzing for positive and negative interaction • Add messages and/or timing commitments to resolve negative plan interactions and to exploit positive plan interactions Interacting situations • Positive interactions between plans • redundant actions • beneficial actions • Negative interactions between plans • harmful actions • exclusive actions • incompatible actions Slides from Sobah Abbas Peterson 49

50. Si E Sf E C B D F A F A B C D C A B B Negative interactions what type? • movehigh(b,x,y) Precond: have_lifter  clear(b)  clear(y)  on(y,z)  z T Postcond: on(b,y)  clear(x)  on(b,x)   clear(y)  free_lifter • pick_lifter Precond: free_lifter Postcond: have_lifter  free_lifter Agent1: { S1:move(B,T,A) < S2: pick_lifter < S3: movehigh(E,T,B) } Agent2: { R1:move(C,T,D) < R2: pick_lifter < R3: movehigh(F,T,C) } if both select same lifter R1 S1 need_l S2 S3 Sf1 free_l R2 R3 Slides from Sobah Abbas Peterson 50