Topic 2: Multi-Agent Systems

1. Topic 2: Multi-Agent Systems a practical example categories of MAS examples definitions: agents and MAS conclusion

2. Think about this … • a practical exercise: “re-arranging entities in a constrained world” • problem domain: • Constrained world • Entities • Position • Mobile • Formation - new position

4. Think about this … • a practical exercise: “re-arranging entities in a constrained world” • problem domain: • Constrained world • Entities • Position • Mobile • Formation - new position • solution ?

5. Think about this … (cont.) collect 1. a centralized solution • how ? • one “overall planner” • everyone gives his information (initial and target position)to the planner • the planner calculates a plan • the planner: instructs individual entities (you move to position X, …) or distributes individual plans plan re-distribute • disadvantages ? • bottleneck • single-point-of-failure • tractable? • who is the planner? • what global knowledge does planner need? #entities, start of protocol, … • advantages ? • simple • easy to maintain • planning quite well-known

6. Think about this … (cont.) 2. a distributed solution • how ? • different tasks in re-arranging e.g. per group of entities • one “planner” per task/group • one overall task manager (coordinates tasks) • collection + redistribution (per task) • … … • advantages ? • more scalable (a bit) • easy distribution … • disadvantages ? • bottleneck / scalability ? • many-points-of-failure • how to designate the planners? • what global knowledge does this require the sorters to have ? #entities, groups, start of protocol, … ?

7. Think about this … (cont.) 3. a distributed, functional decomposition solution • how ? • assign experts collector planner distributor the experts collector planner distributor • advantages ? • more scalable (a bit) • easy distribution • clear roles / responsibilities • disadvantages ? • bottleneck / scalability? • many-points-of-failure • how to designate the experts? • what global knowledge do experts need? #entities, start of protocol, …?

8. Think about this … (cont.) • ok, but what if … • #entities is not known …? • entities may enter or leave the system at all times…? • target positions can be changed during execution? • frequency of change > time to (re)plan? • we do this for 10.000s of entities? • what if a movement did not happen the way it was supposed to? • in general, what if it is not a “single shot application”, but a “going concern” ? • … • it’s a tough world …

9. Think about this … (cont.) 4. an enhanced distributed, functional decomposition solution • how ? • make the experts more intelligent • monitor the system for change • use knowledge to tackle changeefficiently (collecting/planning/distributing) the experts collector planner distributor • disadvantages ? • scalability • more complex experts • global knowledge • extra advantages ? • more flexible to change • awareness of failures

10. Think about this … (cont.) 5. a decentralized solution • how ? • every entity tries to force its wayto target position • local conflict resolution to avoiddeadlocks/starvation • advantages ? • simple (if robots: low cost) • easy distribution • scalable • no single-point of failure • only local actions • only local knowledge (neighbours, order) • disadvantages ? • no central view ==> not for one-shot applications • no hard guarantees?

11. Think about this … (cont.) • ok, but what if … • #entities is not known …? OK • entities may enter or leave the system at all times…? OK • target positions can be changed during execution? OK • frequency of change > time to (re)plan? OK • we do this for 10.000s of entities? OK • in general, what if it is not a “single shot application”,but a “going concern” ? OK • … • it’s a tough world …

12. What’s the point …? • Problems may have many solutions… • solutions have qualities • pick the appropriate solution for the set of required qualities • This course is about… • distributed problems in a dynamic environment • with requirements for • flexibility • adaptability • scalability • …

13. What is a MAS ? • “in essence, a MAS is a philosophy to model systems” • indicating how to solve a problem … • in a complex world … • with autonomous entities … • solutions of example • centralized • distributed • distributed experts • enhanced distributed experts • decentralized multi-agent systems

14. What is a MAS ? Categories of MAS … 1. distributed “experts” • enhanced distributed entities • cooperating entities • mainly functional decomposition • focus on detailed modeling of individual agent • mental states • knowledge representation • planning 2. decentralized systems • collective behaviour • local actions/interactions only • focus on the collection and the environment

15. Application domains • distributed control applications • e.g. AGVs, robots, virtual entities, trafic, logistics • simulation • pure algorithms • …

16. Example: a distributed control system - 1 • Control system for automated warehouse management • Egemin N.V. • AGVs (automated guided vehicles) • control system • the world: • large • dynamic • tasks • AGV failure • batteries

17. e.g. Vredestein tyres (NL) • centralized solution ? • towards a distributed solution in a tough world • distributed  enhanced distributed system (cat. 1) • distributed  decentralized system (cat. 2)

18. Example: a distributed control system - 2 • Reactive Bubbles • moving entities • constraint environment • what if … • massively distributed !? • no global knowledge !? • new bubbles / bubbles disappear !? • environment changes !? • is it efficient !? • efficient ? NO ! • but fascinating! • and a solution of our problem in a tough world …

19. a a a Example: Collective Robotics • Collective robotics • cooperative parts of one robot • e.g. several servos for a single robot arm • multiple cooperating robots • coordinate actions • to accomplish common task

20. Example: MAS – simulation • Goal: study phenomena in • physics • ecology • biology • chemistry • social sciences • geography • … • Approaches • mathematical: relationships of variables • (differential) equations, transition matrices, … • MAS • model individual entities / environment / …

21. MAS – simulation (cont.) • MAS simulations • model individuals’ • behaviour • actions • interactions • … • model environment • advantages: miniature laboratory • simulation model close to real-world entities • allows to study consequence of individual behaviour • reasoning process can be included • …

22. Example: algorithms • edge detection / constraint satisfaction problems / data mining

23. Why use MAS for such systems ? • clean software abstraction for modelling such systems • autonomy • cooperation • agents are (should be) critical by nature • do not rely on “anything” • should be “designed to be flexible” • agents are (should be) adaptive by nature • change their behaviour in this highly dynamic world … • MAS solves problems! • but also introduces new challenges… • communication • coherent behaviour • control • MAS is NOT a holy grail … !! • MAS is NOT suitable for all applications … !!

24. MAS: definition … • A multi-agent system is a system consisting of multiple autonomous entities, called agents, which are situated in an environment that the agents can partially observe and in which they can act and cooperate to achieve system objectives.

25. MAS: definition … • … requires a definition of … • agents • actions • production/consumption/manipulation of objects • perception • an environment • objects in the environment • objectives

26. MAS environment • properties • accessible vs. inaccessible • accessible = can provide complete, up-to-date information about the (entire) environment state • the more “accessible”, the easier the system • most real-world systems: inaccessible state • deterministic vs. non-deterministic • determinism w.r.t. result of agent actions • e.g. up (B,C)  ?? • most real-world systems: non-deterministic environment • agents do not have full control • actions can fail • …

27. MAS environment (cont.) • … • static vs. dynamic • static: does not change between two actions (e.g. planning algorithms) • most real-world systems: dynamic environment • under constant change objects in the environment change environment changes concurrent / autonomous agents  concurrent actions • e.g. software environment real-world environment: • discrete vs. continuous • as in “number of states”

28. agent observation action environment Agents J. Ferber • An agent is physical or virtual entity • capable of perceiving its environment • capable of acting in an environment (not just reasoning) • capable of communicating • driven by goals • possessing resources • having partial representation of the environment • having particular capabilities (skills and services) • acts towards its objectives

29. an agent • properties • reactivity reacts to stimuli (changes in env., communication, …) • autonomy does not require user interaction • proactive-ness aims to achieve its own goals, therefore initiates appropriate actions • social ability cooperates / coordinates / communicates / … • embodied situated in the environment • mobile moves around network sites • learning learn from past experiences • … essential extra

30. MAS: agent actions • what is an agent action ? “an attempt … … to bring about a state of affairs … … in the environment … … or another agent” • non-deterministic • did anything happen ? • did happen what agent expected ? • did something else happen ? • did the message arrive ? • … • this is not a limitation to make things hard …it’s a fact of life … ==> agents need to be flexible / adaptive / …

31. MAS: objects in environment • objects in the environment • passive objects • e.g. pallets, files, … • dynamic objects • e.g. rolling ball, stones cooling down • agents

32. MAS objectives / behaviour • objectives • system objective • e.g. find a solution to a constraint problem • e.g. “ongoing concern” • e.g. continuous schedule of product manufacturing • e.g. ant colony survival • e.g. network management • individual agent objective • directly related to system objective • e.g. efficient expert service • emergent behaviour • system objective “emerges” from achievements of individual objectives • e.g. ant foraging

33. MAS: definition … • A multi-agent system is a system consisting of multiple autonomous entities, called agents, which are situated in an environment that the agents can partially observe and in which they can act and cooperate to achieve system objectives.

34. communicate cooperate coordinate adapt own behaviour! negotiate … Key challenge of MAS imagine that you have to develop software which will work in an environment … • that you can only partially observe • where the results of what you do are not guaranteed • where the environment constantly changes by other activities • which can achieve to perform its objective

35. ConclusionMAS: Issues and Challenges • How do we decompose problems into behaviour for individual agents ? • How do we ensure agents act coherently in making decision or taking action ? • do local actions have harmful global effects • avoiding unstable system behaviour • How to enable agents to communicate and interact ? • communication languages and protocols • interoperation of heterogeneous agents • finding useful existing agents in open environments • How does agent decide what to do ? • action selection mechanisms • How do we build agents ? • actions, plans, and knowledge • coordination actions • How do we compromise different views and conflicting goals of agents trying to coordinate ? • How do we engineer multiagent systems ?

36. Conclusion • MAS is about • software engineering distributed applications • engineering • “synergy” of different techniques/philosophies/applications/… • distributed systems • software engineering • robot control • languages for programming MAS • AI • … • AI is a subfield of MAS “Agents are 99% computer science, and 1% AI.”