Introduction to Multiagent Systems

1. Introduction to Multiagent Systems MohsenAfsharchi

2. An Agent • An agent is anything that can perceive its environment through sensors and act upon that environment through actuators • An agent that always tries to optimize an appropriate performance measure is called a ‘rational agent’. • Human agents (eyes as sensors, hands as actuators) • Robotic agents (cameras as sensors, wheels as actuators) • Software agents (a graphical user interface as sensor and as actuator)

3. Multiagent Systems • A system that consists of a group of agents that can potentially interact with each other. • The corresponding subfield of AI that deals with principles and design of multiagent systems is called distributed AI. • Software agents on the Internet • Soccer playing robots

4. Characteristics of MASs • Agent Design • Heterogeneous, different hardware or implement different behaviors. • Homogeneous, designed in an identical way and have a priori the same capabilities. • Environment • Static, easier to handle and allow for a more rigorous mathematical treatment. • Dynamic, presence of multiple agents makes the environment appear dynamic

5. Characteristics… • Deterministic vs. Non-deterministic, Do actions have guaranteed effect? • Discrete vs Continuous, Is the number of actions and percepts finite? (Does it matter?) • Perception • Distributed, Spatially ; Temporally; Semantically • Observable, Partially or Full • Sensor fusion

6. Characteristics… • Control • Typically Decentralized. • General problem of multiagent decision making is the subject of Game theory • Collaborative or Team MAS where the agents share the same interests • Distributed decision making needs appropriate Coordination mechanisms

7. Characteristics… • Communication • for coordination among cooperative agents • for negotiation among self-interested agents • Ontology • Language • Knowledge • Cooperative agents • Competitive agents • Common knowledge

8. Application • Speedup and efficiency, due to the asynchronous and parallel computation. • Robustness and reliability, in the sense that the whole system can undergo a ‘graceful degradation’ when one or more agents fail. • Scalability and flexibility, since it is easy to add new agents to the system. • Cost, assuming that an agent is a low-cost unit compared to the whole system. • Development and reusability, since it is easier to develop and maintain a modular system than a monolithic one.

9. Applications • The Semantic Web; Web services are just the start. • Automated manufacturing. [Baker et al., 1999] [Huhns et al., 2002] • Distributed sensor fusion. [Vidal, 2003] [Scerri et al., 2002] [Ortiz and Rauenbusch, 2002] • Dynamic resource allocation (who does what). [Mailler et al., 2003] • Supply-chain automation. [Singh and Huhns, 1999] • Coordination of robots: Mars colonization. [Pirjanian et al., 2000] • Any application where independent localized units must coordinate to achieve some desired system behavior.

10. Challenging Issues • How to decompose a problem, allocate subtasks to agents, and synthesize partial results. • How to handle the distributed perceptual information. • How to enable agents to maintain consistent shared models of the world. • How to implement decentralized control and build efficient coordination mechanisms among agents. • How to design efficient multiagent planning and learning algorithms.

11. Challenging Issues • How to represent knowledge. How to enable agents to reason about the actions, plans, and knowledge of other agents. • How to enable agents to communicate. What communication languages and protocols to use. What, when, and with whom should an agent communicate. • How to enable agents to negotiate and resolve conflicts. • How to enable agents to form organizational structures like teams or coalitions. How to assign roles to agents. • How to ensure coherent and stable system behavior.

12. An Agent: Abstract Architecture • c

13. Agent Model • I

14. Perception • a

15. Agents with State • a

16. Rational Decision Making • Assuming a discrete set of time in each of which the agent must choose an action from a set of actions • To optimally choose an action at time , it must use its complete history of observation and action for • Mapping function is called the Policy of the agent • An agent that is simply ignoring the past is called a reflect agent and its policy is called reactive or memory-less • in a Markov world an agent can safely use the memory-less policy for its decision making, in place of the memory-expensive policy.

17. State and Observability • The collective information that is contained in the world at any time step , is called a state of the world and denoted by • Discrete world : characterized by a finite number of states, • Continuous world  • Observable world  • Partially Observable world  (stochastic observation model)

18. Stochastic World… • More on Observability • Noise : e.g. Sensor malfunction • Perceptual Aliasing : different states may produce identical observations to the agent at different time steps. In other words, two states may ‘look’ the same to an agent, although the states are different from each other. • Deterministicworld :  • Stochastic world : 

19. Utility Maximizing Agents • A goal is just a preference over certain states. • Utility function U(s) is the utility of state for the agent. • The agent in should take the action which maximizes its expected utility • The function that implements this choice is the policy. In this case • Optimal Action-Value function

20. Example • A toy world • Unlike path planning in a deterministic world that can be described as graph search, decision making in stochastic domains requires computing a complete policy that maps states to actions. (Reinforcement Learning)

21. Deductive Reasoning Agent • Counterpoint: Rodney Brooks believes that the world should be its own model

22. Agents as Theorem Prover • w

23. Action Selection Function

24. Pragmatics • Building a purely logical agent is impractical. • Proving theorems in first-order predicate logic is slow. • Late actions are based on old information. • But, webservice description languages are built to be used by logical agents. There might be a new renaissance of logical approaches. • For example, OWL-S uses service profiles which define services in terms of their Inputs, Outputs, Pre-conditions.

25. Agent Oriented Programming • Program agents in terms of mentalistic notions. Pre-cursor to a lot of important work in agent research. • The hope is that using these abstractions would simplify the programming of agents. • Introduced by YoavShoham in 1990. • The idea was then implemented as AGENT0 [Shoham, 1991]. Not used anymore.

26. AOP Primitives • An agent has • Capabilities things it can do. • Beliefs • Commitments things it means to do. • Commitment rules that tell it when to create or drop a commitment. • The commitment rules have a message condition and a mental condition (both in the conditional part). • An agent can take a private action which amounts to running a subroutine, or a communicative action which amounts to sending a message. • Messages are limited to: requests, unrequests, and inform messages.

27. Agent0 Example • I • If I receive a message from agent which requests for me to do action at time and I believe that • agent is a friend. • I can do action at time, • I am not committed to doing any other action. • then commit to doing action at time.

28. Going the Wrong Direction? • Deduction is slow. • Deduction is complicated. • Maybe, rationality requires an environment! • Maybe, intelligent behavior emerges from the interaction of simple behaviors [Minsky, 1988]. • Elephants Do not Play Chess !! But?

29. What is Intelligence? • Is a house fly intelligent? • A house fly is much simpler than most of our attempts at artificial intelligence • For example… • It is unlikely that a house fly: • Forms 3D surface descriptions of objects • Reasons about the threat of a human with a fly swatter, in particular about the human’s beliefs, goals, or plans • Makes analogies concerning the suitability for egg laying between dead pigs • Constructs naïve physics theories of how to land on the ceiling

30. Real Intelligence!!! • It is much more likely that a house fly: • Has close connection of sensors to actuators • Has pre-wired patterns of behavior • Has simple navigation techniques • Functions almost as a deterministic machine • And yet a house fly is much more successful in the real world than our attempts at AI

31. Hey Rodney! • Proposed by Brooks in [Brooks, 1986]: for robots. • Extended it into a new view of AI,[Brooks, 1991a, Brooks, 1991b]. • Key ideas: • Intelligent behavior does not require explicit representations. • Intelligent behavior does not require abstract (symbolic) reasoning. • Intelligence is an emergent property of certain complex systems.

32. A New Architecture • Follow the evolutionary path of intelligence • Start with simple intelligence • Easier to implement than human intelligence • After a successful design, extend to higher levels of intelligence • Robust design as higher intelligence levels can fail but the lower levels will still work • After all, there are plenty of examples of successful intelligence in nature that are much simpler than many AI research areas (the house fly example)

33. The Subsumption Architecture • The Subsumption Architecture is: • A layering methodology for robot control systems • A parallel and distributed method for connecting sensors and actuators in robots

34. An Example: Allen • Brooks’ first Subsumption robot • Level 0: Runs away if approached, avoids objects

35. Allen… • Levels 1 and 0: Adds wandering

36. Subsumption Implementation • The agent has a set of behaviors which are purely reactive (action: S  A). • Many of them can fire at the same time. • We choose an action using the subsumption hierarchy. • Behaviors are arranged into layers, so that lower layers inhibit higher one. The lower layers have the higher priority. • Formally, • a

37. Action Selection

38. Limitations • Agents must have enough information in local environment to determine which action to take. • How to take into account old or non-local information? • How do reactive! agents learn? • Emergence (between agent and environment) is hard to engineer. We don’t have a methodology. • It is very hard to build agents that have many behaviors. • On the plus side, it has been a very successful method for building robots, such as Roomba.