1 / 36

Models of agency and architectures Lecture outline

Multi-Agent Systems Lecture 2 University “Politehnica” of Bucarest 2004 - 2005 Adina Magda Florea adina@cs.pub.ro http://turing.cs.pub.ro/blia_2005. Models of agency and architectures Lecture outline. Conceptual structures of agents Cognitive agent architectures Reactive agent architectures

afya
Download Presentation

Models of agency and architectures Lecture outline

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Multi-Agent SystemsLecture 2University “Politehnica” of Bucarest2004 - 2005Adina Magda Floreaadina@cs.pub.rohttp://turing.cs.pub.ro/blia_2005

  2. Models of agency and architecturesLecture outline • Conceptual structures of agents • Cognitive agent architectures • Reactive agent architectures • Layered architectures

  3. 1. Conceptual structures of agents 1.1 Agent rationality • An agent is said to be rational if it acts so as to obtain the best results when achieving the tasks assigned to it. • How can we measure the agent’s rationality? • A measure of performance, an objective measure if possible, associated to the tasks the agent has to execute. 3

  4. An agent is situated in an environment • An agent perceives its environment through sensors and acts upon the environment through effectors. • Aim: design an agent program = a function that implements the agent mapping from percepts to actions. We assume that this program will run on some computing device which we will call the architecture. Agent = architecture + program • The environment • accessible vs. inaccessible • deterministic vs. non-deterministic • static vs. dynamic • discrete vs. continue 4

  5. 1.2 Agent modeling E={e1, .., e, ..} P ={p1, .., p, ..} A ={a1, .., a, ..} Reflex agent see : E  P action : P  A env : E x A  E (env : E x A P(E)) Decision component Agent action Perception Execution component component see action Environment env 5

  6. Interaction component inter Agent modeling I = {i1,…,i,..} Several reflex agents see : E  P env : E x A1 x … AnP(E) inter : P  I action : P x I  A Agent (A1) Decision component action Agent (A2) Execution Perception Agent (A3) component component action see Environment env 6

  7. Agent modeling Cognitive agents Agents with statesS = {s1,…,s,…} • action : S x I Ai • next : S x P  S • inter : S x P  I • see : E  P • env : E x A1 x … AnP(E) 7

  8. Agent modeling Agents with states and goals goal : E  {0, 1} Agents with utility utility : E  R Environment non-deterministic env : E x A P(E) The probability estimated by the agent that the result of an action (a) execution in state e will be the new state e’ 8

  9. Agent modeling Agents with utility The expected utility of an action in a state e, from the agent’s point of view The principle of Maximum Expected Utility (MEU) = a rational agent must choose the action that will bring the maximum expected utility 9

  10. How to model? • Getting out of a maze • Reflex agent • Cognitive agent • Cognitive agent with utility 3 main problems: • what action to choose if several available • what to do if the outcomes of an action are not known • how to cope with changes in the environment 10

  11. 2. Cognitive agent architectures 2.1 Rational behaviour AI and Decision theory • AI = models of searching the space of possible actions to compute some sequence of actions that will achieve a particular goal • Decision theory = competing alternatives are taken as given, and the problem is to weight these alternatives and decide on one of them (means-end analysis is implicit in the specification of competing alternatives) • Problem 1 = deliberation/decision vs. action/proactivity • Problem 2 = the agents are resource bounded 11

  12. Control Interactions Information about itself - what it knows - what it believes - what is able to do - how it is able to do - what it wants environment and other agents - knowledge - beliefs Communication General cognitive agent architecture Reasoner Other agents Planner Scheduler& Executor Output State Input Environment 12

  13. 2.2 FOPL models of agency • Symbolic representation of knowledge + use inferences in FOPL - deduction or theorem proving to determine what actions to execute • Declarative problem solving approach - agent behavior represented as a theory T which can be viewed as an executable specification (a) Deduction rules At(0,0)  Free(0,1)  Exit(east)  Do(move_east) Facts and rules about the environment At(0,0) Wall(1,1) x y Wall(x,y)  Free(x,y) Automatically update current state and test for the goal state At(0,3) 13

  14. FOPL models of agency (b) Use situation calculus = describe change in FOPL • Situation = the state resulting after executing an action Logical terms consisting of the initial state S0 and all situations that are generated by applying an action to a situation Result(Action,State) = NewState • Fluents = functions or predicates that vary from one situation to the next At(location, situation) 14

  15. FOPL models of agency At((0,0), S0)  Free(0,1)  Exit(east)  At((0,1), Result(move_east,S0)) Try to prove the goal At((0,3), _) and determine the actions that lead to it - means-end analysis KB -| {Goal} and keep track o associated actions 15

  16. Advantages of FOPL - simple, elegant - executable specifications Disadvantages - difficult to represent changes over time other logics - decision making is deduction and selection of a strategy - intractable - semi-decidable 16

  17. 2.3 BDI architectures • High-level specifications of a practical component of an architecture for a resource-bounded agent. • It performs means-end analysis, weighting of competing alternatives and interactions between these two forms of reasoning • Beliefs = information the agent has about the world • Desires = state of affairs that the agent would wish to bring about • Intentions = desires (or actions) that the agent has committed to achieve • BDI - a theory of practical reasoning - Bratman, 1988 • intentions play a critical role in practical reasoning - limits options, DM simpler 17

  18. BDI particularly compelling because: • philosophical component - based on a theory of rational actions in humans • software architecture - it has been implemented and successfully used in a number of complex fielded applications • IRMA - Intelligent Resource-bounded Machine Architecture • PRS - Procedural Reasoning System • logical component - the model has been rigorously formalized in a family of BDI logics • Rao & Georgeff, Wooldrige • (Int Ai )   (Bel Ai) 18

  19. percepts BDI Architecture Belief revision Beliefs Knowledge B = brf(B, p) Opportunity analyzer Deliberation process Desires D = options(B,D, I) Intentions Filter Means-end reasonner I = filter(B, D, I) Intentions structured in partial plans  = plan(B, I) Library of plans Plans Executor 19 actions

  20. Roles and properties of intentions • Intentions drive means-end analysis • Intentions constraint future deliberation • Intentions persist • Intentions influence beliefs upon which future practical reasoning is based Agent control loop B = B0 I = I0D = D0 while true do get next perceipt p B = brf(B,p) D = options(B, D, I) I = filter(B, D, I)  = plan(B, I) execute() end while 20

  21. Commitment strategies • If an option has successfully passed trough the filter function and is chosen by the agent as an intention, we say that the agent has made a commitment to that option • Commitments implies temporal persistence of intentions; once an intention is adopted, it should not be immediately dropped out. Question: How committed an agent should be to its intentions? • Blind commitment • Single minded commitment • Open minded commitment Note that the agent is committed to both ends and means. 21

  22. Revised BDI agent control loop single-minded commitment B = B0 I = I0 D = D0 while true do get next perceipt p B = brf(B,p) D = options(B, D, I) I = filter(B, D, I)  = plan(B, I) while not (empty() or succeeded (I, B) or impossible(I, B)) do  = head() execute()  = tail() get next perceipt p B = brf(B,p) if not sound(, I, B) then  = plan(B, I) end while end while Dropping intentions that are impossible or have succeeded Reactivity, replan 22

  23. Revised BDI agent control loop open-minded commitment B = B0 I = I0 D = D0 while true do get next perceipt p B = brf(B,p) D = options(B, D, I) I = filter(B, D, I)  = plan(B, I) while not (empty() or succeeded (I, B) or impossible(I, B)) do  = head() execute()  = tail() get next perceipt p B = brf(B,p) D = options(B, D, I) I = filter(B, D, I)  = plan(B, I) end while end while ifreconsider(I,B) then Replan 23

  24. 3. Reactive agent architectures Subsumption architecture - Brooks, 1986 • (1) Decision making = {Task Accomplishing Behaviours} • Each behaviour = a function to perform an action • Brooks defines TAB as finite state machines • Many implementations: situation action • (2) Many behaviours can fire simultaneously 24

  25. Subsumption architecture • A TAB is represented by a competence module (c.m.) • Every c.m. is responsible for a clearly defined, but not particular complex task - concrete behavior • The c.m. are operating in parallel • Lower layers in the hierarchy have higher priority and are able to inhibit operations of higher layers • c.m. at the lower end of the hierarchy - basic, primitive tasks; • c.m. at higher levels - more complex patterns of behaviour and incorporate a subset of the tasks of the subordinate modules  subsumtion architecture 25

  26. Competence Module (2) Explore environ Effectors Sensors Output (actions) Input (percepts) Competence Module (1) Move around Competence Module (0) Avoid obstacles Module 1 can monitor and influence the inputs and outputs of Module 2 M1 = move around while avoiding obstacles M0 M2 = explores the environment looking for distant objects of interests while moving around  M1 • Incorporating the functionality of a subordinated c.m. by a higher module is performed using suppressors (modify input signals) and inhibitors (inhibit output) Competence Module (1) Move around Supressor node Inhibitor node Competence Module (0) Avoid obstacles 26

  27. More modules can be added: • Replenishing energy • Optimising paths • Making a map of territory • Pick up and put down objects Behavior (c, a) – pair of condition-action describing behavior R = { (c, a) | c  P, a  A} - set of behavior rules   R x R - binary inhibition relation on the set of behaviors, total ordering of R function action( p: P) var fired: P(R), selected: A begin fired = {(c, a) | (c, a)  R and p  c} for each (c, a)  fired do if   (c', a')  fired such that (c', a')  (c, a) then return a return null end 27

  28. Every c.m. is described using a subsumption language based on AFSM - Augmented Finite State Machines • An AFSM initiates a response as soon as its input signal exceeds a specific threshold value. • Every AFSM operates independently and asynchronously of other AFSMs and is in continuos competition with the other c.m. for the control of the agent - real distributed internal control • 1990 - Brooks extends the architecture to cope with a large number of c.m. - Behavior Language Other implementations of reactive architectures • Steels - indirect communication - takes into account the social feature of agents • Advantages of reactive architectures • Disadvantages 28

  29. Layer n Layer n Layer n … … … Layer 2 Layer 2 Layer 2 Layer 1 Layer 1 Layer 1 4. Layered agent architectures • Combine reactive and pro-active behavior • At least two layers, for each type of behavior • Horizontal layering - i/o flows horizontally • Vertical layering - i/o flows vertically Action output Action output Action output perceptual input Vertical Horizontal perceptual input perceptual input 29

  30. TouringMachine • Horizontal layering - 3 activity producing layers, each layer produces suggestions for actions to be performed • reactive layer - set of situation-action rules, react to precepts from the environment • planning layer - pro-active behavior - uses a library of plan skeletons called schemas - hierarchical structured plans refined in this layer • modeling layer - represents the world, the agent and other agents - set up goals, predicts conflicts - goals are given to the planning layer to be achieved • Control subsystem - centralized component, contains a set of control rules - the rules: suppress info from a lower layer to give control to a higher one - censor actions of layers, so as to control which layer will do the actions 30

  31. InteRRaP • Vertically layered two pass agent architecture • Based on a BDI concept but concentrates on the dynamic control process of the agent Design principles • the three layered architecture describes the agent using various degrees of abstraction and complexity • both the control process and the KBs are multi-layered • the control process is bottom-up, that is a layer receives control over a process only when this exceeds the capabilities of the layer beyond • every layer uses the operations primitives of the lower layer to achieve its goals Every control layer consists of two modules: - situation recognition / goal activation module (SG) - planning / scheduling module (PS) 31

  32. Local planning layer Cooperative planning layer Behavior based layer PS SG PS SG SG PS Social KB I n t e R R a P Planning KB World KB World interface Sensors Effectors Communication 32 actions percepts

  33. Options Operational primitive Intentions Situation Goals Beliefs Cooperative situation Joint plans Cooperative goals Cooperative intents Social model Cooperative option Local plans Local goals Local option Local planning situation Local intentions Mental model Routine/emergency sit. Reactions World model Response Behavior patterns Reaction BDI model in InteRRaP options Sensors filter SG Effectors plan PS 33

  34. Muller tested InteRRaP in a simulated loading area. • A number of agents act as automatic fork-lifts that move in the loading area, remove and replace stock from various storage bays, and so compete with other agents for resources 34

  35. BDI Architectures • First implementation of a BDI architecture: IRMA • [Bratman, Israel, Pollack, 1988] M.E. BRATMAN, D.J. ISRAEL et M. E. POLLACK. Plans and resource-bounded practical reasoning, Computational Intelligence, Vol. 4, No. 4, 1988, p.349-355. • PRS • [Georgeff, Ingrand, 1989] M. P. GEORGEFF et F. F. INGRAND. Decision-making in an embedded reasoning system, dans Proceedings of the Eleventh International Joint Conference on Artificial Intelligence (IJCAI-89), 1989, p.972-978. • Successor of PRS: dMARS • [D'Inverno, 1997] M. D'INVERNO et al. A formal specification of dMARS, dans Intelligent Agents IV, A. Rao, M.P. Singh et M. Wooldrige (eds), LNAI Volume 1365, Springer-Verlag, 1997, p.155-176. • Subsumption architecture • [Brooks, 1991] R. A. BROOKS. Intelligence without reasoning, dans Actes de 12th International Joint Conference on Artificial Intelligence (IJCAI-91), 1991, p.569-595. 35

  36. TuringMachine • [Ferguson, 1992] I. A. FERGUSON. TuringMachines: An Architecture for Dynamic, Rational, Mobile Agents, Thèse de doctorat, University of Cambridge, UK, 1992. • InteRRaP • [Muller, 1997] J. MULLER. A cooperation model for autonomous agents, dans Intelligent Agents III, LNAI Volume 1193, J.P. Muller, M. Wooldrige et N.R. Jennings (eds), Springer-Verlag, 1997, p.245-260. BDI Implementations The Agent Oriented Software Group • Third generation BDI agent system using a component based approached. Implemented in Java • http://www.agent-software.com.au/shared/home/ JASON • http://jason.sourceforge.net/ 36

More Related