1 / 93

Agent Technology

Agent Technology. Agent architectures. Content. Abstract architectures for intelligent agents Concrete architectures for intelligent agents Logic-based architectures Reactive architectures Decision-theoretic architectures Belief-desire-intention architectures Layered architectures

niloufer
Download Presentation

Agent Technology

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Agent Technology Agent architectures

  2. Content • Abstract architectures for intelligent agents • Concrete architectures for intelligent agents • Logic-based architectures • Reactive architectures • Decision-theoretic architectures • Belief-desire-intention architectures • Layered architectures • Summary ©Gao Yang, Ai Lab NJU

  3. What is architectures? • Represent the move from specification to implement • How to construct computer systems that satisfy the properties specified by agent theory • What software structures are appropriate ©Gao Yang, Ai Lab NJU

  4. Definitions of agent architectures • Pattie Maes • A particularmethodologyfor building agents. • It specifies how the agent can be decomposedinto the construction of the set of component modules and how this modules should be made to interact. • An architectureencompasses techniques and algorithmsthat support this methodology. ©Gao Yang, Ai Lab NJU

  5. Definitions of agent architectures • Kaelbling • A specific collection of software (or hardware) modules, typically designated by the boxes with arrows with arrows indicating the data and control flow among the modules. • A more abstract view of architecture is as a general methodology for designing particular modular decompositions for particular tasks. ©Gao Yang, Ai Lab NJU

  6. Abstract architectures: overview • Environment states • Actions • AgentFunction • Action: • This model of an agent as a function will be called a standard agent. • A standard agent decides what action to perform based on its history, i.e, its experiences to date. ©Gao Yang, Ai Lab NJU

  7. Abstract architectures: behavior • Behavior can be modeled as • means powerset or set of all subsets. • If • Environment is non-deterministic. • If • Environment is deterministic and behavior can be accurately predicted. ©Gao Yang, Ai Lab NJU

  8. Abstract architectures: history • Present the interaction between agent and environment as a history, i.e., the sequence of state-action pairs. • Where s0 is the initial state and ai is the action the agent performs when it is in the state si. ©Gao Yang, Ai Lab NJU

  9. Abstract architectures: history • For any h to be a possible history of an agent starting from initial state s0, all action are generated from state sequence. • Every new state must belong to the set of possible environmental states reachable from the previous state by applying the selected action. ©Gao Yang, Ai Lab NJU

  10. Abstract architectures: Purely reactive • Purely reactive agents decide what to do without reference to their history. • Formally, ©Gao Yang, Ai Lab NJU

  11. Abstract architectures: Perception • The first, separate of an agent’s decision function into perception and action subsystems. ©Gao Yang, Ai Lab NJU

  12. Abstract architectures: Perception • Formally, • But two different environment states are mapped to the same percept ©Gao Yang, Ai Lab NJU

  13. Abstract architectures: Internal state • Secondly, more natural scheme but equivalent represent history is the internal state. ©Gao Yang, Ai Lab NJU

  14. Abstract architectures: Internal state • Formally, ©Gao Yang, Ai Lab NJU

  15. Abstract architectures: summary • Step1: start initial internal state i0; • Step2: observe the environment of state s, and generate a perception see(s); • Step3: internal state of agent is updated via next function, next(i0, see(s)); • Step4: action selected according to current internal state, action(next(i0, see(s))); • Step5: action is performed and goes into a new cycle, go to step2. ©Gao Yang, Ai Lab NJU

  16. Abstract architectures: references • R. Fagin, J.Y.Halpern, Y. Moses, and M. Y. Vardi. Reasoning About Knowledge. The MIT Press: Cambridge, MA, 1995 • M. P. Georgeff and A. L. Lansky. Reactive reasoning and planning. In Proceedings of the sixth National Conference on Artificial Intelligence (AAAI-87),pages 677-682, Seattle, WA, 1987. • J. Y. Halpern. Using reasoning about knowledge to analyze distributed systems. Annual Review of Compute Science, 2:37-68, 1987. • S. Russell and E. Wefald. Do the Right Thing – Studies in Limited Rationality. The MIT Press: Cambridge, MA, 1991. • S. Russell and D. Subramanian. Provably bounded-optimal agents. Journal of AI Research, 2:575-609, 1995. ©Gao Yang, Ai Lab NJU

  17. Concrete architectures • But how to implement the abstract architecture of intelligent agent? • Logic based architectures: in which decision make is realized through logical deduction. • Reactive architectures: in which decision make is implemented in some form of direct mapping from situation to action. • Decision-theoretic architectures : in which decision make depends on decision theoretic. • Belief-desire-intention agents: in which decision make depends on the data structures representing the beliefs, desires, and intention. • Layered architectures: in which decision make is realized via various software layers. ©Gao Yang, Ai Lab NJU

  18. Logic-based architectures • Symbolic AI • The intelligent behavior can be generated by giving a symbolic representation of its environment and its desired behavior, and syntactically manipulating this representation. • Logic-based architectures • Agents use a symbolic representation of their environment and behavior. • The symbols correspond to logical formulas and the agent reasons using logical deduction or theorem proving. ©Gao Yang, Ai Lab NJU

  19. Logic-based : a simple model • The internal state of agent • Is assumed to be a database of formulae of classical first-order predicate logic. • Open(valve221) • Temperature(reactor4726,321) • Pressure(tank776,28) • This database is the information that agents have about their environment. • This database plays a somewhat analogous role to that of belief in humans. • Deduction rules determine its behavior: • Open(x)^ Pressure(x,y)=>Do(GetRward) ©Gao Yang, Ai Lab NJU

  20. Logic-based : some notions • Let L be the set of sentences of classic first-order logic. • Let be the set of sets of L-formulae. • Write for members of D. • An agent’s decision making process is modelled through a set of deduction rules, • We write if the formula can be proved from the database using only the deduction rules . ©Gao Yang, Ai Lab NJU

  21. Logic-based : pseudo-code ©Gao Yang, Ai Lab NJU

  22. Logic-based : Vacuum world ©Gao Yang, Ai Lab NJU

  23. Vacuum world: perceptions and actions • Perceptions • dirt, null • Actions • forward, suck, turn • Domain predicates • In(x,y) agent is at (x,y) • Dirt(x,y) there is dirt at (x,y) • Facing(d) the agent is facing direction d ©Gao Yang, Ai Lab NJU

  24. Vacuum world: next function • First, write to denote the set of “old” information in a database. • Secondly, give the set of new predicates to add to the database. • new function is not difficult, but rather lengthy. • Given the new and old functions, the next function is defined as follows. ©Gao Yang, Ai Lab NJU

  25. Vacuum world: behaviors • The deduction rules have the form • Examples ©Gao Yang, Ai Lab NJU

  26. Pragmatics of logic-based approach • Logical approach brings simple and elegant semantics – the agent’s behavior can be guaranteed. This may be useful for safety-critical applications. • But theorem proving takes time – by the time the agent proves which action is optimal the environment may have changed. • This leads to the problem of calculative rationality – the decision making apparatus produces action that was optimal when decision making process began. • Only suitable when environment doesn’t change faster than the agent can make decisions. ©Gao Yang, Ai Lab NJU

  27. Problems of logic-based approach • The function see maps the environment into a symbolic percept. It’s often not obvious how to realize this, e.g., • How can an image be transformed into a declarative statement? • Representing and reasoning about temporal information – how a situation changes over time – is extraordinarily difficult. • Representing even simple procedural knowledge can be unintuitive and cumbersome. ©Gao Yang, Ai Lab NJU

  28. Logic-based architectures: references • M. R. Genesereth and N. Nilsson. Logical Foundations of Artificial Intelligence. Morgan Kaufmann Publishers: San Mateo, CA, 1987. • K. Konolige. A Deduction Model of Belief. Pitman Publishing: Lodon and Morgan Kaufmann: San Mateo, CA, 1986. • Y. Lesperance etc. Foundations of a logical approach to agent programming. In M. Wooldridge, J. P. Muller, and M. Tambe, editors, Intelligent Agents II (LNAI volume 1037), pp 331-346, Springer-Verlag: Berlin, Germany, 1996. • J. McCarthy and P. J. Hayes. Some philosophical problems from the standpoint of artificial intelligence. In B. Meltzer and D. Michie, eds, Machine Intelligence 4. Edinburgh University Press, 1969. • S. J. Rosenschein and L. P. Kaelbling. A situted view of represent-ation and control. In P. E. Agre and S. J. Rosenschein, eds, Computational Theories of Interaction and Agency, pp 515-540. The MIT Press: Cambridge, MA, 1996. ©Gao Yang, Ai Lab NJU

  29. Reactive architectures • The motivation for reactive agents • Rejects decision-making based on syntactic manipulation of symbolic representations of knowledge. • Believes the rational behavior cannot be disembodied but is a product of the interaction the agent maintains with its environment. • Believes that intelligent behavior emerges from the interaction of various simpler behaviors. ©Gao Yang, Ai Lab NJU

  30. Reactive architectures • Sometimes called: • Behavioral: developing and combining individual behaviors. • Situated: agent is situated in its environment – not disembodied. • Reactive: reacts to changes in the environment without explicitly reasoning about it. • We will look at the subsumption architecture of Rodney Brooks. ©Gao Yang, Ai Lab NJU

  31. Subsumption architecture • An agent’s decision-making is realized through a set of task accomplishing behaviors. • Each behavior is like an individual action function, continually taking perceptual input and mapping it onto an action to perform. • No complex symbolic representations and no symbolic reasoning at all. • Mainly implemented through rules like: • situation --> action ©Gao Yang, Ai Lab NJU

  32. Subsumption architecture ©Gao Yang, Ai Lab NJU

  33. Rule firing • Many behaviors can be fired simultaneously. • The subsumption hierarchy arranges the behaviors in layers that are used to choose between actions. • Lower layers inhibit (have priority over) higher layers. • Higher layer represent more abstract behaviors. • Example: a mobile robot might have a behavior “avoid obstacles” at a low layer (higher priority) and a behavior “fetch coffee” at a higher layer (lower priority). ©Gao Yang, Ai Lab NJU

  34. Subsumption sub-systems • The function see is unchanged, producing a set of percepts P. However, there is usually tight coupling between perception and action – raw sensor input is not processed or transformed much (no symbols) • The function action is realized through a set of behavior rules, R, together with an inhibition relation, ,over time: • Where c is a set of percepts called that condition and a is an action. ©Gao Yang, Ai Lab NJU

  35. The inhibition relation • A behavior will fire in state s if see(s) c (if the condition is satisfied by the percepts). • The inhibition relation is a total ordering on the behavior rules. • If r1 inhibits r2, we write r1 r2, i.e., • r1 is lower in the hierarchy than r2 and hence will get priority over r2. ©Gao Yang, Ai Lab NJU

  36. Action fn for subsumption agent ©Gao Yang, Ai Lab NJU

  37. Action fn for subsumption agent • The function action computes all the behaviors that fire for a given input and returns the one with the highest priority (lowest in ordering), or null if no behaviors have fired. • The time complexity for action is no worse than O(n2) where n is the large of the number of behaviors or number of percepts. • In fact, if this function is encoded in the agent’s hardware, it can make decision in constant time, I.e., O(n) (nanoseconds for modern hardware) ©Gao Yang, Ai Lab NJU

  38. Example: Mobile robot planet explore • The task is to explore a distant planet and collect samples of precious rock. The location of the rock samples is unknown, but typically they are clustered. Autonomous mobile robots drive around collecting samples and return to the mothership to deposit them. No map is available, but the terrain is full of obstacles(hills, valleys), which prevents the robots exchanging information. ©Gao Yang, Ai Lab NJU

  39. Robot’s sensors, action, goals • Sensors • Can detect obstacles and rocks • Can detect signal from mothership (variable strengths) • Actions • Can pick up and deposit samples • Can move around terrain • Goals • Avoid obstacles, pick up rocks and return to mothership ©Gao Yang, Ai Lab NJU

  40. Robot’s behaviors • B1 if detect an obstacle then change direction • B2 if carrying samples and at base then drop samples • B3 if carrying samples and not at base then travel up signal gradient • B4 if detect a sample then pick up sample • B5 if true then randomly explore terrain. • Inhibition relation ©Gao Yang, Ai Lab NJU

  41. Enhancing robot behavior • Even though they cannot communication directly, by interacting with the environment, the mobile robots can communicate indirectly. • Adjust behavior B3: • B3’ if carrying samples and not at base then drop 2 crumbs and travel up signal gradient. • Add behavior B6; • B6 if sense crumbs then pick up 1 crumb and travel down signal gradient. ©Gao Yang, Ai Lab NJU

  42. Creating cooperation • By adding behaviors and amending the inhibition relation to : • The mobile robots’ individual and group behavior becomes more effective. • They are now cooperating by communicating via the environment. ©Gao Yang, Ai Lab NJU

  43. Revised reactive behavior • With the simple adjustment, near-optimal performance can be achieved: • Robots drop a trial of crumbs that leads to the cluster of rocks • Other agents follow this to find more rock and reinforce the trail on the way back • When all rocks removed, the trail diminishes since it is not being reinforced • This cooperative behavior emerges from simple reactive rules. • This is similar to ant-like behavior using pheromones. ©Gao Yang, Ai Lab NJU

  44. Reactive architectures: Pros and cons • Advantages • Simple • Economic • Computationally tractable • Robust against failure • elegant ©Gao Yang, Ai Lab NJU

  45. Reactive architectures: Pros and cons • Disadvantages: • Only react to local information • Short-term view of decision-making • Not capable of learning about environment since don’t keep track of it • Emergent properties means cannot be fully understood – engineering specific emergent behavior involves experimentation, trial and error. ©Gao Yang, Ai Lab NJU

  46. Decision-theoretic architecture • Agents use decision theory to determine best action to take. • Decision theory = Probability Theory + Utility Theory • Basically a state-based agent for which decision-theory is used in the action function. • Internal state of agent consists of current beliefs plus: • A probabilistic model of their environment • A utility (or reward) function • Agent selects the action that maximizes its expected utility, given its belief. ©Gao Yang, Ai Lab NJU

  47. When next state is uncertain • Usually, an agent will not know exactly which state an action will lead to, but will have a transition table of probabilities and a reward function for each state. • For each state-action pair, the agent uses the transition probabilities to compute the expected reward of the action. ©Gao Yang, Ai Lab NJU

  48. Example: Playing the national lottery • Agent has two action: to bet or not to bet. • To bet the agent pays $1 and chooses 6 numbers from 49 • If he doesn’t bet, the agent keeps his $1 • If three or more of his numbers com up in the weekly draw of six balls(plus a bonus ball), agent stands to win a cash prize • What will the agent decide to do ? ©Gao Yang, Ai Lab NJU

  49. States and rewards in national lottery ©Gao Yang, Ai Lab NJU

  50. Expected reward • If the agent bets on the National lottery, its expected reward is the sum of rewards from all possible resulting states. • If the agent doesn’t bet, he keeps his $1. • What would a decision-theoretic agent do? ©Gao Yang, Ai Lab NJU

More Related