Intelligent Agents

Intelligent Agents Katia Sycara katia@cs.cmu.edu The Robotics Institute Joseph Giampapa garof@cs.cmu.edu www.cs.cmu.edu/~softagents

What is an agent? • An agent is an autonomous computational entity, which: • is reactive and proactive • is goal driven • is intelligent: • able to reason, plan and sometimes learn • has domain specific intelligence • interacts with humans, other agents, and the environment via sensors and effectors in a high level language/protocol • anticipates user needs and reacts based on them • wish list: friendly, understands natural lang.,etc

Multi-Agent Systems (MAS) • An agent is more useful in the context of others: • can concentrate on tasks of its expertise • can delegate other tasks to other experts • can take advantage of its ability to intelligently communicate, coordinate, negotiate • But, a MAS is not just a collection of agents • it needs meaningful ways for agents to interact • it needs some system design and performance evaluation

MAS - Two Approaches 1. Centralized design • Build a system that is comprised of agents - should provide good performance • Advantages may arise from: • possibility to develop each agent as an expert • incorporation of non-local expertise • rather simple to have multiple developers working concurrently • Example: a system within an organization

MAS - Two Approaches 2. Open MAS • Usually, the system has no prior static design, only single agents within • Agents seek others to provide services, without knowing in advance who they are • There is a need for agent finding mechanism • Other agent may be non-cooperative or untrusted or malicious • Example: markets, Internet

Design and Architecture - Outline • Design philosophies • Information processing and needs • Reactive architectures • Deliberative architectures • Layered architectures • Belief, Desire, Intention (BDI) • Concurrent architecture (RETSINA)

Agent Design Philosophies • Agents reside in the environment: the world and other agents • The environment can be characterized by a set of states S={s1,s2,...} • Agents execute actions A={a1,a2,…} • An action is a function action: SS, that affects the environment via state change • So, in general, an agent is a set of actions that receive input from the world and manipulate the state of the world

Agent - Environment Interaction Agent Action Output Sensor Input Environment

Architecture Design • A map of the internal structures of the agent, includes: • data structures • operation that can be performed on them • control and data flow between the structures • Starts from a high-level definition and traverses through refinements • Design decisions result in adding details, getting closer to code level, reducing generality

Information and Processing Needs • System architecture design needs knowledge of: • what the expected inputs are? • what the required/expected outputs are? • what processing can provide this relation between input and output? • For agents, in particular: • what are the possible/expected states of the world? • how should the world state be perceived? • how should the agent’s reasoning be affected by the world state? • how should agent reasoning result in agent action?

Agent Information Needs • Is the “state of the world” sufficient? • Yes. Usually it is too much: • some or most of it may be inaccessible • dynamic and possibly non-deterministic • includes other agents, users, internet, etc • Perception filters and reduces amount of info. • Design question: what is the minimal set of data and how to filter/extract it?

Agent Architectures • Reactive architectures • Deliberative architectures • Layered architectures • Belief, Desire, Intention (BDI) • Concurrent architecture (RETSINA)

Agent Architectures (cont) • Reactive vs. deliberative • Reactive agents: upon input from the environment, they react with action execution • Deliberative agents: upon input, a reasoning process is invoked. Action is based on the results of the reasoning • Agents may populate the whole spectrum between purely reactive and maximally deliberative (rational)

Agent Processing Needs • Reactive agents only need to map between world states and actions • Deliberative agents need to reason for action. May include: • taking into account historical states of world • creating and maintaining an internal state • reasoning about world and states • planning, re-planning for current/future action • learning

Agent Processing Needs (continued) • Collaborative (social) agents need, in addition: • maintain models of other agents and the society • reason about others • plan collaborative activity • reason about interaction: communication, coordination, collaboration

Required Agent Attributes • Perception - a function perceive: SP, where P={p1,p2,…} a set of percepts: • required for both reactive and deliberative agents • may be provided via sensor or any other input • Internal state I (records history) • not necessary for reactive agents • deliberative agents need to maintain information regarding past activity to allow for deliberation • Reasoning: performed mainly by deliberative agents, but may be useful for reactive, too • Learning: only in deliberative agents

Reactive Architecture Agent Perception Action Environment

Agents without State (Reactive) • Perception is a function perceive: SP • Action is a function action: SS, • Action selection is a function as: PA • The world state results in a percept via perception, the percept results in an action selection, and the action transforms the state of the world

Example: Subsumption (Brooks) • An agent decision making is performed by a set of task accomplishing behaviors (TAB) • In Brooks’ implementation, each TAB is a finite state machine • In other implementation, TABs are rules of the type situationaction, which maps percepts to actions • In the subsumption architecture, multiple behaviors can be activated simultaneously • Action selection is based on a subsumption hierarchy, behaviors arranged in layers, which are at different layers of abstraction (layered architecture)

Reactive: Pros and Cons • Pros: • simplicity, economy • computational tractability • fault tolerance • overall behavior emerges from component interaction • Cons: • without a model of the environment, agents need sufficient info. to determine action • agents are “short-sighted”, may limit decision quality • relationship between components not clear.

Agents with State = Deliberative Agent Perception Action Reasoning State Environment

Agents with State (Deliberative) • Perception is still a function perceive: SP • Action is still action: SS • But action selection (was as: PA), is now the function as: IA • In addition, update: P×II is a function that update the internal state based on percepts (may include complex reasoning)

Agents with State: Refinement Agent Perception Action Reasoning State planning learning inference Environment

Layered Agent Architectures • Usually, but not always, deliberative architectures • Decision making is performed via separation to several software layers • Each layer reasons at a different level of abstraction. Layers interact • Two major types: • vertical layers: perception input and action output are dealt with by a single layer each • horizontal layers: each layer directly connects to perception input and action output

Layers’ design • Typically, at least two layers, one for reactive behavior and one for proactive • No reason not to have multiple layers • Typology: information and control flow between the layers, e.g.: Agent Perception Action

Layer n Layer n Layer n Layer 2 Layer 2 Layer 2 Layer 1 Layer 1 Layer 1 Information and Control Flow Action output Perceptual input Action output Perceptual input Perceptual input Action output Horizontal Vertical (one pass) Vertical (two pass)

Layers Pros and Cons • Horizontal • each layer acts like an agent - provides independency, simplicity • for n different behaviors we implement n layers • competition between layers can cause incoherence • need for mediation between layers: exponentially complex, a control bottleneck • Vertical • Low complexity, no control bottleneck • Less flexible and not fault tolerant: one decision needs all layers

TOURINGMACHINES • Three layers produce suggestions for action: • reactive: implements situation-action rules as in Brooks’ subsumption architecture • planning: achieves proactiveness via plans based on a library of schemas • modeling: model of world, other agents, self, predicts conflicts, generates goals to resolve them • Domain of implementation: multiple vehicles

Example: TOURINGMACHINES Perception input Modeling layer Perception subsystem Action subsystem Planning layer Reactive layer Action output Control subsystem

INTERRAP • A vertically layered two pass architecture • Layers have similar purposes as in TOURINGMACHINES • Each layer is associated with a knowledge-base • Layers interact with each other: • bottom-up: activation • top-down: execution

Example: INTERRAP Cooperation layer Social knowledge Plan layer Planning knowledge Behavior layer World model World interface Perception input Action output

Belief-Desire-Intention Architecture • Based on practical reasoning and decision on actions. Involves: • decision on what goals we want to achieve: deliberation • decision on how to achieve these goals: means-ends reasoning • Choosing some options creates intentions. These: • usually lead to action • should persist: • once adopted, an agent should persist with the intention, attempt to achieve it • the intention should be dropped if • it is clearly non-achievable • it was already achieved • the reason for the intention is not there anymore • are related to beliefs about the future

BDI Architecture Components • A set of current beliefs about the environment • A belief revision function (brf) - updates current beliefs based on perception • An option generation function - determines available options (desires) based on beliefs and intentions • A set of desires (current options) - possible courses of action available • A set of current intentions - the options the agents is committed to trying to perform • A filter function (deliberation) - determines new intentions based on current beliefs, desires, intentions • An action selection function - selects actions based on intentions

Schematic BDI Architecture Perception input brf beliefs Generate options desires filter intentions action Action output

BDI Pros and Cons • Key problem: difficult to balance between mental activities • Example: dropping intentions requires reconsideration, which is costly but needed • Rate of environment change helps set re-consideration • Questionable: what advantage do mental states provide? • Intuitive, provides functional decomposition • Easy to define formally, using logic, and convert to code

Concurrent Architectures(RETSINA: Sycara & al.) • Include multiple functional and knowledge modules that work concurrently • Coherence between the functional modules is achieved via shared databases • Typical functional separation: • communication and collaboration • planning and reasoning • action scheduling • execution and monitoring

Example: RETSINA Agent Architecture

Functional Components • Communicator: handles incoming and outgoing messages in an ACL. Converts requests into goals/objectives • Planner: takes objectives and devises detailed plans to achieve them. Creates tasks, actions and new objectives. Uses plan fragments from libraries • Scheduler: schedules actions for execution • Execution monitor: executes actions and monitors • Coordination/collaboration: reasons for such activities, may be internal to planner or to communicator • Self-awareness: maintains self model: load, state, etc

Planning • By incremental instantiation of plan fragments • Conditional planning mechanisms • Interleaving planning, information gathering, and execution • Declarative description of information flow and control flow requirements 11

Task decomposition

Knowledge Components • Objective DB: holds the agent’s objectives • Task DB: holds the agent’s tasks and actions, before they are scheduled for execution • Schedule: holds scheduled actions • Task reduction library: includes a set of possible task decompositions • Task schema library: includes plan fragments, each provides details on how to perform a task • Beliefs DB: holds the beliefs of the agent regarding information relevant to its activity

Architecture Attributes • Functional components do not directly interface or synchronize with each other • Knowledge components do not directly interface or synchronize with each other • Functional components work concurrently • These provide: • reusability and substitutability of components • efficient utilization of computational resources • timely task performance • reduced development effort

RETSINA Agent Functionality • Interacts with humans and other agents • Anticipates and satisfy human information needs • Provides decision support • Integrates planning, information gathering and execution • Acquires, use and disseminate timely and relevant information • Adapts to user, task and situation 10

Intelligent Agents