Intelligent Agents -Agent Mobility and Cloning

1. Intelligent Agents -Agent Mobility and Cloning Katia Sycara The Robotics Institute email: katia@cs.cmu.edu www.cs.cmu.edu/~softagents

2. Information server Host Agent1 Network Agent2 Idle Host Host Agent1 Network Agent2

3. Mobile Agents Overview • In some types of applications, it is useful to have mobile agents: • software agents that can migrate between machines to better utilize resources • agents that reside on a mobile platform, and need to adjust to changes in the machine location • Mobile agents can improve task performance and efficiency of using bandwidth and CPU • Mobile agents require special services to allow for their application

4. Mobile Agents • Programs typically written in a script language that may be dispatched from a client computer to execute remotely on a server • sending a program without state to a remote CPU to execute there; program possibly communicates by RPC with other CPUs and then terminates (e.g., Java programs) • sending a program that carries its execution state to a remote CPU; program executes and then moves to a third CPU or returns. This is called a mobile agent

5. Mobile Agents (cont) • The mobile agent execution environment is called the Agent Meeting Place. It allows • passing parameters between the application program and the modules of the agent • binding of user interface libraries of the client’s operating system • binding to other operating system functions such as the memory manager, the file system, etc, in particular to the message transport service (sending and receiving agents via the communication infrastructure)

6. Mobility Process • Suspend current process or create daughter process • Collect new or suspended process including execution state into a message expressed in machine independent form • Send message to • a particular known a priori machine • a post office that performs address resolution • intermediate designation that routes the agent based on message content (semantic routing)

7. Mobility Process (cont) • Upon receipt of the message, the destination message subsystem delivers it to the server’s agent execution environment • In server execution environment, message is reconstituted into the execution thread(s) and is dispatched • Execution continues at the next instruction following the instruction that initiated mobility

8. Mobility Process (cont) • During execution at the server, the agent passes information from the client application and receives information from it, or • Interacts with other agents on the server • The agent may: • terminate its execution • suspend its execution at the server waiting for some event from a server application or some other agent • repeat the migration process, returning to its origin or going somewhere else, depending on the situation

9. What to Move? • When agents collect and process information on the web, they consume bandwidth • Attempting to reduce bandwidth consumption, a mobile agent can move to the information site instead of bringing the information to it • But this requires bandwidth for code transfer, so if the agent is “heavy weight”, or it needs a little information - migration is not beneficial • In practice, only when information transfer is significantly greater than code transfer migration is beneficial

10. Infrastructure Requirements • A mobile agent that arrives at a foreign machine needs: • to be accepted by that machine: be provided with access to machine resources • to be written in a transportable language, able to run on any platform • not to violate privacy or harm host machine • be able to maintain state, to run itself, load code dynamically, stop itself • protect itself from malicious host

11. What is the Price? • Locally, a mobile agent needs to reason about itself, its needs, the network, and decide how to proceed • Altogether, agent mobility imposes overheads: • in computation for decision making • in infrastructure for receiving agents • in bandwidth for migration

12. And What is the Promise? • In the right settings, agent migration can improve performance significantly • In some cases, mobility is unavoidable • E.g., mobile platforms (laptops, palms): • when a machine is about to disconnect from the network it is beneficial for an agent to: • get off the machine with essential information, state and processing • continue living on the network until the machine is re-connected, possibly somewhere else • move back to the machine, update it with changes, state

13. Promise (cont) • Reduction of overall communication traffic over low bandwidth, high latency, high cost access networks employed by mobile computers • Ability of the agent to engage in high bandwidth communication to search large free text databases on the server • Ability of lightweight mobile computers to interact with heavy applications without prior detailed knowledge of the server’s abilities

14. Promise (cont) • Ability of the agent to integrate knowledge from the client and server while performing interfacing at the server • Ability of the user to personalize services by customizing agents that are resident at a server

15. Current State • Only a hand full of mobile agent systems support strong mobility, ie the ability to capture and transfer the full execution state of the agent. • Since strong mobility requires capture of thread execution state, and since this is not currently possible in current Java VM, modifying the Java VM has been the strategy (e.g. NOMADS, ARA, Sumatra, WASP)

16. Current State (cont) • In current Java Security Manager, there is no way to control: • -disc storage • -number of I/O operations • -number fo simultaneous print jobs • -thread and process priorities • -number of allowable open windows

17. Additional Mobile Agent Info • Mobile agents list: http://www.informatic.uni-stuttgart.de/ipvr/projecte/mole/mal/ • lists over 60 mobile agent systems • voyager, oddyssey, aglets, D’agents are some mobile agent frameworks • mobile agent languages: safe-tcl, telescript, Java, Milog

18. Agent Cloning/Merging • Cloning supersedes agent mobility • Cloning could be done with or without the agent’s knowledge • Cloning goals: • --To mitigate against agent overload • --To achieve load balancing in a distributed system • --To achieve higher throughput

19. Agent5 c5 Agent1 c1 Agent6 c6 T4 Agent7 c7 T5 T7 T9 T8 T T2 T T1 T T3 Idle Host Agent9 c9 T6 T10 Agent10 c10

20. Overloads in MAS • Agents receive a stream of tasks T • Tasks require capabilities and resources • Agent’s capacity is limited. Therefore: • an agent’s resources for task execution may be insufficient. As a result: • agents may be overloaded • at the same time others, on other machines are under-loaded • the system utilizes resources inefficiently

21. Cloning • Self-aware agents reason and anticipate overloads (predictive cloning) • Self-aware agents detect overload after it has occurred (reactive cloning) • Agents explore computational environment to find other agents, machines, resources • Agents plan for task delegation, migration and cloning to resolve overloads • Cloning is the creation and activation of other agents (on local or remote machines)

22. Create a clone Agent5 c5 Agent1 c1 Agent6 c6 T8 T8 T7 T9 Agent7 c7 T8 Agent2 (clone) T T2 T1 T1 T T3 Create a clone T T Idle Host Task delegation Agent9 c9 T6 T9 Agent8 (clone) Agent10 c10

23. The Cloning Algorithm(1) (performed by each agent) • Self awareness activity (do periodically, forever (in a separate thread of control)): • retrieve self available resources R(t0) (cpu, memory, disk space, comm. band width, etc) • retrieve self current loads L(t0) • anticipate current and expected Tl- capabilities and resources required, deadlines. Accordingly: • compute future (expected) loads L(t), resources R(t) • compare R(t), L(t). Locate periods of overload OV(t) (what type of overload, what tasks cause it).

24. Algorithm(2) - Reasoning • If OV(t) not empty, reason for cloning: • locate other agents and hosts • for each located entity E: • retrieve current loads LE (t0) • anticipate future tasks and loads LE (t) • solve optimization problem to allocate tasks G in Tl that may cause OV(t) to entities. Refer to: • the remote loads LE (t) • task deadlines and required capabilities, resources • if E not agent - include creation/activation overhead

25. Algorithm(3) - Task Delegation • The optimization process provides: • S, set of agents that can execute tasks in G • for each ai c S, a unique, timed task list Tai (for each task in Tai, est and lst) • H, set of hosts with available resources • a timed task list TH for remote hosts • For each ai c S, • delegate tasks in Tai at appropriate time

26. Algorithm(4) - Create, Activate, Reason After • While TH is not empty • select a preferred host h in H • create and activate a clone ah on h • delegate tasks from TH to ah • Reason, learn after cloning (subsequent slide) • Reason about underloads. Consider: • merging with under-loaded agents • self-extinction

27. Reasoning Guidelines • Given the current task list and expected task distribution (provided or learned): • If required resources at any time exceed agent maximum capacity- • move tasks of the overloaded time to split-list • if no agents can perform these tasks: • locate a host, create a clone, pass split-list

28. Reasoning for Cloning

29. Cloned-Agent Activity • Introduces itself to MAS (via middle agent) • Receives self task list Tl from creator’s TH • Performs tasks (independently from creator) • When Tl completed, if no new tasks arrived (reasons periodically w.r.t. underload): • self-extincts, or • merges with creator (or with another agent)

30. Reasoning After Cloning • Collect info. about success, loads and availability of agents and hosts w.r.t. time • Compute averages, variances and predictors of this information • Use these results to update probabilities (of success of using agents/hosts) to the optimization problem • Future cloning decisions will be affected

31. Cloning Simulation • Setting: • 10 to 20 agents, up to 10 additional clones • up to 1000 dynamically arriving tasks • an agent can perform 20 tasks simultaneously • Results: • up to 100 tasks - close to 100% of tasks are executed w/wo cloning. • 100 to 350 tasks: • w/o cloning, MAS overall task execution poor • with cloning approx. 85% of tasks met deadlines

32. System Throughput 44

33. Additional Utility of Cloning • Can be used for agent migration and mobility by: • creating a clone • passing all tasks to the clone • original agent performs self extinction

34. Agent Merging • When an agent or the underlying infrastructure detects underutilization of an agent: • kills the agent, if its capability is replicated, notifies its current customers, or • selects agent with replicated capability and passes it the tasks of the underutilized agent, notifying its customers