ALFworld: Mobile Rational Agents for Complex Computations

1. CADIP 2002 Program Jerry Stach, Ph.D. Eun Kyo Park, Ph.D. Agent Life Forms Laboratory School of Interdisciplinary Computing and Engineering University of Missouri – Kansas City

2. CADIP Project Biologic Agency for Search of Intractable Spaces Opening Soon at UMKC-ALFworld!

3. Research Vision To develop a mobile, rational agent, capable of accepting a payload and by process or data driven signature, migrating to locations in a network to optimally complete its computational task.

4. Applicable Domains • Software robots • Web Computing • Grid Computing • Document Processing • Intelligent Search • Collaborative Computing

5. Limitation • CPU becomes band limited as problem scales • mobility decision/agent planning and reasoning time for site/service selection • Service Place executions implying control of arrival rates • communication between mobile agents and to situated agents (e.g. Traders)

6. Limitation • Links become band limited as problem scales • message exchanges between agents, both situated and mobile • agent transportation (payload and code) • Trader Updates

7. Approach provide computational autonomy, strong mobility and self regulation to damp bandwidth • adopt a biological (A-Life) computing model for the MAS • provide an artificial world infrastructure for agent computing • seek performance and scaling via emergent behaviors as opposed to policy and protocol • endow agents with social conscience (e.g. congestion avoidance when possible), preferences and rationality in decision making

8. Research Problems • since there is no MAUF, perception is by characteristic function • fuzzy reasoning over Service Places • desirable population behaviors are emergent, not first order effects of policy • operating system and network support of strong mobility does not exist • existing agent architecture and design patterns are not robust enough to support architecture

9. Completed Work Mobility Decision Simulations • modeled optimal mobility decisions based upon graph theoretic solution • results provided experiential basis for proceeding to perception • simulator provides an observational basis for prediction of agent colony behavior and performance • results do not scale

10. Completed Work WhitePaper on Strong Mobility In strong mobility, not only code and data state are moved, but also the execution state, in order to restart the execution exactly from the point where it was stopped before movement. Strong Mobility is frequently used in load leveling applications

11. Strong Mobility Strong mobility has the ability to store and retrieve computations as variables (continuations) and passes these to the other agents (remote continuations). Strong mobility also usually communicates in an asynchronous fashion in which one agent sends messages to other agents but does not wait for answers. Whenever one of the communication partners of an agent dies, the agent continues, even if it is waiting for some action of the dead partner.

12. Completed Work continued Built ALFworld – a 40 node Beowulf

13. Plan for Today • Sketch of artificial world - situated and mobile agents • Take a look at a few perception functions – congestion, reliability, difference • Conclude with 2003 Activities

14. World Architecture AutoPilot Network Components Certificate Authority Router World Places Annihilator Docked AutoPilot Service Places Docking Agent Service Planner Topologist Trader Trader Places

15. Docking Agentcertifies arriving AutoPilots are legitimate in network and at site Service Plannerreceives request from AutoPilot, requests service locations from Trader based upon AutoPilot passport and security clearance; returns service advertisements to AutoPilot. Situated Agents Certificate Authority Router Annihilator Docked AutoPilot Docking Agent Topologist Service Planner Trader

16. Service Planneralsorequests geodesics from Topologist and computes Cost of Service to advertise in next Trader update to indirectly manage agent arrival rates. • Annihilatorkills off rogue processes and is last resort for population control • Topologistinterfaces to network router and requests current shortest path lengths to Service Places being considered by AutoPilot Certificate Authority Router Annihilator Docked AutoPilot Docking Agent Service Planner Topologist Trader

17. Trader requests environmental and service statistics from Service Places and updates advertisements. • Tradercertifies validity of Service Place requests for advertisement. If Traders are multiple or hierarchical, Trader also implements the Trader to Trader security protocol. Certificate Authority Router Annihilator Docked AutoPilot Docking Agent Service Planner Topologist Trader

18. AutoPilots are Mobile, Autonomous Agents • AutoPilot picks up payload consisting of signature, metadata and preferences from birth location. • Docking Agent obtains passport and security clearance from Certificate Authority. • Autopilot requests for each service in its signature, a set of advertisements from the Service Planner and may prune them. Autopilot returns a set of acceptable advertisements and attribute preference set to Service Planner. Certificate Authority Router Annihilator AutoPilot Docked Service Place Docking Agent Service Planner Topologist Trader Places Trader

19. Service Planner computes attribute values for each Service Place. • AutoPilot reasons next site based upon perception of attribute values returned by Service Planner. • After last service executed, AutoPilot returns payload and meta data to originating Docking Agent at birth location. • Annihilatorterminates AutoPilot and reports returns Passport to Certificate Authority. Certificate Authority Router Annihilator Docked AutoPilot Docking Agent Service Planner Topologist Trader

20. Perception by Characteristic Function Congestion, Reliability, Difference

21. Agent Framework • To discriminate between service locations in a network, an agent must be able to sense its environment. • Since all nodes exhibit all attributes, agent preference must govern perception. We perceive local conditions by characteristic functions applied to sensed values. • Reasoning is “fuzzy” being approximate and over uncertain and aged data. • Behavior is limited to non-deterministic choice of service and migration • Meta data provides the context for decision making Meta data Behavior Reasoning Perception Sensing Autonomous, Rational Agent

22. Perception Function Goals • Functions should produce “reasonable” output • Perceptions should have good correspondence to the subjective notions they represent • Functions should be based in theory, i.e. a characteristic function with PDF and CDF over the universe values of the attribute

23. Perception occurs in subjective time • An Agent’s life is finite in the system • An Agent carries a task signature yielding an expectation of the duration of work • An Agent must “sense” its own mortality in subjective time by half life i.e. sense urgency where N0 and NT are initial and decayed attribute values respectively

24. As urgency increases, risk aversion decreases measure of risk aversion OR

25. Role of Origin and Limit in Perception Functions • In subjective time, the origin corresponds to the agent’s threshold of sensitivity to the service attribute. • In subjective time, the limit corresponds to the agent’s tolerance, i.e. value of indifference for the attribute, beyond which all values are unacceptable. • These two values set the slope over which the perception function is differentiated in order to return a fuzzy membership value.

26. Perceiving Expected Waiting Time • The congestion function is F(x)=r/m(1-r) Waiting Time Utilization

27. Service Place Population First Order Approximation • let l be arrivals per unit of time and m be services per unit of time.

28. Service Place Effectiveness

29. Service Place Effectivenesscontinued

30. The Case for Congestion Set Membership • Simply computing a wait time is not sufficient because it is without subjective value to the agent • there may be no wait time below the desired delay contribution • the wait time and utilization alone are insufficient to determine whether a node is in an “unsafe” state • any node not idle is “congested” • “Unsafe state ” is a condition subject to the Agent’s tolerence,i.e., corresponding to the attribute limit for the sensed value

31. Differentiating Rate of Change demands an origin and limit • The rate of change for the exponential function is constant over any interval. • The point at which the derivative of the function matches the average rate of change is the congestion point for the Service Place Utilization. • Average rate of change is of the form f’(x) = f(b)-f(a)/b-a where a,b are the origin and limit respectively.

32. Unsafe States and Subjective Time • If an Agent has a priority task or is aged relative to its remaining work, its tolerance for delay (r) decreases • This tolerance though subjective, is always bounded 0  origin  perceived value  limit 1. • To find this point in the congestion function requires we compute rho (r) [on the x axis] and waiting time W(r) [on the y axis].

33. Finding the Subjective Unsafe States These equations give the Waiting Time and Service Place Utilization values above which Service Places are “unsafe” with respect to the agent’s perception of time.

34. Service Place Membership In The Fuzzy Set Congestion r(Lq) • Suppose at inquiry the Trader reported [SPname, Lq,μ,λ].

35. Validation of Perception Function • Extreme Valued Test Case List of Service Places at time of Trader Update were arbitrarily configured with some nodes congested SpID qLength (ServiceRate) (ArrivalRate) 1 0 5.1 7.5 2 1 5.1 7.5 3 2 5.1 7.5 4 3 5.1 7.5 5 4 5.1 7.5 695 5.1 7.5 797 5.1 7.5 899 5.1 7.5 9101 5.1 7.5 10103 5.1 7.5

36. Output of Perception Function Service Places Configured as Congested List of Agents AgentID Tolerance SP1 SP2 SP3 SP4 SP5 SP6 SP7 SP8SP9 SP10 1 0.500000 --- *** *** *** *** *** *** *** *** *** 2 0.800000 --- *** *** *** *** *** *** *** *** *** 3 0.900000 --- --- *** *** *** *** *** *** *** *** 4 0.940000 --- --- --- *** *** *** *** *** *** *** 5 0.970000 --- --- --- --- *** *** *** *** *** *** 6 0.980000 --- --- --- --- --- *** *** *** *** *** 7 0.990000 --- --- --- --- --- *** *** *** *** *** 8 0.999000 --- --- --- --- --- *** *** *** *** *** 9 0.999900 --- --- --- --- --- *** *** *** *** *** 10 0.999990 --- --- --- --- --- --- --- --- --- --- Legend *** indicates Service Places perceived as congested --- indicates Service Places not perceived as congested

37. Interpretation • Service Places 6 to 10 should be considered congested by all agents since they were configured in that condition. • Agents 1 - 9 did perceive Service Places 6 to 10 as congested. • Agents 1 to 5 were to conservative according to their tolerance for delay • Agent 10 was greedy according to its lack of constraint on delay.

38. Perceiving Reliabilityagain note need for origin and limit values

39. The Bath Tub Function General definition of the bathtub-function f(x) = β*x(β-1)*e(α1*x)[1] This is a special combination of the Weibull rate and the log linear rate. β is a shaping parameter, which is responsible for the curve and α1 is a nuisance parameter for fine-tuning the curve (it “controls” the Wear-Out-Phase). Derivative of the Bathtub-function f΄(x) = β2*xβ–2 *eα1*x- β*xb-2*eα1*x + β *xβ-1* α1*eα1*x [2]

40. Mortality Sensing Requirements • The agent must know its chronological age since birth • The agent must know its current half life and (origin,limit) in order to determine its “indifference time” from the mortality function i.e. at the limit all solutions are unacceptable. Origin may be set to zero. • The agent must know αiβi from the Trader for the SP or service being evaluated. These are provided from the Sponsor’s empirical experience.

41. Perceiving Mortality • mortality age y = βix(βi-1)*e(αix) • membership in the fuzzy failure set is determined

42. Mortality should be contrasted to the stability of the entity’s mortality age • f’(x) is a kind of stability factor. It indicates the amount of change of the expected mortality, which might be important if the agent is interested in a long-term cooperation or is risk averse. A positive value will indicate an increasing risk, a negative value a decreasing risk.

43. Stability of the mortality age relative to the entity is given f’(x) =βi2xβ i–2 *eαix-βi*xbi-2*eαix + βxβi-1αieαix

44. Age : 1.0 AgentMortalityValue: 0.60 EntityMortality : 0.51 FailureSetMembership : 0.86 Stability : -0.18 Age : 1.5 AgentMortalityValue: 0.60 EntityMortality : 0.46 FailureSetMembership : 0.76 Stability : -0.07 Sample Output

45. Perceiving Difference • The natural log function can be useful in assessing “real differences” between attributes or entities f(x) = log x [1] f΄(x) = 1/x [2] x = e y (inverse) [3]

46. We still need origin and limit • Let origin = 0, limit be the point of indifference • Let x be the sensed value • Let y be the preference value • To determine the “perceived difference” between sensed value and preferred value dx = ey , i.e. transform y to x not x to y

47. Compute membership value and stability of perceived value • fuzzy membership in the “difference” set is x/dx. • stability of the perception is 1/x XValue = servicePlace->xValue; YValue = logf(XValue); AgentYValue = this->maxYValue; AgentXValue = exp(AgentYValue); StrengthConsiderability = XValue/ AgentXValue; if (StrengthConsiderability>1)StrengthConsiderability=1;

48. Sample Perceptions ServicePlace[4] XValue: 1.50 YValue: 0.41 AgentYValue: 1.00 AgentXValue: 2.72 StrengthConsiderability:0.55 Stability: 0.67 ServicePlace[5] XValue: 2.00 YValue: 0.69 AgentYValue: 1.00 AgentXValue: 2.72 StrengthConsiderability:0.74 Stability: 0.50 ServicePlace[6] XValue: 3.00 YValue: 1.10 AgentYValue: 1.00 AgentXValue: 2.72 StrengthConsiderability:1.00 Stability: 0.33

49. 2003 Activities • Select method of fuzzy reasoning for mobility decision and validate • Determine COS function using first order queuing approximations • Implement essential ALFworld agents and Trader Update policy • Begin work on design pattern • Begin work on genetic representation of mobile agent