Temporal Probabilistic Agents

1. Temporal Probabilistic Agents J. Dix, S. Kraus, VS Subrahmanian Clausthal (DE)/ Bar-Ilan(IL)/ Maryland (USA) 1

2. Motivation • Agents must be able to leverage existing code rather than reinvent everything from scratch. • Agents must be able to reason about • State • Time • Uncertainty in the state

3. Applications • Built 4 applications that motivated the need for this theory. 2 examples: • US Navy: Finding where and when an enemy submarine will be in the future, and determining which of many friendly assets will try to attack it. Joint with BBN, Lockheed, NRL and many other partners. • Terrain DB • Sensor DB • Predictive Models • Route Planner • US Army: Finding where and when an enemy tank will be in the future, and determining which of many coalition assets will try to attack it, where, and when. Joint with BAE. • Terrain Mobility Map • Sensor DB • Optimization code • Route/flight Planner

4. Overview • Part I: A Bird’s Eye View on IMPACT • Part II: Adding time • Part III: Adding uncertainty • Part IV: Adding time and uncertainty

5. I.1 Motivations • Agents are for everyone!Agentizearbitrary Legacy Code. • Knowledge is distributed and heterogenous. Code-Call machinery • Agents act wrt a clearly articulated semantics.Agent Program

6. I.2 Data Access • Code Call:d:f(arg1,…,argk). • Code Call Atom: in(X,d:f(arg1,…,argk)). Execute function f defined in agent d on the specified arguments.Returns a set of objects. • stock:portfolio( ‘person’). • pred:dest(‘stock’) Succeeds if X is in the set of objects returned • in(X,stock:portfolio( ‘dix’)). Find all stocks in dix’ portfolio. • in(X,pred:dest( ‘ibm’)). Find all pairs (T,C) s.t. the price of ibm will change by C% within the next T units of time.

7. I.3 Data Access: Code Call Conditions • A conjunction of • code call atoms and • comparison atoms. • in(X, pred:dest( S )) & X.percent> 30& X.time < 2 weeks&in(S,stock:portfolio( dix)). • The above says: find X,S such that: • S is a stock in the portfolio of dix, • X is a prediction that the stock S is going up by at least 30% within the next two weeks.

8. I.4 What is an Agent? Interface/API Implementation • Agents are built on top of legacy or specialized software modules or systems. • They access data using API functions. Software module

9. I.4 What is an Agent? Interface/API Implementation msg box Agent state (distributed) • There is a message box with suitable functionality. • Code manipulates data structures, the contents of which (including msgbox) describe the agent’s state.

10. I.4 What is an Agent? Interface/API Implementation Actions msg box Agent state (distributed) • Agents have available actions to change the state. • Actions are declaratively described through add/delete lists containing code call conditions. They are implemented through software code.

11. I.4 What is an Agent? Interface/API Implementation Agent Program Actions msg box Agent state (distributed) • Agents have an “agent program” encoding the agent’s operating principles • The program consists of rules that are declarative and can be easily written or modified.

12. I.4 What is an Agent? Interface/API Implementation Agent Program Actions msg box Integrity Constraints Action Constraints Agent state (distributed) • Integrity Constraints (to preserve properties of the state) • Action constraints (to ensure actions do not interfere)

13. I.4 What is an Agent? Interface/API Implementation Agent Program Actions Actions msg box Integrity Constraints Action Constraints Semantics Agent state (distributed) • Based on semantics and current state, actions are executed. • These actions change its own state and other agents states as well.

14. I.5 Agent Program (1) Set of rules of the form Opa(arg1,…,argn)<--- <code call condition> &Op1a1(<args>) & … & Opan(<args>) • Op is a “deontic modality” and is either • P - permitted • O - obligatory • Do - do • F – forbidden • If code call condition is true and the deontic modalities in the rule body are true, then Opa(arg1,…,argn) is true.

15. Most important part of the agent. Agent program rules must be carefully crafted to avoid inconsistencies. IMPACT provides facilities to create such rules. A class of programs is defined that can be efficiently implemented: Regular Agents. EXAMPLE: Dobuy(S) in(X, pred:dest( S )) & X.percent> 30 & X.time < 2 weeks Fbuy(S) in(S, stock:portfolio(dix)). & Dodiversify_portfolio(dix) I.5 Agent Program (2)

16. II Adding Time • Basic framework does not allow to express time. • If a prediction package expects a stock to rise K% after T units of time and K>25, then buy the stock some units between (tnow + T –5) and (tnow + T –2). Dobuy(S): [(tnow + T –5),(tnow + T –2)]<--- in( ...): [tnow,tnow] • Semantics: Time is 1, 2, 3, ... • We introduce temporal annotations [t1,t2]. Semantics needs to be defined for all timepoints. • Semantics has to reflect the obligations and committments (into the future) made in the program. • (see Dix/Kraus/Subr.: Temporal Agent Programs, AIJ 2001)

17. III TP Adding Probabilities • Basic framework does not allow to express uncertainty. • Requires changes in the code call mechanism: Code calls return random variables. • Status atoms are true only with a certain probability. • This is modelled by probabilistic annotations. • Evaluating code call conditions is difficult: What is the probability of a conjunction of events? (see Dix/Nanni/Subr.: Probabilistic Agents, TOCL 2000)

18. IV Adding both Time and Probabilities • Most commercial prediction packages compute (based on observations of the market) probability intervals linked with time intervals: • If a stock is expected to go over $50 per share (with 80% probability) in the next two weeks, then we should buy it. If the probability is less than 40%, we should not.

19. IV. 1 TP Agents: Annotations • Define annotations carrying both temporal and probabilistic information:[strategy,tc,l,l‘,pdf] • strategy: determines probability of conjunctive events • tc: temporal constrainttnow+5<t< tnow • l,l‘: probabilistic items valued in [0,1], e.g X, (X+1)/2 where X is a RV with values in [0,1] • pdf: probability distribution over solutions oftc Dobuy(ibm):[strat, tnow +5<t< tnow, 0.5,0.7,uniform]

20. IV.2 TP Agent Program • Set of rules of the form Opa(arg1,…,argn) [ ]<--- <code call condition> [ ] &Op1a1(<args>) [ ]& …& Opan(<args>)[ ] where [ ] are temporal probabilistic annotations. • Formal definitions of what it means that a <code call condition> [ ] is true in a state. • SEMANTICS: Fixpoint operator associating a meaning to TP agent programs.

21. IV. 2 TP Agents (2) Dobuy(X):[s, tnow +5<t< tnow, 1, 1,u]  in(X,stock:portfolio( dix)): [s, tnow, 1, 1,u]& in(yes, stock:over(50,X )): [s, tnow<t< tnow+14, 0.8, 1,u] • We defined formal semantics and algorithms for computing feasible status sets of TP programs.(This is quite involved due to action histories, deontic/state consistency, machinery of fixpoint operators.) • Complexity:Existence of a feasible SS is NP-complete. • Implementation within IMPACT is on its way.

22. Conclusions • Developed an agent programming language that allows us to build agents that • Leverage commercial software without reinventing it; • Take actions in accordance with an agent program and integrity constraints in the presence of time and uncertainty about the state of the world • Developed a declarative semantics and the concept of a feasible TPSI. • Developed fixpoint computation algorithm to find feasible TPSis when they exist.