Reactive Systems

1. Reactive Systems Yolanda Gil CS 541, Fall 2003 (Thanks to Karen Myers from SRI International)

2. The problem with plans (I) Attack Goliath • Gather pile of rocks • Grasp slingshot • Fire at giant • Hit on the head

3. The problem with plans (II) • Unknown how many stones • Unknown if stones • Unknown how many attempts • Conditions for termination • What if failure • Check state Attack Goliath • Gather pile of rocks • Grasp slingshot • Fire at giant • Hit on the head

4. Reactive Systems • Embedded in the real world • Have sensors and effectors • Actively test the external environment • Need to respond to events in dynamic environments • Failure may require aborting and generating new response • Do we need deliberate reasoning (planning)?

5. Outline and Informal Roadmap • Control systems • Networks of “variables” (arcs) and “functions” (nodes) • Reactive Action Packages (RAPs) • Networks of “conditions” and “tasks” • Task Control Architecture (TCA) • Network arranged according to “vertical capabilities” • Procedural Reasoning System (PRS) • Integrates planning, BDI, and reactive techniques • Anytime algorithms • When time is short, managing what you think about • Other approaches and issues

6. Readings • RAP (http://people.cs.uchicago.edu/~firby/raps) • Firby, J “Task Networks for Controlling Continuous Processes”, Proceedings of Artificial Intelligence Planning conference, 1994. • TCA (http://www-2.cs.cmu.edu/afs/cs/project/TCA/release/tca.orig.html, http://www-2.cs.cmu.edu/afs/cs/project/TCA/release/tca.html) • Simmons, R. “Structured Control for Autonomous Robots”, IEEE Transactions on Robotics and Automation, Feb 1994. • PRS (http://www.ai.sri.com/~prs) • Reactive reasoning and planning: an experiment with a mobile robot, M. Georgeff and A. Lansky, in Proceedings of AAAI, 1987. • Anytime algorithms • Zilberstein, S. “Using Anytime Algorithms in Intelligent Systems”, AI Magazine, 1996.

7. Control Systems: An Example (I)Control of temperature profile for a spray deposition process. Jones, P.D.A.; Duncan, S.R.; Rayment, T.; Grant, P.S. IEEE transactions on control systems technology special issue on control of industrial spatially distributed processes, Sept 2003.

8. Control Systems: An Example (II)Control of temperature profile for a spray deposition process. Jones, P.D.A.; Duncan, S.R.; Rayment, T.; Grant, P.S. IEEE transactions on control systems technology special issue on control of industrial spatially distributed processes, Sept 2003.

9. Beyond Stimulus-Response • Address problems that require a combination of: • Coordinated activity to accomplish tasks • Reactivity to world dynamics • Situate control decisions within an explicit, persistent decision-making framework

10. Reactive Action Packages (RAP)

11. A Symbolic Discrete Task

12. Waiting for a signal to proceed

13. Concurrent tasks

14. More Complex Task Networks

15. Task Control Architecture (TCA) • Vertical task decomposition: several task-specific modules communicate through a central control module • Deliberation: top-down task-subtask, resolve constraints • Central control routes messages • Inform, query, command, monitor

16. Ambler Walking Robot

17. Ambler Modules

18. Ambler Task Tree

19. TCA: Monitoring • Central control traverses tree and handles messages: • asks gait planner to traverse arc, • gait planner asks terrain mapper for elevation map in order to take steps • Gait planner asks leg recovery planner to place leg, move leg, move body, • Gait planner activates monitor whether achieved position

20. TCA: Control • Ordering and temporal constraints • Delay planning constraint: goal cannot be issued until previous task achieved • Can do place leg planning while monitoring achieve position • Exception handling: error recovery modules examine and modify task trees • Eg: if position not achieved, add take steps subtask

21. Ambler Planning and Execution

22. An Alternative to TCA’s Vertical Capabilities:Horizontal Layered Control Reason about behavior of objects Plan changes to the world Identify objects Monitor changes Build maps Explore Wander Avoid objects

23. Procedural Reasoning System (PRS) • Framework for symbolic reactive control systems in dynamic environments • Eg Mobile robot control • Eg diagnosis of the Space Shuttle’s Reaction Controls System

24. PRS: Main Features • Pre-compiled procedural knowledge • BDI (Belief, Desires, Intentions) foundation • Combines deliberative and reactive features • Plan selection, formation, execution, sensing • Plans dynamically and incrementally • Integrates goal-directed and event-driven behavior • Can interrupt plan execution • Meta-level reasoning • Multi-agent planning

25. PRS Architecture User Tasks Procedures Interpreter Database Intentions World

26. PRS Architecture: Database • Contains beliefs or facts about the world • Includes meta-level information • Eg goal G is active User Tasks Procedures Interpreter Database Intentions World

27. PRS Architecture: Tasks • Represent desired behavior • Conditions over some time interval • eg (walk a b): set of behaviors in which agent walks from a to b) User Tasks Procedures Interpreter Database Intentions World

28. Expressing Tasks in a Dynamic Environment • (! P) -- achieve P • (? P) -- test P • (# P) -- maintain P • (^ C) -- wait until C • (-> C) -- assert C • (~> C) -- retract C

29. PRS Architecture: Intentions • Currently active procedures • Procedure currently being executed User Tasks Procedures Interpreter Database Intentions World

30. PRS Architecture: Procedures • Pre-compiled procedures • Express actions and tests to achieve goals or to react to conditions User Tasks Procedures Interpreter Database Intentions World

31. Representing Procedures with Act Formalism • Environment conditions • Purpose (goal or condition) • applicability criteria • Plot • directed graph • partially ordered conditional & parallel actions, loops • Successful node execution by achievement of node’s goals • If no body: primitive action Metapredicates • Achieve – Achieve-By {proc} • Test – Conclude {effects} • Wait-Until – Use-Resource • Require-Until Cross-Country Delivery Cue: (ACHIEVE (DELIVER CUSTOMER.1 GOODS.1)) (ACHIEVE (RECORD-INVOICE Preconditions: CUSTOMER.1 GOODS.1 (TEST INVOICE.1) ) (AND (LOCATED CUSTOMER.1 CITY.2) (LOCATED GOODS.1 CITY.1) (DISTANCE CITY.1 CITY.2 DISTANCE.1) (> DISTANCE.1 1000) ) ) (ACHIEVE-BY (ACHIEVE-BY (LOCATED (LOCATED Setting: AIRCARGO.1 LANDCARGO.1 CITY.2) CITY.2) (TEST SHIP-BY-AIR) ) SHIP-BY-RAIL) ) (AND (AIR-SHIPMENT AIRCARGO.1 GOODS.1) (LAND-SHIPMENT LANDCARGO.1 GOODS.1) ) ) Resources: (ACHIEVE (LOCAL-DELIVERY - no entry - CUSTOMER.1 GOODS.1) ) (CONCLUDE Propertities: (COMPLETED-INVOICE INVOICE.1) ) (AUTHORING-SYSTEM ACT-EDITOR) Comment: Long distance delivery of goods to customers

32. New Facts & Tasks 2 (overpressurized fuel-tank) 7 (ACHIEVE (position ox-valve closed)) Act Library 1 ACT2 Cue: Act Execution (TEST (overpressurized tank.1)) Facts & Tasks ACT1 Cue: (ACHIEVE (position valve.1 closed)) 6 External World 8 5 Goal2 Goal3 ACT8 ACT3 3 sleeping sleeping (ACHIEVE (position ox-valve closed)) ACT1 current Fact1 ACT2 4 normal Intention Graph PRS Interpreter Execution Cycle 1. New information arrives that updates facts and/or tasks 2. Acts are triggered by new facts or tasks 3. A triggered Act is intended 4. An intended Act is selected 5. That intention is activated 6. An action is performed 7. New facts or tasks are posted 8. Intentions are updated

33. Meta-Reasoning • Can include meta-level procedures • eg: choose among multiple applicable procedures • eg: evaluate how much more reasoning can be done within time constraints • eg: how to achieve a conjunction or disjunction of goals

34. Shuttle’s RCS Malfunction Handling RCS Controls • Automates specification and execution of RCS malfunction procedures. • Reacts to changes in RCS. Ensures safe operation while carrying out diagnosis and remediation procedures. RCS Jets Jet Fail - On Achieve: Position valve.ox closed, Cue Position valve.fu closed Test: Alarm sounding, Shuttle MESSAGES RCS warning light on, GPC Achieve: Status RCS jet.1 is failed-on, Notify "Thruster jet.1 failed-on" GPC displays dir.1 for jet.1 for rcs.1 Preconditions Test: External External Test: Test: Direction jet.1 is dir.1 TASKS FACTS Not high-usage of jet.1 High-usage of jet.1 Regulator Test Setting Test: Jet Fail - On Connected manifold.ox to jet.1, FACTS Test: Test: Dump Propellant & Connected manifold.fu to jet.1, TASKS Not type jet.1 vernier Type jet.1 vernier BELIEFS Connects valve.fu by leg.fu to manifold.fu, Connects valve.ox by leg.ox Achieve: to manifold.ox , Pressure manifold.ox is pres.ox , Oxidizer-subsystem ox.1 of rcs.1 , Executing Pressure manifold.fu is pres.fu procedures can post Fuel-subsystem fu.1 of rcs.1 , Procedure GOALS, FACTS, & Part valve.ox of ox.1 , Library BELIEFS Part valve.fu of fu.1 or Test: Test: send MESSAGES Determine new Achieve: > pres.ox 130, ≤ pres.ox 130, procedures Notify "Thruster jet.1 failed-on > pres.fu 130 ≤ pres.fu 130 that are eligible Jet Fail - On ELECTRICALLY" for execution Regulator Test Achieve: Achieve: Notify "Thruster jet.1 failed-on Notify "TURN-OFF rcs.1 manifold.ox INPUT CARD" & manifold.fu DRIVER" Select procedures for execution

35. Multiple Tasks, Multiple Agenst • Multithreaded operation: multiple tasks being performed, runtime stacks where tasks are executed, suspended, and resumed • Supports distributed planning: several PRS agents run asynchronously and communicate through message passing

36. Anytime Algorithms • Time to deliberate about events varies • Algorithms to compute the best answers they can in the time available • Anytime algorithms • Can be suspended and resumed with little overhead • Can be terminated at any time and return some answer • The answers returned improve with time

37. A time-dependent planning problem • Observe (O) • React (E): time required to carry out reaction of type E • Herald (C): earliest observation time that enables prediction of condition C requiring a response • Utility (C,E): utility of reacting to with E to C • Response (C): time between having information to predict C and C occurring

38. When Time is Short… • Prediction time: time required to predict event given info available • Deliberation time: max time for committing to a reaction (if reaction is needed) • Reaction time: time required to react to event • React(E) + Response(C)

39. Deliberation • Decision procedure D for each C: given t time to deliberate, D returns best guess E about how to react • Utility(C, D(C,T)) • Deliberation scheduling: • Given several deliberation procedures, determine how to best allocate deliberation time

40. Utility versus time Linear improvement, bounded utility One-shot improvement Linear improvement, unbounded utility Diminishing returns

41. Other Approaches and Issues • Blackboard architectures (Guardian) • Universal plans • Related issues covered in the course: • Reasoning about uncertainty • Learning • from the environment • Becoming increasingly reactive

42. Summary • Control systems • Networks of “variables” (arcs) and “functions” (nodes) • Reactive Action Packages (RAPs) • Networks of “conditions” and “tasks” • Task Control Architecture (TCA) • Network arranged according to “vertical capabilities” • Procedural Reasoning System (PRS) • Integrates planning, BDI, and reactive techniques • Anytime algorithms • When time is short, managing what you think about • Learning and uncertainty reasoning