Autonomy

1. Autonomy Model-based Embedded and Robotic Systems Group (MERS) Graduate Open House

2. The Need for Autonomy Europa Probe • Space systems must handle • Faults and anomalies • Cooperative exploration • Long-term science operations • Increasingly ambitious goals MER Memory Leak Apollo 13 quintuple fault Mars Outpost Earth Imager Mars Polar Lander failed due to a faulty sensor.

3. Future Autonomous Vehicles Space Technology 3 Europa Hydrobot In-Situ Propellant Plant Orbital Space Plane Mars Life Support Facility 2009 Mars Science Lab

4. Mission Collaboration MER Shadow Mode Orbital Express MIT SPHERES Earth Observing-1 Mars Science Laboratory 2009 Images courtesy of NASA JPL

5. Embedded programs interact withthe system’ssensors/actuators: Read sensors Set actuators Model-based programs interact with the system’s hidden state: Read state Set state S Plant S Plant S’ Model-based Executive Model-based Embedded Program Robust Systems Should be“Fully State Aware” Embedded Program Control Observations Observations Control • Programmer must map between hidden state and sensors/actuators. • M-B Executive maps between hidden state and sensors/actuators.

6. Example: Orbital Insertion Scenario Engine Model (thrust = zero) AND (power_in = zero) Off 0.01 off- cmd Failed standby- cmd (thrust = zero) AND (power_in = nominal) 0.01 EngineA EngineB EngineA EngineB Standby fire- cmd standby- cmd 0.01 (thrust = full) AND (power_in = nominal) Firing Science Camera Science Camera Systems engineers think in terms of state trajectories: Camera Model Off • must fire one of the two engines • set both engines to ‘standby’ • prior to firing engine, camera must be turned off to avoid plume contamination • in case of primary engine failure, fire backup engine instead (power_in = zero) AND (shutter = closed) turnoff- cmd turnon- cmd (power_in = nominal) AND (shutter = open) On

7. Model-based Program • Control program specifies state trajectories: • Concurrency • Preemption • Queries hidden state • Asserts (assigns) hidden state OrbitInsert():: (do-watching ((EngineA = Firing)OR (EngineB = Firing)) (parallel (EngineA = Standby) (EngineB = Standby) (Camera = Off) (do-watching (EngineA = Failed) (when-donext( (EngineA = Standby) AND (Camera = Off) ) (EngineA = Firing))) (when-donext ( (EngineA = Failed)AND (EngineB = Standby) AND (Camera = Off) ) (EngineB = Firing)))) • Plant Model describes behavior of each component: • Nominal and off-nominal • Qualitative constraints • Likelihoods and costs Models are reusable and easy to articulate at the conceptual stage

8. Valve Stuck open 0.01 Open 0. 01 Open Close 0. 01 Stuck closed Closed 0.01 inflow = outflow = 0 Titan Model-based Executive Model-based Executive RMPL Control program Sequencer / Planner Generates target goal states conditioned onstate estimates • Executes concurrently • Preempts • Queries (hidden) states • Asserts (hidden) state State estimates State goals System model Mode Estimation tracks likelyplant state Mode Reconfiguration tracks least costgoal states Commands Observations Plant

9. Example: The model-based program sets the state to thrusting, and the M-B executive . . . Oxidizer tank Fuel tank Plans actions to open six valves Deduces that thrust is off, andthe engine is healthy Deduces that a valve failed - stuck closed Determines that valves on the backup enginewill achieve thrust, andplans needed actions.

10. Purpose: Ideal mode estimation would maintain a complete belief state Belief State: Probability distribution across all combinations of possible states in the system Challenge: Combination of states is exponential in the number of modes Solution: Simplifying assumption to track only k trajectories (shown below) reduces the space requirement to linear! RMPL Model-based Executive sw1 OR Control Program Sequencer LED sw2 State estimates Configuration goals System Model Mode Estimation Mode Reconfiguration out = high 0.97 out = low Nom On Off cmd = turnOff cmd = turnOn 0.03 0.05 0.05 Bkn Bkn 1 Flight System Control sw1 OR Unconstrained Observations Commands RT Control Layer Mode Estimation Concurrent Constraint Automata Switch and OR-Gate System • Tracking a history of the system state over time t=0 t=1 t=2 • cmd = sw1-turnOn • Obs = LED-off • cmd = sw2-turnOn • Obs = LED-off sw1=bkn, sw2=off, or=nom sw1=off, sw2=off, or=bkn likelihood sw1=off, sw2=off, or=nom sw1=off, sw2=bkn, or=nom sw1=bkn, sw2=bkn, or=nom sw1=off, sw2=off, or=bkn sw1=bkn, sw2=bkn, or=bkn sw1=bkn, sw2=bkn, or=nom …

11. Dissent: A mapping from observations to conflicts sw1 OR LED sw2 0.03 0.97 (or=nom) (or=bkn) (sw1=on) (sw1=bkn) (sw1=off) 0.922 0.049 (sw2=on) (sw2=off) (sw2=on) (sw2=bkn) (sw2=bkn) 0.046 0.046 0.002 Compiled Mode Estimation Compiled Mode Estimation Online Offline Projected Prime Implicate Generation Dissents Best-first Conflicts Partial Most Likely Kernel System Diagnosis Diagnosis Diagnosis Model Trigger Enabled Modes Generator Compiled Transitions Discrete Continuous Monitors Observations Observations • Off-line Operations (Removes the need for NP-complete online satisfiability) (LED=off)  sw1=on  sw2=on  or=nom .... Model Compilation • On-line Operations (Reduced to an optimal search instead of OCSP) • Most Likely Diagnosis: • Or-gate = Nominal • Switch1 = On • Switch2 = Broken Partial Diagnosis Trigger (LED=off)

12. failures can manifest themselves through coupling between a system’s continuous dynamics and its evolution through different behavior modes must track over continuous state changes and discrete mode changes symptoms initially on the same scale as sensor/actuator noise need to extract mode estimates from subtle symptoms Hybrid Model Continuous Dynamics Hidden Markov Models m1 t11 t13 t12 t21 m3 t22 t23 t33 m2 Hybrid Mode Estimation old estimate: Xk-1={mi,xk-1} newestimate: Xk={mj,xk} X+k-1={mj,xk-1} yc(k) ^ KalmanFilter Bank ^ Xk Mode Estimation xci(k) Hybrid Mode Estimation tracks a set of trajectories Pi(k) uc(k-1) • Methods: • K-best filtering, • Rao-Blackwellised particle filtering Ck

13. Gesture recognition: Stereo vision system Tracks head and hand motion of human associate Hybrid model supports Robonaut’s recognition of human gestures Gestures of interest include pointing to a tool, holding hand up to indicate stop, “come closer” gestures, etc. Continuous dynamics model of human arm includes inertial and damping terms HMM model takes output of stereo vision system as observation Transitions between motion control point states Application: Gesture Recognition Robonaut: • EVA astronaut’s assistant • Humanoid design requires no specialized robotic tools • Controlled by teleoperator, but autonomous modes under development

14. INPUT Configuration Goal Thrust = on Current State Tank = full Pressure = nominal Driver = off Valve = closed Thruster = off RMPL Model-based Executive Control Program Sequencer Configuration goals State estimates Configuration goals System Model Current State Mode Estimation Mode Reconfiguration Goal Interpreter Goal State Flight System Control Observations Commands RT Control Layer Reactive Planner Command Mode Reconfiguration OUTPUT • Command • Turn driver on

15. Configuration goals Current State Goal Interpreter Goal State Goal Interpreter INPUT • Current State • Tank = full • Pressure = nominal • Driver = off • Valve = closed • Thruster = off • Configuration Goal • Thrust = on OUTPUT • Goal State • Tank = full • Pressure = nominal • Driver = off • Valve = on • Thruster = on Generate optimal goal state that achieves the Configuration Goal! • Goal Interpreter • Compiled Goal Interpreter Minimize online deduction by generating all partial goal interpretation offline! Online: Goal Configuration Goal State Partial Goal Interpretation Best-first Kernel Goal State Generator

16. Goal State Current State Reactive Planner Command Reactive Planner INPUT • Current State • Tank = full • Pressure = nominal • Driver = off • Valve = closed • Thruster = off • Goal State • Tank = full • Pressure = nominal • Driver = off • Valve = on • Thruster = on OUTPUT • Command • Turn driver on • Planner guarantees to: • Only generate non-destructive actions • Never propose actions that lead to dead-end plans • Ensure progress toward the goal • Operate at reactive time scale Driver Valve Reconfiguration Order • Tank = full • Pressure = nominal • Valve = on • Thruster = on • Driver = off Goal Goal Current Current On Open Off Closed idle cmd = off idle driver = on cmd = close On Open cmd = on idle driver = on cmd = open idle Off Closed cmd = reset cmd = off fail fail Resettable Stuck

17. Verification of RMPL Programs • Motivation: • Want robust autonomous systems. • Extend traditional scenario-based testing to verification and validation (V&V). Approach: • Goals: • Verify RMPL model-based programs (control program + plant model) against goal specification. • e.g., ((EngineA = Firing)OR (EngineB = Firing)) for OrbitInsert() • Extract probabilistic information about program’s possible executions.

18. Programming Cooperative TeamsTo Perform Global Science TWO ONE HOME Enroute COLLECTION POINT RENDEZVOUS Landing Site: ABC Diverge Landing Site: XYZ SCIENCE AREA 1’ SCIENCE AREA 1 SCIENCE AREA 3 • Mission controller specifies abstract set of goals for a robot team • System must handle: • Low-level planning and execution • Dynamic environment • Inter-vehicle communication

19. Heterogeneous Robots • Orbiter: • Earth Com link, • Large scale feature detection • Science observation • Tethered Blimp: • Reconnaissance: Rover tracking, feature detection, local map generator • Sensor network deployment • Rover Com link • Smart Mobile Lander • Slow mobile base station • Orbiter Com link • Large science package • Scout Rovers • Fast agile rovers • Sensor package for identifying science objectives • Terrain mapping functionality • Sensor Network • Highly constrained sensing/effecting communication array • Science sensing • Aids in robot localization High Tier Mid Tier Low Tier

20. (Group-Enroute() [l,u] ( (sequence choose ( (do-watching (PATH1=OK) ((Group-Traverse-Path(PATH1_1,PATH1_2,PATH1_3,RE_POS))[l*90%,u*90%]) ) (do-watching (PATH2=OK) ((Group-Traverse-Path(PATH2_1,PATH2_2,PATH2_3,RE_POS))[l*90%,u*90%]) )) (parallel ((Group-Transmit(OPS,ARRIVED))[0,2]) (do-watching(PROCEED=SIGNALLED) ((Group-Wait(HOLD1,HOLD2))[0,u*10%])) ))) Programming Teams in RMPL RMPL Programs • Describe concurrent sensing, actuation and movements activities. • Choose specifies redundant strategies and contingencies. • [A,B] Specifies timing constraints. Corridor 2 Rendezvous Rescue Area Corridor 1 Enroute

21. Planning and Execution RMPLCompiler Represents all possible executions RMPLProgram Temporal Plan Network Temporal Planner Mission Specification (choose (parallel (power = low) [10,15]) (actB() [20,30]) ) (if-thennext-elsenext (state = true) (macroA() [5,5]) (macroB() [6,8]) )) SolutionPlan Plan Runner/ Dispatcher Hardware HardwareCommands Planning and Execution

22. Distributed Planning and Execution • Why distributed? • to avoid mission being dependent on a single planning and execution robot • distributed execution is essential when communication is limited • resource constrained robots or sensor networks cannot run a big planning system mission • Challenges • Limited communication • Plan failures • Robot failures • Temporarily flexible plans

23. Integrated Activity Planning and Path Planning: Search a temporal plan network in best-first order Dynamically compute collision-free paths for those plan activities that require moving between locations and the estimated cost of flying along this path Continuously interleave activity and path planning to pursue the most promising plan. Collision-free path Cost estimate = 20 units of fuel Cost estimate = 10 units of fuel Integrated Activity and Path Planning for Agile Teams Path Planning Method 1: Explore state space using Rapidly-exploring Random Trees (RRTs) Location A: start state Location B: goal state RRT Maneuver Automaton: Describes a set of agile maneuvers with respect to the vehicle’s dynamics Path Planning Method 2: Clausal Linear Programming

24. Path Planning through Clausal LP A simple example: • Mathematically solving the problem of vehicle control normally involves straightforward Linear Programming • But the addition of obstacle avoidance introduces an Integer Programming element • This makes the problem difficult to solve “online”: fast enough for actual vehicles in motion • To resolve this we transform obstacle and collision avoidance into a Constraint Satisfaction Problem: For each obstacle, the domain is split into four regions (above, below, left, right), one of which is selected • Integrating the selection of domains with the standard vehicle control leads to a an algorithm that can be used as a Hybrid CSP/LP Solver O2 O2 = B O1 O3 = A O3 O1 = L • si+1 = Asi + Bui State Evolution Equation • sij≤ wij, etc. State Space Constraints • xi≥ xmin xi≤ xmax yi≥ yminyi≤ ymax Obstacle Avoidance(for all time i) • Similar equation for Collision Avoidance (for all pairs of vehicles)

25. Simulate Mission Objectives of Mars ’03 Use NASA’s MERBoard to visualize the environment and control the rovers. Demonstrate the ability to achieve mission goals autonomously Mars Shadow Mode Project Rover Sensors • Stereo camera head • Laser range scanner • Sonar array • DGPS • Digital compass • Inclinometer Analyze this rock! Remote operations center Mars yard