Model-based Programming as Estimating, Planning and Executing based on Hidden State

1. Model-based Programming as Estimating, Planning and Executing based on Hidden State Brian C. Williams Artificial Intelligence and Space Systems Labs Massachusetts Institute of Technology

2. JSC BIO-Plex Mars Entry, descent & Landing Objective Create a hybrid estimation, monitoring, diagnosis and model learning capability for physical devices that exhibit complex discrete and continuous behaviors. DEMONSTRATION:

3. NASA BIO-Plex • Support: • NASA IS A Hybrid Discrete/Continuous System for Health Management • Failures can manifest themselves through a coupling of a system’s continuous dynamics and its evolution through different behavior modes • must track over continuous state changes and discrete mode changes • Symptoms are initially subtle; on the same scale as sensor/actuator noise • need to extract mode estimates from subtle symptoms

4. x 0.6 Continuous Dynamics x 0 Hidden Markov Models m m m m1 r1 r2 r3 x 0.6 x 0 t11 0 x x x 0 0.6 0.6 0.9 t13 0.9 t12 t21 u 0.1 0.1 m3 c1 t22 m m u t23 t33 r4 r5 PHA m2 d1 u d2 Concurrent Probabilistic Hybrid Automaton (CPHA): internal variable PHA component y c1 w y c1 continuous input uci c2 A A A 1 2 3 output / observed variable yci (cont.) discrete input udj CPHA Hybrid Plant Model for HME

5. Hybrid Mode / State Estimation CPHA Model ^ Hybrid Mode Estimator Xk estimated mode/state x= {xd ,xc} and its belief state h[x] yc(k) KalmanFilter Bank ^ sensor signals ycand control inputs uc , ud xci(k) Pi(k) uc(k-1) Ck Hybrid State Estimator Maintains the set of most likely hybrid state estimates as a set of trajectories. Hybrid Mode Estimator: Determines for each trajectory the possible transitions, and specifies (dynamically) the candidate trajectories to be tracked by the continuous state estimators.

6. How to handle the exponential blowup? • Generalize beam search to track the mostpromising hybrid states. • Factor state space into lower dimensional subspaces through automated decomposition and filter synthesis. Hybrid Mode / State Estimation • HMM-style belief state update determines the likelihood for each discrete mode transition. • Kalman-filter-style updatedetermines likelihood of continuous state evolution. X+k-1={mj,xk-1} newestimate: Xk={mj,xk} old estimate: Xk-1={mi,xk-1} # transitions at each time step is very large: e.g. model with 10 components, each with 3 successor modes has 310 =59049 possible successor modes for each trajectory!

7. PGC 560 6 540 4 520 CO2 concentration [ppm] mode number 500 2 480 460 0 850 900 950 1000 1050 1100 1150 1200 850 900 950 1000 1050 1100 1150 1200 time [minutes] time [minutes] Lighting System 6 4 mode number 2 0 850 900 950 1000 1050 1100 1150 1200 time [minutes] Simulation Result components: 6 ( FR1, FR2, PIV1, PIV2, LS, PGC) total # of modes: 9600 fringe size: 20 (400 estimation steps): average candidates: 90.2 (< 1% !)max. candidates: 428 (< 5 %!)filter calculations: 242filter executions: 36050 average runtime: ~1 s/step (PII-400, 128mb)

8. Future Directions • Model-Learning as Hybrid EM • Automated Decomposition of HPCA using Dissents • Model-based Hybrid Execution

9. Automated Decomposition x v y u o1 c1 c1 c1 x v v v y c2 s1 s2 s3 v y MIMO Filter u w c1 x o2 c2 c1 c1 y c3 u A A A c2 1 2 3 d1 P u O d2 CA v u v x s1 o1 c1 c1 Filter 1 y P u c1 A y A 1 O c1 1 2 c1 u x c1 c2 v v v y s3 s2 x o1 v Filter 2 c1 c3 y o2 y c1 y P A c2 u c2 3 2 O c1 Filter Cluster

10. To Address the Scope of Mars 98 • Polar Lander Leading Diagnosis: • Legs deployed during descent. • Noise spike on leg sensors latched by software monitors. • Laser altimeter registers 50ft. • Begins polling leg monitors to determine touch down. • Latched noise spike read as touchdown. • Engine shutdown at ~50ft. Responding to the failures of Mars Polar Lander and Mars Climate Orbiter is a Hybrid control problem. Idea: Support programmers with embedded languages that avoid commonsense mistakes, by reasoning from hardware models. Reactive Model-based Programming

11. Model-basedControl Programs Mars Entry, Descent & Landing S’ Plant Model Model-based Executive Demonstration: Discrete Controller Discrete Mode Est. Cntrl Obs Continuous Controller Continuous State Est. S Plant State Reconfiguration State Estimation Hybrid Model-based Programming • Hybrid Executives: • Can hook into existing estimation and control approaches. • Should target “comfort zone” of systems engineers. RMPL Hybrid Model-based Executive Control Program Sequencer cont & discr state estimates h/w config goals att/pos goals Plant Model Estimation & Control Engines Plant Control actions Observations

12. Recent Publications Hybrid Mode Estimation: • Hofbaur, M. W. and B.C. Williams, “Mode Estimation of Probabilistic Hybrid Systems,” International Conference on Hybrid Systems: Computation and Control, March, 2002. Hybrid Expectation Maximization (preliminary): • Melvin Henry, Simulators that Learn: Automated Estimation of Hybrid Automata, June 2002 Hybrid Decomposition: • Hofbaur, M. W. and B. C. Williams, “Hybrid Diagnosis with Unknown Behavioral Modes,” International Workshop on Principles of Diagnosis, Austria, May 3-5 2002.