Johann Schumann and Pramod Gupta NASA Ames Research Center schumann@email.arc.nasa

1. Bayesian Verification & Validation tools for adaptive systems Johann Schumann and Pramod Gupta NASA Ames Research Center schumann@email.arc.nasa.gov pgupta@email.arc.nasa.gov

2. Motivation for NN V&V • Fixed gain controllers cannot deal with catastrophic changes or degradation in plant • Adaptive systems (e.g., NN) can react to unexpected situations through learning • Relevance and potential: • IFCS NN controlled aircraft (F-15, C-17) • UAV control • Space exploration • Any safety-critical application of NN control • Basis for Case Study I: • Neuro-adaptive control (IFCS Gen-II) • Network “learns” to compensate for deviations between plant and model • Previous work: • SW V&V process for NN-based control • “Confidence tool” for dynamic monitoring The major obstacle to the deployment of adaptive and autonomous systems is being able to verify their correct operation – In Flight

3. V&V Issues & our Approach • Verification: how to specify an unforseen event? • Validation: not possible to test all configurations While traditional V&V methods will remain useful, these methods alone are insufficient to verify and certify adaptive control systems for use in safety-critical applications • Our approach combines mathematical analysis, intelligent validation, and dynamic monitoring and supports specific software V&V process, • targets multiple aspects and phases of V&V of adaptive control systems, and • uses a unique combination of research in • Neural Networks • Control Theory • Numerical Methods • Bayesian Statistics

4. Our Bayesian Approach How good is the network performing at the moment? • Traditional: NN as a Black Box • Here: Look at probability distribution of the NN output • Variance (confidence measure) depends on: • How well is the network trained? • How close are we to “well-known” areas Small variance = good estimate Large variance = bad estimate; no reliable result, just a guess Our approach, based on a Bayesian approach, provides a measure of how well the neural network is performing at the moment

5. Milestone I: Envelope Tool • Basis: Adaptive NN-based controller • Lyapunov error bound defines regions of eventual stability • Regions where confidence is small might cause instability • Informally: a safe envelope is a region where the confidence level is sufficiently high • Bayesian approach combined with sensitivity analysis • Challenge: methods for efficient determination of safe envelope • Can help answer questions like • How large is the current safe envelope? • How far is the operational point from the edge? Current status: mathematical background formulated, prototypical Matlab/Simulink implementation designed, first simulation experiments

6. Confidence Envelope Confidence Surface bad Safety Envelope: area of good confidence 1/confidence good airspeed altitude The Envelope tool uses a Bayesian Approach to calculate the current safety envelope

7. Conclusions & next steps • Current work as scheduled toward deliverable (9/2004) • prototypical implementation in Matlab/Simulink • report on mathematical background and tool • Getting Case Study I ready: IFCS Gen-II simulink model • Next steps in research: • system identification (sysID): estimate confidence of parameters • other model representations (e.g., parameter tables with polynomial interpretation) • Preparation of Case Study II and III