1 / 7

Johann Schumann and Pramod Gupta NASA Ames Research Center schumann@email.arc.nasa

Bayesian Verification & Validation tools for adaptive systems. Johann Schumann and Pramod Gupta NASA Ames Research Center schumann@email.arc.nasa.gov pgupta@email.arc.nasa.gov. Motivation for NN V&V.

dunnc
Download Presentation

Johann Schumann and Pramod Gupta NASA Ames Research Center schumann@email.arc.nasa

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Bayesian Verification & Validation tools for adaptive systems Johann Schumann and Pramod Gupta NASA Ames Research Center schumann@email.arc.nasa.gov pgupta@email.arc.nasa.gov

  2. Motivation for NN V&V • Fixed gain controllers cannot deal with catastrophic changes or degradation in plant • Adaptive systems (e.g., NN) can react to unexpected situations through learning • Relevance and potential: • IFCS NN controlled aircraft (F-15, C-17) • UAV control • Space exploration • Any safety-critical application of NN control • Basis for Case Study I: • Neuro-adaptive control (IFCS Gen-II) • Network “learns” to compensate for deviations between plant and model • Previous work: • SW V&V process for NN-based control • “Confidence tool” for dynamic monitoring The major obstacle to the deployment of adaptive and autonomous systems is being able to verify their correct operation – In Flight

  3. V&V Issues & our Approach • Verification: how to specify an unforseen event? • Validation: not possible to test all configurations While traditional V&V methods will remain useful, these methods alone are insufficient to verify and certify adaptive control systems for use in safety-critical applications • Our approach combines mathematical analysis, intelligent validation, and dynamic monitoring and supports specific software V&V process, • targets multiple aspects and phases of V&V of adaptive control systems, and • uses a unique combination of research in • Neural Networks • Control Theory • Numerical Methods • Bayesian Statistics

  4. Our Bayesian Approach How good is the network performing at the moment? • Traditional: NN as a Black Box • Here: Look at probability distribution of the NN output • Variance (confidence measure) depends on: • How well is the network trained? • How close are we to “well-known” areas Small variance = good estimate Large variance = bad estimate; no reliable result, just a guess Our approach, based on a Bayesian approach, provides a measure of how well the neural network is performing at the moment

  5. Milestone I: Envelope Tool • Basis: Adaptive NN-based controller • Lyapunov error bound defines regions of eventual stability • Regions where confidence is small might cause instability • Informally: a safe envelope is a region where the confidence level is sufficiently high • Bayesian approach combined with sensitivity analysis • Challenge: methods for efficient determination of safe envelope • Can help answer questions like • How large is the current safe envelope? • How far is the operational point from the edge? Current status: mathematical background formulated, prototypical Matlab/Simulink implementation designed, first simulation experiments

  6. Confidence Envelope Confidence Surface bad Safety Envelope: area of good confidence 1/confidence good airspeed altitude The Envelope tool uses a Bayesian Approach to calculate the current safety envelope

  7. Conclusions & next steps • Current work as scheduled toward deliverable (9/2004) • prototypical implementation in Matlab/Simulink • report on mathematical background and tool • Getting Case Study I ready: IFCS Gen-II simulink model • Next steps in research: • system identification (sysID): estimate confidence of parameters • other model representations (e.g., parameter tables with polynomial interpretation) • Preparation of Case Study II and III

More Related