Verification and Validation of Adaptive Systems

1. Verification and Validation of Adaptive Systems Bojan Cukic, Eddie Fuller, Marcello Napolitano, Harshinder Singh, Tim Menzies, Srikanth Gururajan, Yan Liu, Sampath Yeramalla West Virginia University

2. Introduction • System performance evolves over time. • Evolution needed to address complex interactions between system and the environment. • Improved performance achieved through learning. • Off-line and on-line adaptation. • Proper reactions to conditions not envisioned/identified/analyzed by system designers. • NASA interests. • Autonomous navigation. • Intelligent flight control.

3. Architectures for Adaptive NN-based Control Desired Response Reference Model Error Actual Response + Physical Process - Command Feedback Control Adaptive NN • DCS • Sigma-Pi • Multilayer Perceptron (SHL) • Radial Basis Function Learning Rule

4. NASA IFCS Architecture(Gen 1) New V&V Techniques

5. pilot inputs Implemented In SCE 3 Dynamic Inversion Controller Feedback Error Control Allocation Model Following - + Direct Adaptive Neural Network Implemented In ARTS II Sensors NASA IFCS Architecture (Gen II) New V&V Techniques

6. Our Approach to V&V Desired Response Reference Model Error Actual Response + Command Physical Process - 3. Estimate Trustworthiness of Outputs Feedback Control Adaptive NN Learning Rule 2. Monitor Stability of Learning 1. Monitor Inputs to NN,Novelty Detection

7. The Role of Novelty Detection • Block the penetration of unreliable or unreasonable training data into the online adaptive component. • “Warn” the system about the incoming uncertainty. • Provide architectural framework for backward and forward recovery capabilities.

8. Support Vector Machines (SVM) • Developed by Vapnik et. al., SVM is designed based on the Structural Risk Minimization Theory. • In pattern recognition, used for efficient clustering in highly dimensional spaces. • What SVM does? • Maps input space (by means of a nonlinear transformation) into a high dimensional (hidden) feature space.

9. Support Vector Machine (2) • SVM finds the maximum margin hyperplane in the feature space. • Thishyperplane maximizes the minimum distance to the closest training point. • The maximum margin hyperplane is represented as linear combination of training points with non-zero weights (called support vectors).

10. Support Vector Data Description • Developed by Tax et. al., • Finds a sphere with the minimal volume that contains all data points. • Basically a one class classifier.

11. Support Vector Data Description (2)

12. Experiments 5 Failure Modes, 3 Pairs of Parameters. Mode 1: Actuator Failure – stuck left stabilator at current position. ( 0 degree ) Mode 2: Actuator Failure – stuck left stabilator at pre-defined deflection. (+3 degree) Mode 3: Actuator Failure – stuck left stabilator at pre-defined deflection. (-3 degree) Mode 4: Actuator Failure – stuck right aileron at pre-defined deflection. (+3 degree) Mode 5: Actuator Failure – stuck right aileron at pre-defined deflection. (-3 degree) Pair 1: ( the pitch rate, the average of the left stabilator and the right stabilator) Pair 2:( the angle of attack, the average of the left stabilator and the right stabilator) Pair 3: ( the pitch altitude, the average of the left stabilator and the right stabilator)

13. Data Collection

14. SVDD and IFCS Simulation

15. Novelty Detection for IFCS Parameterization of SVDD affects the treatment of novelties

16. Monitoring Stability of Learning • Traditional V&V techniques for a Neural Network code are not applicable. • The system changes following the deployment. • The goal of the proposed approach: • Prove stability property of the learner. • Monitor convergence towards the stable state of the learner in real-time. • Provide a time varying measure of reliance that can be justifiably placed on the adaptive component. • Smells like dependability, doesn’t it?

17. Lyapunov (like) Stability Analysis • We emphasize the role of Lyapunov-type self-stabilization analysis for V&V. • Think of NN as dynamical system • Lyapunov’s theorem says: find V such that V>0 and DV·0 then system moves towards given solution (stable). • Perturbations tend back to given solutions so variations in data will still give learnable states. • DCS Neural Network algorithm is internally self-stabilizing.

18. Lyapunov Theory and V&V • Built in to this approach is a real-time measure for self-correction of a neural network based adaptive system

19. A Simple Example • Learning in DCS neural network. • Two Spirals data set. • Move neurons to match data. • Connect regions that represent similar data. • Grow network. • Monitor self-correction.

20. Stabilization Within DCS NN • THEOREM: • During DCS network’s learning and representation from a fixed input manifold, the evolving state of the network due to neural unit’s positional adjustment is self-stabilizing in a globally asymptotically stable manner.

21. Experimental Evaluation • An F-15 simulator customized for WVU/NASA research needs. • Input data sets obtained from simulated flights. • DCSCz network (Mach, altitude, alpha (angle of attack) WVU F-15 Flight Simulator

22. Experimental Evaluation (2) • Input data set (blue) • Network training (red/green) • Lyapunov monitoring function

23. Summary • The project is developing fundamentally new V&V technology applicable to a fundamentally different type of systems. • Critical for future NASA missions (ASAP). • Autonomy and adaptivity anticipated to be critical. • Multidisciplinary approach in research is a key to success. • Research solves an engineering problem, which happens to have software implementation. • This is a systems engineering exercise.

24. Summary (2) • Separating nominal flight conditions from failure modes by SVDD appears very promising research direction. • Computationally efficient. • Lyapunov self-stabilization theory can guarantee that the network actually preserves and learns the input feature data manifold • The central V&V issue of IFCS. • Effective monitoring of performance aspects of the neural network controllers.

25. Further work • Investigating real-time application of SVDD. • Extending Lyapunov type analysis to variable input manifolds. • Stochastic stability analysis. • Continual maintenance of the flight simulation capability. • Investigating the “output trustworthiness” issue. • Continuing collaborations with NASA (Ames, Dryden , IV&V), ISR, contractors… • Keeping up with V&V needs of GEN II and beyond.