Anomaly Detection for Prognostic and Health Management System Development

1. Anomaly Detection for Prognostic and Health Management System Development Tom Brotherton

2. New Stealth Technology

3. Outline • What is Anomaly Detection • Different types of anomaly detectors • Radial Basis Function Neural Net Anomaly Detector • The basics • Comparison with other neural net approaches • Feature ‘off-nominal’ distance measures • Training • Implementations • Continuous = Gas turbine engine monitoring • Snap shot = Web server helicopter vibration condition indicators • RBF NN & Boxplots • Application to detection of helicopter bearing fault • Application to monitoring fish behavior for water quality monitoring

4. What is Anomaly Detection? • Anomaly Detection = The Detection of Any Off-Nominal Event Data • Known fault conditions • Novel event = New - never seen before data • New type of fault • New variation of ‘known’ nominal or fault data • What is ‘Nominal’ • Sets of parameters that behave as expected • Physics models • Statistical models

5. Accuracy & Cost Approaches • Ex: Gas Turbine Engine Deck: Component level physics model Physics • State Variable Models (derived from physics) • Hybrid Model: Combine Physics + Empirical Parametric- Estimate of physics • JPL: BEAM (coherence = model of linear relationships) • Neural nets (non-linear relationships) Empirical- Derived from collected data • Fused empirical: BEAM + NN • Academic: Support Vector • Simple statistics Applicability

6. Empirical Modeling An anomaly Idea: Theoretical boundary (multi-dimensional ‘tube’) that data should lie within: - Nominal data is inside the boundary - Anomaly data is outside Problem: How to estimate / approximate the boundary? Collected ‘Nominal’ Data Problem: What measurement(s) caused the anomaly? Problem: How far off-nominal is the anomaly / feature?

7. NN = Model for Nominal Data = Sample of nominal data = Sample of anomalous data ? ‘Distance’ from Nominal Model Yes RBF Neural Net Anomaly Detection: The Idea Radial Basis Function (RBF) Neural Net Model • Dynamic data = Lots of NN basis units to model • Piecewise stationary approximation • Distance measure = Function of the signal set • Individual signal distances from nominal = distance from “closest” basis unit • Detection can be for set of signals when no single signal is anomalous • The model can be adaptively updated to include additional data / known fault classes • Trajectories of features relative to basis unit = Prognosis

8. MLP NN RBF NN ? ? Why Use Radial Basis Function Neural Nets? • Radial Basis Function Neural Net • Nearest neighbor classifier • Distance metric : Measure “nominal” • Multi-layer perceptron (MLP) does not have these properties

9. Support Vector Machine Model RBF Model Support Vector Machine • In some sense, much better model of ‘truth’ …. but • Automated selection of number of basis units • Lots! • Trade off between fidelity vs smoothness • Not practical for on-wing • How to compute individual signal distances • Loss of intuition Training data

10. Mahalanobis Mahalanobis Distance s2 Distance s1 Feature Distance Calculation NN = Model for Nominal Data ? • Nearest Neighbor Distance

11. Closest Basis Unit Truth - Truth: Single Feature X = ‘Bad’ • Report: Feature X = ‘OK’ & Feature Y = ‘Bad’ Alternative Distance Calculation NN = Model for Nominal Data • Alternative Distance = Which Basis Unit gives the smallest number of individual off-nominal features -> Hamming Distance (from digital communications decoding)

12. Weights Input features  • • • Is output for Nominal? =1  Yes > 1- Likely < 1-  ? < 1-  No 0<  <  <1 Basis Units ‘RBF’ NN Architectures DetectorOutput Gaussian elliptical basis function : Rayleigh basis function : Fuzzy membership basis function : Good for magnitude spectral data * Basis function is ‘matched’ to the data distribution For those who like things fuzzy = Gaussian Mixture Model

13. Training : Neural Net Architectures – How to select parameters • Small number of clusters Small number of basis units Low False Alarms - Large number of clusters  Good ‘tracking’ of data dynamics Large number of basis units  Very general Missed detections Too General ?  More sensitive to outliers More false alarms Over Trained ? Don’t know a-priori what are the ‘best’ settings

14. False alarms? Only 2 points = false alarm Small scale factor 4 points persist over time = detection M of N Detection Idea: M of N detection allows one sample high false alarm rate – Then integrate over time to remove • Trade off single point detection capability vs false alarm rate • Large Scale Factor / Small N • Short – high SNR anomalies • Small Scale Factor / Large N • Long – persistent – low SNR anomalies Large scale factor Detection?False alarm?

15. Alternatives • This technique works well • Demonstrated by Pratt & Whitney for C-17 F117 applications • Transient engine operations • Long time to train – lots of different types of transients • Model can become very complex • Engine control system • On-wing memory and timing constraints • Alternative • Combine equipment operating regime recognition with anomaly detector • Ex: Identify steady operation and then take a snapshot of the data • Simple statistics may suffice

16. Input Signal Vector Scale Signal RegimeRecognition Neural NetSelect Neural NetDetection Neural NetDetection Neural NetDetection Median Filter Trained NNs Off-Nominal SignalDistance DetectionFlag Example Gas Turbine Operations Break the big problem in to a set of small problems • Regime recognition • Regimes: • Transient Throttle up • Transient Throttle down • Steady state – B14 open • Steady state – B14 closed

17. Anomaly Detection of Stationary Regime Detected Data • Web Server Implementation for Helicopter Vibration Data • Condition Indicators (CIs) = Features derived from on-board vibration measurements • Two types of problems: • Single CI for a component • Simple statistics solution = Boxplot • Intuitive = Army user’s like it • RBF neural net implementation as well • Multi-CIs for a component • RBF neural net implementation

18. FWDLAT FWDVRT FWDSP CPITVRT CPITLAT FWDXMSNVRT FWDXMSNLAT HB2 HB3 HB4 HB5 HB6 HB7 AFTLAT AFTVRT AFTSP ENG1COMP ENG1NOSE ENG1AXIAL ENG1LAT ENG2COMP ENG2NOSE ENG2AXIAL ENG2LAT CBOXOCFA CBOXOCLAT APU AFTFANLAT AFTXMSNVRT AFTXMSNLAT XSHAFT1 XSHAFT2 • Configuration • 36 Vibration Sensors • 2 Speed Sensors • 1553 connection to HUD Main D/S Main Rotor Cockpit Control Head USB Memory Drive Parameter Data CVR-FDR USB Download IAC-1209 Modern Signal Processing Unit (MSPU) Accelerometer Ethernet Tach Sensor +28VDC Power Other Connections On Board System Tail Gearbox Advanced Rotor Smoothing / Engine Diagnostics Engines Transmissions Intermediate Gearbox Cockpit VMU Absorbers Hanger Bearings • 18 Sensors Installed – Vibration • Automated Exceedance Monitoring using HUD data • Automated engine HIT, Max Power Check and exceedances • Complete aircraft vibration survey in under 30 seconds

19. Aircraft / Server Physical Connectivity SCARNG USB Memory Stick Data Download AIRCRAFT OEMs VMEP PARTNER Browser PC-GBS Remote PC-GBS Facility AARNG INTERNET Wireless link PC-GBS Remote PC-GBS Facility Deployed Unit PC-GBS Remote

20. Browser Aircraft / Server Logical Connectivity Facility Systems Support Team- e-mail notification- Fleet level reports- Automated s/w upgrades Portable System • Army P-GBS Aircraft Maintenance-Electronic help desk- Automated data archive- Automated s/w upgrades - Army F-GBS Web Client MDS Server Help Desk Network Security Automated Data Archive Data ArchiveA/C config files Help Training Base Electronic ManualsFAQs Diagnostics Prognostics Anomaly Detection Fleet Statistics & Reports Anomaly Detection

21. Advanced Engineering on the Web The role of anomaly detection on the website is to detect and bring to engineering’s attention the MOST INTERESTING data = Something that has NOT been encountered before - More normal data not really of interest

22. Single Feature Anomaly Detection Boxplots = Simple statistics - single feature anomaly detector. No Gaussian assumption, just counting points. They seem to work very well! Default based on boxplot statistics User set

23. Threshold Setting

24. Anomaly Analysis Summary of all aircraft

25. The Raw Data

26. Original Transformed Gaussian Transformation Data • Problem: How to select a “matched” basis function • Gaussian assumption? Usually violated! • Statistical Model Fit • Transform data to be Gaussian • Transformation stored and is part of the model • Almost always only a single basis unit is required! • Works on single feature data • All processing “behind the scenes” done on transformed data

27. RBF Anomaly Detection

28. RBF Anomaly Detection

29. Case Study: Apache Swashplate Bearing Spectral Server Data • Anomalous data identified with RBF NN AD running on the Server • Aircraft was in Iraq • Automatic email alert sent to users • “Evidence” sent as well • Data reviewed by AED-Aeromechanics and IAC via iMDS website • Large peak in spectral data at 1250 Hz for tail #460 • Sidebands spaced at intervals corresponding to bearing fault frequencies • Suspected bad swashplate bearing Main SP Spectra Other A/C Tail 460 Tail 460 Other A/C

30. Case Study Apache Swashplate Bearing • AED-Aeromechanics acquired raw vibe data Apr 04 and received swashplate May 04 before aircraft was turned-in for D model conversion • Swashplate disassembled by PIF per DMWR Aug 04 • Minor spalling, corrosion and broken cage discovered • Additional algorithms developed from raw data and implemented into VMEP for release Sep 04 Broken Cage Spalling/Corrosion

31. Follow Up • Specific algorithms to identify this fault now included with the on-board system • US Army now uses ‘on-condition’ information from the system to perform maintenance • True condition-based maintenance (CBM)

32. Other Applications Water Quality Bio-Monitor • IAC 1090 is a mobile, web-enabled automated biomonitoring system that utilizing the ventilatory and body movement patterns of the bluegill fish as a bio-sensor, much like a canary in a coal mine. • Sixteen Bluegills are placed in individual flow-through Plexiglas chambers. Each chamber is equipped with an individual water input and drainage system. By utilizing sixteen different Bluegills, the IAC 1090 samples more biosensors than any other system on the market resulting in lower false alarm rates. • All fish generate a micro volt level electric field. Each individual fish is monitored by non-contact electrodes suspended above and below each fish in a Plexiglas chamber. • The electrical signals generated by the fish’s normal movement is amplified, filtered and passed on via the internet to IAC’s Bio-Monitoring Expert (BME) software system for automated analysis.

33. Water Quality Bio-Monitor • BME is a neural network based expert system that provides for rapid, real time assessment of water toxicity based on the ventilatory behavior of fish. BME has shown excellent detection capabilities for toxic compounds with a low false alarm rate. False alarms, common in other similar systems, are typically generated by normal, non-toxic variations in the environment. • Automated data collection and management tools, user interfaces, and real-time data interpretation employing advanced (artificial intelligence) models of fish ventilatory behavior make BME easy to use. • Remote (Internet) access to IAC 1090 is provided through an easy-to-use graphical user interface. BME’s modular design provides users with the ability to reconfigure the system for different biomonitoring applications and biosensors

34. Questions? Conference papers / case studies available at: www.iac-online.com