DTC Project 8.5 “Intelligent Sensing”

1. DTC Project 8.5“Intelligent Sensing” University of SouthamptonSchool of Electronics and Computer Science Prof C.J.Harris and Prof N.M.White Dr D. Karatzas and Dr A. Chorti

2. Teaching Old Sensors New Tricks Intelligent Sensors are systems combining sensing elements and sophisticated processing in the sensor housing. They are able to process information locally and autonomously react to specific situations.

3. Intelligent Sensorscan go where humans cannot Humans use intuition to respond to a tricky situation. Intelligent Sensors can also respond in a clever way! It is desirable to employ Intelligent Sensors to keep human beings away from hazardous situations Advantages of Intelligent Sensors Redundancy of simple sensors is not always a good idea. In the case of a catastrophic event they will all suffer the same damage. Intelligent Sensors on the other hand can reconfigure themselves

4. INTELLIGENT SENSOR (DTC Project 8.5) DTC Project 8.5 investigated algorithms for optimal signal extraction from sensors in the context of an Intelligent Sensor Software Architecture INTELLIGENTSENSOR (DTC Project 8.5)

5. SENSOR MANAGEMENT (DTC Project 8.1) PERFORMANCE METRIC DATA FUSION SYSTEM DECISION / CONTROL INTELLIGENT SENSOR N (DTC Project 8.5) … By processing information locally, the sensor management system (DTC Project 8.1) can assume higher confidence on reported data INTELLIGENTSENSOR 1 (DTC Project 8.5)

6. INTELLIGENT SENSOR (DTC Project 8.5) State of the Art Research Few industry standards exist for Intelligent Sensors DTC Project 8.5 covers this gap! IEEE1451 BS7986 Maintains compatibility to existing industry standards IEEE 1451 defines low-level aspects of intelligent sensors and communication protocols Formally defines a framework for Intelligent Sensors implementation BS 7986 describes higher levelin-sensor processing but does not define algorithmic implementation details Investigated algorithmic approaches to tackle common issues of real-world applications

7. Intelligent Sensor Software Architecture INTELLIGENT SENSOR (DTC Project 8.5) INTELLIGENT SENSOR (DTC Project 8.5) A generic, modular architecture was developed as an advantageous intelligent sensor implementation The Project 8.5 Intelligent Sensor Software Architecture introduces the following functionality: • Self-validation To self-validate sensor data and identify inconsistent inputs To self-adjust to drift/bias effects usually caused by the sensing element ageing or poisoning To communicate sensor’s condition to the sensor management level, so proper action can be taken To adapt to environmental changes To autonomously reconfigure in order to continue operation despite any sensor degradation • Self-adjustment to drift/bias • Communication of sensor’s condition • Adaptation to environmental changes • Autonomous reconfiguration

8. Primary Measurand Internal Fusion Sensor Interface #1 Fault Detection Sensor Model Provider VV Timing Sensor Interface #2 Fault Detection Sensor Model Provider … … VVstatus Sensor Interface #n Fault Detection IEEE 1451TEDS Sensor Model Provider Intelligent SensorControl Environment Measurand #1 VU Sensor Interface #1 Fault Detection InternalFusion IEEE Sensor Identification Sensor Model Provider … … Sensor Interface #n Fault Detection VUstatus Sensor Model Provider IEEE 1451 Communi-cations Environment Measurand #2 Sensor Interface #1 Fault Detection InternalFusion VDstatus Sensor Model Provider Module Messaging … … Sensor Interface #n Fault Detection Sensor Model Provider The Fault Detection modules can make use of theoretical sensor models provided by Sensor Model Provider modules, which: Intelligent Sensor The fused measurements can be checked for drift/bias by a Drift Estimation & Compensation module: The outputs from each array of sensors are fused by Internal Fusion modules: All modules are bound together by the Intelligent Sensor Control module: DriftEstimation & Compensation The Sensor Interface modules are responsible to: The Fault Detection modules: T A primary measurand, as well as any number of secondary environmental measurands can be monitored in parallel Modules are combined in a mix-and-match fashion to create specific Intelligent Sensor implementations that address real-life scenarios Sensor ModelProvider Estimates different types of drift from historical data Assess incoming data and produce an uncertainty value Communicate with the sensing element hardware Communicates the final corrected measurement to higher processes Act as sensor model libraries Generate a single value and uncertainty at each iteration Environment Communications Interface (IEEE 1451) T Can select the best model to use at any given time Corrects for drift and updates the uncertainty value accordingly Ensures compatibility with IEEE and BS standards Correct incoming data if possible Obtain measurements on demand Filter out inconsistent values Indicate how the above values were calculated Perform basic signal processing (linearisation, A/D conversion etc) H Makes use of physical sensor models Is responsible for timing, messaging between modules etc.

9. Additive Drift #1 • Additive Drift #2 • Multiplicative Drift • … Available AlgorithmicImplementations Drift Estimation &Compensation DTC Project 8.5 investigated alternative algorithmic implementations for each of the modules Fault Detection Intelligent SensorControl Each module in the architecture addresses a distinct common issue of real life implementations But, There is no single way to tackle each of these issues! Sensor ModelProvider Internal Fusion Sensor Interface

10. VV Pressure VVstatus VU Fault Detection Sensor Interface Sensor Model Provider VUstatus Fault Detection Sensor Interface VDstatus Sensor Model Provider Intelligent Sensor Drift Estimation &Compensation Sensor Interface Sensor Model Provider IntelligentSensorControl Temperature Communications Interface (IEEE 1451) Internal Fusion As an example, we will see how the Intelligent Sensor Software Architecture of Project 8.5 can be used to implement an Intelligent Sensor featuring a piezoresistive pressure sensor as the primary sensing element The Sensor Interface module feeds directly a Drift Estimation & Compensation module which corrects the input for additive and multiplicative drift using models provided by a Sensor Model Provider module The physical model for the pressure sensor is dependent on temperature, and the Sensor Model Provider needs to know this information to select the appropriate model We can use an array of two temperature sensors (not necessarily of the same type), which interface with the Intelligent Sensor through their own Sensor Interface modules The output of each of these sensors is assessed by a Fault Detection module, which makes use of physical models supplied by Sensor Model Provider modules The Intelligent Sensor Control is responsible to communicate temperature information back to the pressure sensors’ Sensor Model Provider module The piezoresistive pressure sensor is interfaced with the architecture through a Sensor Interface module The temperature measurements are fused by an Internal Fusion module, before passed to the Intelligent Sensor Control module Finally, the corrected pressure information is passed to the Intelligent Sensor Control to communicate to higher processes

11. Intelligent Sensor Intelligent Sensor Pressure Pressure Drift Estimation &Compensation Drift Estimation &Compensation Sensor Interface Sensor Interface Sensor Model Provider Sensor Model Provider IntelligentSensorControl IntelligentSensorControl Temperature Temperature Fault Detection Fault Detection Internal Fusion Internal Fusion Sensor Interface Sensor Interface Sensor Model Provider Sensor Model Provider Fault Detection Fault Detection Sensor Interface Sensor Interface Sensor Model Provider Sensor Model Provider On the right appears a snapshot of Project 8.5’s Intelligent Sensor Demonstrator, showing the implementation described The correspondence of the modules between the architectural design and the implementation is highlighted The demonstrator is now shown in action. As new measurements become available, each module performs its own processing on the data The estimated additive and multiplicative drift can be seen in the Drift Estimation & Compensation module The Fault Detection modules identify outliers and remove their effects on the signal

12. INTELLIGENT SENSOR (DTC Project 8.5) Project’s 8.5 developed a low level Intelligent Sensor Software Architecture which is generic, modular, compliant with existing industry standards and can be used to implement any sensory application Some of the key attributes of the Intelligent Sensor Software Architecture are: Project 8.5’s Intelligent Sensor requires minimal maintenance and therefore minimal human intervention Its modular character makes it easy to use through a mix-and-match fashion It is trivial to upgrade individual modules without altering the overall implementation For each module a number of alternative implementations are available to cover a variety of application specific needs

13. The list of potential applications for Project 8.5’s research output is endless. Two characteristic examples stemming from active research projects in the University of Southampton are: Potential Applications Biometric Keypad Southampton Artificial Hand

14. Potential Applications Biometric Keypad Southampton Artificial Hand Combines “Chip and PIN” technology with biometric keystroke recognition to identify or verify an individual A long-running project at the University of Southampton utilising state of the art sensor technology Intelligent Sensors can be introduced on each key (current implementation features 2 piezoelectic sensors behind each pad) Range of real-life Applications: Features 3 sensors on each fingertip: Force, Temperature and Slip Relationship to Project 8.5: Introduce Intelligent Sensors on its fingertips Banking Security Systems (access control) Reduce cabling requirements

15. Data Information Fusion Defence Technology Centre Project 8.5 Intelligent Sensing Contact Details Publications Address University of Southampton School of Electronics and Computer Science Room 1001, Building 86 Southampton, SO17 1BJ Telephone +44 (2380) 599204 Websitewww.dtc.soton.ac.uk Prof N.M.White nmw@ecs.soton.ac.ukProf C.J.Harris cjh@ecs.soton.ac.ukDr D.Karatzas dk3@ecs.soton.ac.ukDr A.Chorti ac2@ecs.soton.ac.uk • D. Karatzas, A. Chorti, C.J. Harris and N.M. White, “Teaching Old Sensors New Tricks: Archetypes of Intelligence”, accepted at IEEE Sensors journal (Special Issue on Intelligent Sensors) • A. Chorti, D. Karatzas, N.M. White and C.J. Harris, “Use of the EKF for state dependent drift estimation in weakly nonlinear sensors” accepted at Sensors Letters • A. Chorti, D. Karatzas, N.M. White and C.J. Harris, “Intelligent Sensors in Software: The Use of Parametric Models for Phase Noise Analysis”, submitted to ICISIP 2006 conference on Intelligent Sensing and Information Processing, IEEE • N.M. White and P.J. Boltryk, “Advances in Intelligent Sensors”, book chapter, to appear in “Adaptronics”, Springer • P.J. Boltryk, C.J. Harris and N.M. White, “Intelligent Sensors – a generic software approach”, Sensors & their Applications XIII, University of Greenwich at Medway, Chatham Maritime, Kent, September 2005 • P.J. Boltryk, C.J. Harris and N.M. White, “An Algorithmic Approach to the Optimal Extraction of Signals from Intelligent Sensors”, Nanotech 2005, Anaheim, USA, May 8-12, 2005 • N.M. White, “Intelligent Sensors, Systems or Components?”, Invited Paper, Nanotech 2005, Anaheim USA, May 8-12, 2005