Achieving Better than Human Design in Detecting Events of Interest in Bandwidth Constrained Sensor Networks

1. Achieving Better than Human Design in Detecting Events of Interest in Bandwidth Constrained Sensor Networks Kalyan Veeramachaneni, Lisa Osadciw Development and Research in Evolutionary Algorithms for Multi sensor Smart Networks (DREAMSNet) Department of Electrical Engineering and Computer Science Syracuse University, New York GECCO 2008, Human Competitive Results Awards, July 14, Atlanta, U.S.A

2. What are we detecting? • Modern day society relies on detection or determining the meaning of the presence or absence of a signal • Digital Communications • Pipeline/Bridges crack detection • Genuine User detection using biometrics • Presence of aircraft, ships, or motor vehicles • Locating emergency personnel • Weather Phenomena • Building Security • Sensors are located in remote areas making decisions using a variety of criteria • Maximum A-Posteriori Criterion • Maximum Likelihood Criterion • Minimum Error Criterion

3. Sensor1 u1 X1 Fusion Rule Sensor2 X2 u2 AND OR Second Classifier Only First Sensor Only Bandwidth Constrained Detection Networks Noise only Event Likelihood density model for a sensor

4. Event is declared only in this quadrant, i.e. AND rule False Alarms: detecting an event that did not occur Threshold on Sensor 2 Noise Misses: Fail to detect an event *Event Threshold on Sensor 1 Bandwidth Constrained Detection Networks • Two types of Errors need to be reduced • If the entire observation value is transmitted to a central processing node, an efficient machine learning technique can be designed to achieve better accuracy • Shown below are 20000 samples of observations, 10000 belong to events, 10000 to noise. • 9 to 32 bits required per sample if all bits are transmitted • Reduces to 1 bit decision if decision is transmitted instead

5. Region where an event is declared Region where an event is declared LRT (Human) Based Design: 2 thresholds on each sensor 2 Sensor only fusion rule PSO Based Design: Simple 1 Threshold for each sensor AND fusion rule Very few errors (E)Competitive Result: Correlated Sensors Designs for 0.1 Correlation

6. Humies Categories Covered • (G) The result solves a problem of indisputable difficulty in its field. • (B) The result is equal to or better than a result that was accepted as a new scientific result at the time when it was published in a peer-reviewed scientific journal. • (D) The result is publishable in its own right as a new scientific result - independent of the fact that the result was mechanically created. • (E) The result is equal to or better than the most recent human-created solution to a long-standing problem for which there has been a succession of increasingly better human-created solutions.

7. HUMIES Category G • The result solves a problem of indisputable difficulty in its field. • Amount of Research and Publications on Topic Indicates Complexity • Quick Check Research Publications • 120 Journal Articles with Approximately 45 Discussing Similar Design Issues • 48 Textbooks At Least Currently On Sale In This Area • 5 Dissertations deal with same problem and provide human developed designs • Paper Published that Addresses the Difficulty • John N Tsitsiklis, Michael Athans, “On Complexity of Decentralized Decision making and detection problems” 23rd IEEE Conference on Decision and Control, 1984 • Optimizing Distributed Detection for 2 Sensors • Independent sensors: Intractable • Correlated sensors: NP Complete -

8. HUMIES Category B • The result is equal to or better than a result that was accepted as a new scientific result at the time when it was published in a peer-reviewed scientific journal • Much Research Published in Area Since the 50s/60s Beginning in Radar • Type I/Type II Errors • Fail to detect the event • Detect an Event that did not occur • Decouple the two problems: optimize thresholds and design best fusion rule separately • When only labeled training datasets are available performance is sensitive to threshold search precision When likelihood models are available Optimize Threshold Exceed Ratio of Conditional Distributions

9. Human Design Solution: Person-by-Person Optimal (PBPO) for Independent Sensors Human Design Solution: Likelihood Ratio Test (LRT) Design Human Competitive Result: Particle Swarm Optimization (PSO) Based Design Sensor1 u1 X1 Fusion Rule Sensor2 X2 u2 Optimize thresholds individually by keeping other thresholds and fusion rule constant Use LRT for independent or correlated deriving fusion rule Joint optimization of thresholds and Fusion Rule No closed form solution exists

10. HUMIES Category B (cont) • Particle Swarm Optimized Detector Simplifies Sensor Network Adaptation • Able to combine performance parameters to simultaneously handle a variety of situations • consider resources (energy and communication bandwidth) • Reduce type I or type II errors across different degrees of correlation • Simpler Receiver • Single threshold design compared to LRT based designs that can lead to multiple thresholds as the Likelihood ratio becomes non linear • Adapt to design for either probability density models or a labeled training dataset provided • Automatically Handles the Heterogeneity in Practical Sensor Networks

11. HUMIES Category D • The result is publishable in its own right as a new scientific result - independent of the fact that the result was mechanically created. • 5 Papers Published Including a Best Paper • Correlated Sensors • Kalyan Veeramachaneni and Lisa Osadciw, “Design of Distributed Detection Systems with Heterogeneous Correlated Sensors," 44th Annual Allerton Conference on Communications and Control, Allerton Park, Illinois, September, 2007. • Independent Sensors • Kalyan Veeramachaneni, Lisa Osadciw, Pramod Varshney“Adaptive Multimodal Biometric Management Algorithm,” IEEE Transactions on Systems Man and Cybernatics : Part C: Applications and Reviews, Vol. 35, No. 3 August 2005. • Applications • Biometrics: Kalyan Veeramachaneni, Nisha Srinivas, Lisa Osadciw, and Arun Ross, “Designing Optimal Fusion Strategies for Correlated Biometric Classifiers”, IEEE CVPR Conference, Anchorage, Alaska, June, 2008. (Best Paper Award) • Pipeline Crack Detection: Kalyan Veeramachaneni, Weizhong Yan, Kai Goebel, and Lisa Osadciw, “Improving Classifier Fusion Using Particle Swarm Optimization”, IEEE Multi-Criteria Decision Making (MCDM) Symposium, Honolulu, Hawaii, April, 2007. • Adaptive Sensor Management • Lisa Osadciw and Kalyan Veeramachaneni , “Sensor Management through Efficient Fitness Function Design," Proceedings of 41st Annual Asilomar Conference on Signals, Systems, and Computers, Asilomar, CA, November, 2007.

12. HUMIES Category E • The result is equal to or better than the most recent human-created solution to a long-standing problem for which there has been a succession of increasingly better human-created solutions. • Long-standing problem since the 1950s in Radar Research • Succession of better solutions as discussed in Category B • First Single Detectors Derived for the Following Criterion • Maximum A-Posteriori Criterion – maximize the posteriori probability of belonging to one event to the other possible event • Maximum Likelihood Criterion – maximizes probability of belonging (likelihood) to event • Minimum Error Criterion – minimize the number of errors in decisions

13. HUMIES Category E (cont): • Matched Filter Designed in 50s and 60s from Radar • Maximum Signal to Noise Criterion – maximize signal over the noise background to assist detection by matched filter (North, Van Vleck, Middleton) • Inverse Probability Criterion – (Wald, Neyman, Pearson) • Likelihood Ratio - based on Shannon’s information theory (Woodward & Davis) • Distributed Detection (Tenney & Sandell-1979 through today) • Chair Z., P. K. Varshney, "Optimal Data Fusion in Multiple Sensor Detection Systems," IEEE Trans. on Aerospace and Elect. Systems, Vol. AES-22, No. 1, pp. 98-101, Jan. 1986. • Tang, Z. -B., K. R. Pattipati, and D. L. Kleinman, "An Algorithm for Determining the Decision Thresholds in a Distributed Detection Problem," IEEE Trans. on Systems, Man, and Cybernetics, Vol. SMC-21, pp. 231-237, Jan./Feb. 1991. • Kam., M., Q. Zhu., and W. S. Gray, "Optimal Data Fusion of Correlated Local Decisions in Multiple Sensor Detection Systems," IEEE Transactions on Aerospace and Elect. Syst., Vol. 28, pp. 916-920, July 1992. • Peter Willet, Peter F. Swaszek, Rick S. Blum, "The Good, Bad, and Ugly : Distributed Detection of Known Signal in Dependent Gaussian Noise," IEEE Transactions on Signal Processing, Vol. 48, No. 12, December 2000. • Kalyan Veeramachaneni, Lisa Ann Osadciw, Pramod K Varshney, "An Adaptive Multimodal Biometric Fusion Algorithm Using Particle Swarm", Proceedings of SPIE, Aerosense, April 21-25, 2003, Orlando. • Saeed A. Aldosari, Jose M. F. Moura, “Fusion in Sensor Networks with Communication Constraints”, International Symposium on Information Processing in Sensor Networks, April 26-27, 2004, Berkeley, CA.

14. HUMIES Category E (cont): Swarm Solution: • Type I/Type II Errors • Errors are Balanced in Real Time Based on Current System Needs • Simultaneously reduce, “Failure to detect the event”, “Detect an Event that did not occur” • Reduce Communication Bandwidth • Decisions at Sensor to Reduce Message Size Saving Bandwidth • Fusion Architecture Can Be Modified in Real-Time Based on Bandwidth and Energy Needs • Minimize Energy • Save in communications with Smaller Messages and Fewer Through Fusion • Reduce computations with Simpler designs and Fusion Rules • Ease of Adaptation to Other Applications • Communication Management for Any Wireless Sensor Network and Architecture • Various Sensor Networks for Aircraft Routing at Airports, First Response Networks, Large, Remote Sensor Networks, Health Monitoring Sensor Networks, and etc.

15. (E) Competitive Result: Independent Sensors Human Design Accuracy PSO Resulting Accuracy PBPO-Person-By-Person Optimal PSO – Particle Swarm Optimization

16. HumanDesign 2.5% 13% 54% (E)Competitive Result: Correlated Sensors

17. Humies Categories In Summary • (G) The result solves a problem of indisputable difficulty in its field. • This is an NP complete problem which also becomes too complex as the sensors become dependent. • (B) The result is equal to or better than a result that was accepted as a new scientific result at the time when it was published in a peer-reviewed scientific journal • Problem Studied Since 1950s with Suboptimal Solutions • PSO Allows the Coupled Threshold – Fusion Rule Problem to Remain Coupled • PSO is able to solve the problem for different problem types • (D) The result is publishable in its own right as a new scientific result - independent of the fact that the result was mechanically created. • We have published 5 papers including 1 that recently received “Best Paper” in an application domain • (E) The result is equal to or better than the most recent human-created solution to a long-standing problem for which there has been a succession of increasingly better human-created solutions. • As the network grows, the PSO performance also grows w.r.t. the human-created solution.

18. Why this is the best? • Significance of the Impact on a Wide Range of Applications • Military - Health Monitoring • Homeland Security - Environmental Monitoring • Smart and Safe Buildings - Vehicle Health Monitoring • Ease of Adapting Solution to the Complexities of Practical Problems - Imperfect Propagation Channels - Multi-User Interference • Changing Architectures - Resource Constraints • Solves Distributed Detection Problems Too Complex in Past • Multiple Imperfections • Complex Sensor Architectures • Complex Interference Environments

19. Thank you!

20. LRT based fusion rule for independent sensors Sensor1 u1 X1 Fusion Rule Sensor2 X2 u2 LRT based Fusion Rule for correlated sensors Human Design Solution:Likelihood Ratio Test (LRT) Design

21. (E)Competitive Result: Correlated Sensors Designs for 0.9 Correlation Region where an event is declared Region where an event is declared LRT (Human) Based Design: 2 thresholds on each sensor 2 Sensor only fusion rule PSO Based Design: Simple 1 Threshold for each sensor AND fusion rule Higher number of errors, but still better