Some Applications of Distributed Architectures in Image-based Surveillance Systems

1. Some Applications of Distributed Architectures in Image-based Surveillance Systems Graduate Seminar in CIS 750 Video Processing and Mining Spring 2003 Presented by: Benjamin Garrett

2. Agenda • Distributed Multi-Sensor Surveillance: Issues and Recent Advances Pramod K. Varshney and Ioana L. Coman • Sensing Devices • Architectural Issues • Information Processing • Case study: concealed weapons detection

3. Distributed Multi-Sensor Surveillance • Primary aspects to take into consideration: • Sensing device development: some added processing capabilities needed. • System and network design: global considerations and architectural issues need attention. • Information processing tasks: the abundance of data requires elaborate information fusion techniques.

4. Distributed Multi-Sensor Surveillance • Intelligent Distributed Systems (IDS) • Includes some work on specialized OS • Intelligence Surveillance and Reconnaissance (ISR) • Data processing issues arise due to large quantities of data • Intelligent Real-time Integrated Sensor (IRIS) systems • Redundant sensors for added reliability

5. Sensing Devices • Early efforts were dedicated to the development of various types of sensing devices – acoustic/sonar, IR, seismic, magnetic. • One example is the Remote Battlefield Acoustic Sensor System (REMBASS)

6. REMBASS • Ground-based, all-weather, day-and-night, battlefield surveillance, target development, and early warning system capable of remote operation under field conditions. • Basic purpose of REMBASS is to detect, locate, classify, and report personnel and vehicular (wheeled and tracked) activities in real-time within the area of deployment. • It uses remotely monitored sensors placed along likely enemy avenues of approach.

7. REMBASS • Sensors respond to seismic-acoustic energy, IR energy, and magnetic field changes to detect enemy activities. • The sensors process the data and provide detection of classification information which is incorporated into digital messages and transmitted through short burst transmission to the system sensor monitor programmer set. • The messages are demodulated, decoded, displayed, and recorded to provide a time-phased record of enemy activity.

8. REMBASS - Problems • Sensors had limited processing power and over-loaded the central unit with data. • Limited bandwidth and information fusion capabilities at the central unit did not allow optimum utilization of the retrieved data.

9. MEMS • Micro-Electro-Mechanical Systems (MEMS) • integration of mechanical elements, sensors, actuators, and electronics on a common silicon substrate through micro-fabrication technology.

10. MEMS • Promises systems-on-a-chip capabilities. • Microelectronic integrated circuits can be thought of as the "brains" of a system and MEMS augments this decision-making capability with "eyes" and "arms", to allow microsystems to sense and control the environment. • Sensors gather information from the environment through measuring mechanical, thermal, biological, chemical, optical, and magnetic phenomena. • The electronics then process the information derived from the sensors activate mechanical devices.

11. The Georgia Tech Wearable Motherboard • Promises a multitude of applications in sports medicine, advanced health care, and monitoring of astronauts, law enforcement personnel, and combat soldiers. • Optical fibers can detect bullet holes, and special sensors and interconnects monitor vital signs of the body.

12. The Georgia Tech Wearable Motherboard • Plastic optical fibers woven throughout the fabric of the shirt. • Flexible data bus transmitting information from sensors mounted on an inside shirt. • Bus also transmits information to the sensors (and hence, the wearer) from external sources. • The optical fiber can be used to pinpoint the location of a bullet penetration in combat causality care.

13. Distributed Multi-Sensor System Architecture • Operational Independence • Managerial Independence • Evolutionary Independence • Emergent Behavior • Geographic Distribution

14. Distributed Multi-Sensor System Architecture • Sensor-level intelligent subsystems – one or a few devices configured for fast reaction time. • Regional or local subsystems – where data fusion takes place. • Central Intelligence Units – usually few if not only one. Makes complex decisions and can override decisions of lower level units.

15. Distributed Multi-Sensor System Architectural Issues - IDS • Encompasses a wide range of activities. • Intelligent Interactive Distributed Systems group - Vrije Universiteit (VU) in Amsterdam • Agent Operating System: a platform for managing mobile processes.

16. Information Processing • Refers to effective means for coordinating the data coming from multiple sensors. • Data/image/information fusion is a vast research field with many open projects in progress. • Video Surveillance and Monitoring Team at CMU

17. VSAM at Carnegie Mellon

18. VSAM at Carnegie Mellon • Data fusion: Every observed object is positioned in a 3D geodetic coordinate system using geolocation. • Sensor Tracking: Sensors considered as precious resource to be allocated according to user-specified tasks. • Scene Visualization: Employs a GUI giving a synthetic view of the environment.

19. VSAM at Carnegie Mellon

20. Case study: concealed weapons • Uses two different types of sensors: MMW and IR wave sensors. • Infrared waves give better resolution. • Millimeter waves penetrate better.

21. IR waves and Millimeter waves

22. Case study: concealed weapons • Image Registration – The process of finding the corresponding points from two or more images. • IR image is superimposed over the MMW to evaluate the accuracy of registration task.

23. Distributed Surveillance Systems – Concluding remarks • High amounts of funding being invested in distributed multi-sensor surveillance systems. • Many of the issues presented are open research problems, some of which are still in their initial stages of development. • Encompasses a wide variety of disciplines and fields.

24. Sources Consulted [1] Bult K. et. al. “Low Power Systems for Wireless Microsensors”, Proc. of the 1996 Intl. Symposium on Low Power Electronics and Design, Monterey, CA, Aug. 1996, pp. 17-22 [2] Lin T.-H., Sanchez H., Kaiser W. J. and Marcy H. O. “Wireless Integrated Network Sensors (WINS) for Tactical Information Systems”, Proc. of the 1998 Government Microcircuit Applications Conference. [3] Sungmee Park, Kenneth Mackenzie, Sundaresan Jayaraman. “The wearable motherboard: a framework for personalized mobile information processing (PMIP). 170-174 Electronic Edition (DOI: 10.1145/513918.513961) [4] Babak Firoozbakhsh, Nikil Jayant, Sungmee Park, and Sundaresan Jayaraman. “Wireless Communication of Vital Signs Using the Georgia Tech Wearable Motherboard”, IEEE Intl. Conference on Multimedia & Expo. 2000, New York, NY, Electronic Proceedings.

25. Sources Consulted cont. [5] Y. Wang and B. Lohmann. Multisensor image fusion: concept, method and applications. Technical report, University of Bremen, 2000. [6] H. Qi, X. Wang, S. S. Iyengar, and K. Chakrabarty, “Multisensor data fusion in distributed sensor networks using mobile agents”, Proc. Intl. Conf. Information Fusion, pp. 11-16, August 2001. [7] R. Collins, A. Lipton, H. Fujiyoshi, and T. Kanade. “Algorithms for cooperative multisensor surveillance. Proceedings of the IEEE, Vol. 89, No. 10, October, 2001, pp. 1456 – 1477. [8] The Intelligent Interactive Distributed Systems group web site: http://www.iids.org/. [9] The Remote Battlefield Acoustic Sensor System web site: http://www.fas.org/man/dod-101/sys/land/rembass.htm [10] The Micro-Electro-Mechanical Systems Clearinghouse web site: http://www.memsnet.org/mems/