Human in the loop: From Search and Rescue to Smart Buildings

1. Human in the loop: From Search and Rescue to Smart Buildings Alberto Cerpa Assistant Professor Computer Science School of Engineering, UC Merced CENS Seminar, UCLA December, 7 2007

2. Outline • SenSearch • GPS and witness assisted tracking for DTNs • How does it work? • Performance evaluation • SCOPES

3. Design Goals for Family of Applications • Self-*, self-operate, self-configure • Long lifetime • Small and light weight • Non intrusive; no infrastructure needed • Power and memory efficient • Low cost • Meets security and privacy requirements

4. Search and Rescue • Goal: To build a search and rescue system that can pinpoint missing person’s last seen point and expected location in wilderness areas • Lost hikers, stranded climbers, injured skiers, … • Difficult because of lack of timely information about the current location • “Last seen point” is critical for search and rescue actions

5. A real case of a last seen point

6. Current Search and Rescue Technologies • The Old School Way – Ask • Personal GPS receiver and Satellite transmitter – Power greedy; Must operate manually to send your location • Localization system and GSM transmitter – Need GSM network coverage • Avalanche beacon/RFID reflector – Limited usage • Need a better, cheaper, reliable system

7. SenSearch • GPS and Witness Assisted Tracking for Delay Tolerant Sensor Networks • Collaboration with Jyh Huang, John Ledbetter, Shivakant Mishra and Rick Han from University of Colorado at Boulder, together with Lun Jiang, Ankur Kamthe and Ian Freeman from UC Merced • Comprised of: • RF sensors • GPS receivers • Access points • Control center

8. 1 2 SenSearch – How it Works (I)

9. 1 SenSearch – How it Works (II)

10. Hot Search Zone x3, y3, z3 x2, y2, z2 Inferred location at 23:59 x1, y1, z1 SenSearch – How it Works (III) Hiker 6 is reported missing at 23:59

11. SenSearch – Architecture Overview Witness Witness Search & Rescue Team Control Center

12. Main Design • Duty cycle: independent duty cycle periods for both the GPS unit and the radio • Paramount for increased system lifetime • GPS cold start takes 8-12 sec, need coordinates before the possible encounter • Memory reuse: limited number of entries per source • Group communication and storage: • When nodes move in groups, leader election algorithm based on energy reserves • Data replication among members of the group

13. SenSearch Implementation • Full implementation and testing ongoing at UCM (June-July ‘06, Nov-Dec ‘07) and Boulder (Oct-Nov ’06, Sep-Nov ‘07) • Some preliminary results: • Bounded localization error (70 meters average) • Increase system lifetime with the use of duty cycling techniques for both the GPS and radio units • Bounded memory usage without degradation in localization error

14. Localization Error

15. Power and Memory Usage

16. Some lessons learned • GPS duty cycle is critical for this type of systems • No significant increase in localization and tracking error • Large number of local GPS readings decrease overall performance • Increase in memory usage and total data transferred • Group of people moving together dramatically affect performance • Nodes within the group constantly updating tables increase network traffic and memory usage • The effect gets propagated to other nodes when they come in contact with members of the group • Memory bounds • Storing only last n recent entries per node helps bounds the size of each node’s database

17. Future Work • Data richness is not being properly exploited: • Simple linear regression for inferred localization • Machine learning methods that exploit past history and topological (hike) path structure seem quite promising • Need larger scale experiments (20+ nodes) • Problem: Very costly to get localization ground truth for validation as the number of nodes increases • Other applications: • Cattle tracking for health and epidemic control • Kids tracking in theme parks

18. Outline • SenSearch • SCOPES: Smart Cameras Object Position Estimation System • System design overview • Local processing algorithms • Performance evaluation • Other projects • Summary

19. SCOPES: Distributed Smart Cameras • Goal: build a distributed vision based tracking system for human density estimation inside buildings • Automatic distributed control of HVAC systems • Efficient lighting use and control • Building usage and design • Part of the “Living Laboratory” concept • Synergy with other disciplines: • designing and constructing green buildings that produce net-zero energy use and carbon emissions • being the first comprehensive research university that is energy independent and carbon neutral!

20. Economic Impact A larger fraction of electricity goes to buildings in rich countries! Data provided by Paul Waide, graphics by Shoibal Chakravarty, from Robert Scollow

21. 35 nodes deployed in the second floor of SE1 Building and 65 more nodes (100 total) to be deployed by Jan 15 (hopefully!) • It can measure temperature, humidity, light and movement • It will interface with the building’s EM&CS • Close the loop: temperature and air flow control commanded from the network • Integration with demand-response systems

22. Testbed Components USB/IP server USB (data + power) • Hacked version of USB/IP driver to support unlimited number of devices. • Provides multiple local usb devices as if the nodes were directly connected to the main server • Running EmStar/EmTOS as well as stand-alone TOS POE (data + power) USB/IP client Ethernet (data)

23. Software Architecture

24. Image Capture • Cyclops board has 512 KB of SRAM, in 8 banks of 64 KB each (max addressable by the MCU) • Most efficient image capture: • We capture as many consecutive nFrames (images) as possible to fill a single memory bank • We switch banks and continue capturing images • Interleaving image capturing and processing proves to be less efficient

25. Object Detection + Background Update • Perform background subtraction of each image with the reference background image • Keep EWMA of both the mean and the deviation: • Depending on the values of Mean and Deviation we apply the following algorithm:

26. Grouping and Direction Inference • Raster scanning for connected components • Check if pixel x is connected to a,b,c,d • Group labeling is done according to: • Object counting and center of mass for each object • Tracking above for multiple images provides direction of travel and speed of each object

27. Local Processing Performance • The local processing time ultimately depends on the number and frequency of objects detected over time. • This introduces a variable duty cycling, since the camera does not capture images while processing the data in the banks

28. Occupancy and Flow Estimation Maps • 16 nodes, covered 150 sq. meters of SE1 • Main source of error: missed events due to duty cycling

29. Position Estimation and Detection Error • The position estimation error remains roughly constant at different times of the day • The detection failure probability decreases as we increase the density of the nodes

30. Detection Probability and Latency • The detection latency increases as we increase the memory used. • The detection probability however, significantly increase as we increase the memory used.

31. Integration: lighting and HVAC control Concentrated and filtered natural daylight is conveyed to the interior through a liquid light pipe with minimal roof penetration Collaboration with Prof. Roland Winston

32. Thank You!

33. Energy Management

34. Memory Management