Watchdog Confident Event Detection in Heterogeneous Sensor Networks

1. WatchdogConfident Event Detection in Heterogeneous Sensor Networks Matthew Keally1, Gang Zhou1, Guoliang Xing2 1College of William and Mary, 2Michigan State University

2. Overview • Problem Statement • Challenges • Related Work • Contributions • Design • Evaluation

3. Confident Event Detection • Many applications for event detection have stringent accuracy requirements and demand long system lifetimes • Vehicular traffic monitoring • Falls in elderly patients • Military/intrusion detection • Perform confident event detection • Meet user-defined false positive and false negative rates in the presence of in-situ sensing reality • Reduce energy usage to extend system lifetime

4. Challenges of Confident Event Detection • How to cluster the right sensors to meet user accuracy requirements? • Learn the detection capabilities of individual sensors and clusters • Use part of the detection capability to meet user requirements and save energy • How to efficiently perform collaboration between heterogeneous sensors to meet user requirements? • Difficult for modality-specific models and data fusion • Need a generic solution • How to adapt detection capability to runtime observations? • Easier observations and harder observations need different detection capabilities

5. Related Work • Sensing Coverage • Do not address user accuracy requirements • Do not explore detection capability of deployment • Modality-specific Sensing Models and Data Fusion • User requirements not met in reality • Difficult to perform heterogeneous sensor fusion • Do not cluster the right sensors to meet user requirements • Machine Learning • Do not address user accuracy requirements • Do not adapt sensing capability to runtime observations

6. Motivation: Related Work Shortfalls • Vehicle Detection: sensing irregularity • Same distance, different accuracies • Accuracy can increase with distance • Sensing Coverage may overdetect or underdetect events • Theoretical sensing models assume all sensors are identical

7. Motivation: Related Work Shortfalls • Different clusters (C1,C2,C3) have the same accuracy, 100%, better than individual sensors • Difficult to capture for existing works: Due to lack of knowledge of detection capability of different sensors and clusters

8. Watchdog Contributions • A confident and energy efficient event detection framework • Choose the right sensors to meet user requirements • Generic framework that provides heterogeneous sensor fusion • Adapt detection capability to runtime observations • Easy observations: low-power sentinel sensors • Hard observations: higher-power reinforcement sensors • Performance evaluation: two scenarios • Monitor traffic entering and leaving computer science building • Vehicle detection using Wisconsin trace data • Compare against sensing coverage and signal attenuation model

9. Node Aggregator Sensor Cluster Generation Local Aggregation Sentinel and ReinforcementSelection Request Reinforcement Data Training Results Runtime Event Detection Observations • Local Aggregation • Cluster Generation • Sentinel & Rein. Selection • Runtime Event Detection Watchdog Design Overview • Efficient heterogeneous collaboration • Explore detection capability of a deployment • Cluster the right sensors to meet user requirements • Adapt detection capability to runtime observations

10. Local Aggregation • Cluster Generation • Sentinel & Rein. Selection • Runtime Event Detection Cluster Generation • Goal: determine detection capability of • Individual sensors and sensor clusters • A specific deployment • Method • Randomly generate up to M clusters for each cluster size • For each generated cluster • Step 1: Train a Hidden Markov Model for the cluster • HMM is good for heterogeneous sensor fusion • HMM captures time and space correlation of sensor data • Step 2: Determine cluster FP/FN based on the HMM decision and ground truth at each time interval

11. Local Aggregation • Cluster Generation • Sentinel & Rein. Selection • Runtime Event Detection Step 2: Determine cluster FP/FN based on the HMM decision and ground truth • At each aggregation interval: • Determine event detection decision with trained HMM • Compare cluster detection decision with ground truth • Get the cluster FP/FN (accuracy) • Determine FP/FN for each possible event probability

12. Local Aggregation • Cluster Generation • Sentinel & Rein. Selection • Runtime Event Detection Sentinel and Reinforcement Selection • Choose sentinel cluster: low detection capability • Meets user's FN requirement • Makes easy detection decisions • Choose reinforcement cluster: higher detection capability • Meets both FP and FN requirements • Used to make more difficult detection decisions • All other sensors go to sleep

13. Local Aggregation • Cluster Generation • Sentinel & Rein. Selection • Runtime Event Detection Runtime Event Detection • Goal: adapt detection capability to runtime observations • Easier observations and harder observations need different detection capabilities • Method: • Sentinels and reinforcements form local observations at each aggregation interval • Sentinels report non-default observations to the aggregator to make detection decisions • Reinforcements requested when sentinel event probability false positive rate exceeds user requirements • Reinforcements return non-default observation data and aggregator makes a confident decision

14. Local Aggregation • Cluster Generation • Sentinel & Rein. Selection • Runtime Event Detection Runtime Event Detection User requirements: u.FN = u.FP = 0.05 Reinforcements Acoustic Seismic 56 60 52 54 Sentinels Aggregator t=1: No Event, s.FN = .01 < u.FN t=2: Event, s.FP = .02 < u.FP t=3: No Event, s.FN = .01 < u.FN t=4 :Undecided, s.FP = .45 > u.FP t=4 :Event, r.FP = 0.3 < u.FP t=5: No Event, s.FP = 0.2 < u.FP Time interval 0 1 2 3 4 5

15. Evaluation • App1: Wisconsin SensIT trace data • Vehicle detection at a fixed location • 75 nodes with acoustic, seismic, and infrared sensors • 100ms aggregation interval • App2: Computer Science Building Traffic Monitor • Five IRIS motes mounted on main entrance door • MTS 310: 2-axis accelerometer, 2-axis magnetometer, acoustic, and light sensors • Define event as when someone opens the door and walks through • 4s aggregation interval • Compare with a modality-specific sensing model • Distance-based signal attenuation • Data fusion for event decisions • Compare with V-SAM, a state of the art protocol for handling sensing irregularity • Measure data similarity between sensors • Keep awake only sensors with dissimilar readings

16. Exploring Detection Capability & Meeting Requirements • Only a limited & discrete number of FP/FN rates supported by the deployment • For a specific FP/FN rate, a large number of clusters may be available • During runtime detection, Watchdog meets FP/FN explored during training

17. Compare with V-SAM: Accuracy • V-SAM with k-coverage and similarity coverage • Watchdog outperforms all with near perfect accuracy

18. Compare with Modality-Specific Sensing Model: Accuracy • Vehicle detection with acoustic sensors • Select clusters with two different ranges to target location: near (<25m) and far (>40m) • Watchdog always meets user requirements • Modality-specific model ignores in-situ sensing reality

19. Compare with Modality-Specific Sensing Model: Energy • Watchdog clusters the right sensors to meet user requirements • Meets requirements with reduced energy • Watchdog adapts its capability to runtime observations to save energy • Modality-specific sensing model uses all sensors in the cluster

20. Adapting Detection Capability to Runtime Observations • Experimental setting • Vehicle trace data and sensors from <25m • User requires 0% false positives and false negatives • Watchdog clusters the right sensors to meet user requirements • Neither V-SAM nor the modality-specific sensing model adapts detection capability to runtime observations

21. Conclusions and Future Work • Existing works do not provide event detection with confidence, we need to • Cluster the right sensors to meet user requirements • Provide a generic approach for heterogeneous deployments • Adapt detection capability to runtime observations • Watchdog: a confident event detection framework • Meets user accuracy requirements • Exceeds accuracy of existing approaches • Uses knowledge of detection capability to save energy • Future Work • Online and distributed detection

22. Thanks to NSF grants ECCS-0901437 and CNS-0916994

23. Compare with V-SAM: Training Length • Watchdog achieves maximum performance with a short training • V-SAM requires little training, but is less accurate

24. Local Aggregation • Cluster Generation • Sentinel & Rein. Selection • Runtime Event Detection Local Aggregation • Allows for heterogeneous sensor fusion • Raw data is combined to form a single observation • Use a common aggregation technique • Discrete, finite number of possible observations • Same number for each sensor and modality • Allow for comparison between sensors of all modalities • We use two discrete observations

25. Local Aggregation • Cluster Generation • Sentinel & Rein. Selection • Runtime Event Detection Event Probability Discussion • Differentiate the accuracy between different event probabilities • Some observations are more reliable than others • Probabilities near 0.5 are more inaccurate • Determine FP and FN for each of p probability ranges (p=10) • Probability between .1 and .2 has zero false negatives • Probability between .9 and 1.0 has 6% false positive rate • Ranges with no events have 100% false positive or false negative rates