NEST Final Experiment: the application layer

1. NEST Final Experiment:the application layer Phoebus Chen, Songhwai Oh, Bruno Sinopoli EECS UC Berkeley September 8, 2005

2. Outline • The application: • Multiple-target tracking problem • Pursuit Evasion Game • The implementation • The nitty gritty • The result: • Demonstration

3. Tracking Algorithms inNEST Final Experiment Songhwai Oh EECS UC Berkeley 2005

4. Previous Work:Multiple-Target Tracking in Sensor Networks • Traditional – computationally intensive • [Chong et al. 1990] Distributed multitarget multisensor tracking – not designed for sensor networks • Classification-based – multiple single target tracking problems • [Li et al. 2002] Detection, classification and tracking of targets • [Shin et al. 2003] A distributed algorithm for managing multi-target identities in wireless ad-hoc sensor networks • [Liu et al. 2004] Distributed state representation for tracking problems in sensor networks • Ad-hoc – not robust • [Liu et al. 2003] Distributed group management for track initiation and maintenance in target localization applications • [Kreucher et al. 2003] Multi-target sensor management using alpha-divergence measures – centralized algorithm • No general algorithm suited to sensor networks

5. Requires a New Algorithm • Track multiple targets reliably without classification information • No or partial target classification information • Targets of the same type • Handle measurement inconsistencies • False and missing measurements • Transmission failures • Communication delays • We don’t know the number of targets in advance • Real-time algorithm

6. Outline • Multiple-target tracking problem • Markov chain Monte Carlo data association (MCMCDA) algorithm • Hierarchical multiple-target tracking algorithm for sensor networks

7. Multiple-Target Tracking Problem

8. Solution Space of Data Association Problem (a) Observations Y (b) Example of a partition  of Y

9. Two Possible Solutions to Data Association Problem

10. Outline • Multiple-target tracking problem • Markov chain Monte Carlo data association (MCMCDA) algorithm • [Oh, Russell, Sastry, 2004] • Hierarchical multiple-target tracking algorithm for sensor networks

11. Markov Chain Monte Carlo (MCMC) • A general method to generate samples from a complex distribution • For some complex problems, MCMC is the only known general algorithm that finds a good approximate solution in polynomial time [Jerrum, Sinclair, 1996] • Applications: • Complex probability distribution integration problems • Counting problems (#P-complete problems) • Combinatorial optimization problems • Data association problem has a very complex probability distribution

12. MCMC Data Association (MCMCDA) • Start with some initial state 12 

13. MCMC Data Association (MCMCDA) • Propose a new state ’» q(n,’) • q: £ 2! [0,1], proposal distribution q(n,’) = probability of proposing ’ when the chain is in n ’ n propose • q(n,’) is determined by 8 moves:

14. MCMC Data Association (MCMCDA) • If accepted, • Accept the proposal with probability () = P(|Y), Y = observations n+1=’ • If not accepted, n+1=n

15. MCMC Data Association (MCMCDA) • Repeat it for N steps 

16. MCMC Data Association (MCMCDA) • Repeat it for N steps 

17. MCMC Data Association (MCMCDA) • Repeat it for N steps 

18. MCMC Data Association (MCMCDA) • Repeat it for N steps 

19. MCMC Data Association (MCMCDA) • Repeat it for N steps 

20. MCMC Data Association (MCMCDA) • Repeat it for N steps 

21. MCMC Data Association (MCMCDA) • Repeat it for N steps 

22. MCMC Data Association (MCMCDA) But how fast does it converge?

23. Polynomial-Time Approximation to Joint Probabilistic Data Association [Oh,Sastry 2005]

24. MCMCDA Highlights • Optimal Bayesian filter in the limit • Provides approximate solutions to both MAP and MMSE • Avoids the enumeration of all feasible events • Single-scan MCMCDA approximates JPDA in polynomial time with guaranteed error bounds [Oh,Sastry, 2005] • Outperforms Multiple Hypothesis Tracking (MHT) algorithm [Oh, Russell,Sastry, 2004] • Statistically sound approach to initiate and terminate tracks • Can track an unknown number of targets • Suitable for an autonomous surveillance system • Easily distributed and suitable for sensor networks

25. Outline • Multiple-target tracking problem • Markov chain Monte Carlo data association (MCMCDA) algorithm • Hierarchical multiple-target tracking algorithm for sensor networks • [Oh, Schenato, Chen, Sastry, 2005]

26. Hierarchical MTT Algorithm Requirements • Autonomous • Ability to initiate and terminate an unknown number of tracks • Required for distributed tracking • Low computation and memory • Robust against • transmission failures • communication delays • localization error • Scalable • Low communication load Hierarchical MTT (HMTT) MCMCDA Hierarchy Local data fusion

27. Multiple-Target Tracking and Pursuit Evasion Games in Sensor Networks • Goal • Track an unknown number of multiple targets using a sensor network of binary sensors without classification information • Coordinate multiple pursuers to chase and capture multiple evaders in minimum time using a sensor network • Technologies • Multi-sensor fusion [Oh, Schenato, Chen, Sastry, 2005] • Provides finer estimates of target positions using correlation among sensors • Markov chain Monte Carlo data association (MCMCDA) [Oh, Russell, Sastry, 2004,2005] • Optimal Bayesian filter in the limit; Outperforms Multiple Hypothesis Tracking (MHT); Fully polynomial randomized approximation scheme for Joint Probability Data association (JPDA) • Statistically sound approach to track an unknown number of targets in real-time • Robust against transmission failures, communication delays, and localization error • Robust minimum time feedback control and multi-agent coordination algorithm [Schenato, Oh, Sastry, 2005]

28. Multiple-Target Tracking and Pursuit Evasion Games in Sensor Networks

29. Experiment/Demo Overview • Infrastructure Setup • Node Level Software • Application Level Software • Lessons Learned

30. TOSBase PC Ethernet cable Ethernet Hub Laptop Experiment Infrastructure • Tracking Region consists of 144 nodes in 12x12 grid, 5 meter spacing • Connect network to a PC basestation via TOSBase • Connect basestation via TCP/IP to a laptop running Matlab algorithms

31. Kraken: Software Infrastructure Tools • Deluge • RPC and Registry • Prometheus Power System • Drip/Drain Routing Layer • Code space limitations • A few kBs short, could not include Nucleus

32. Choice of Sensors • Chose to use PIR Sensors over Acoustic and Microphone • PIR gave 20-25 ft range • less signal processing • more sensitivity to people

33. Detection Software on Motes • Four Modes of Generating Detection Events • Simple Threshold Crossing for PIR Sensor • PIR Detection Algorithm by UVA • 3 stages: High-Pass Low-Pass Filter, Adaptive Threshold, Window • Timer Generated Detection Events • User Button Generated Detection Events • For simplicity, no data aggregation

34. Pytos Interactive Environment • Used primarily to set/get Registry Values • Toggles between Detection Modes • Set Parameters like Threshold Values and Report Rate • Check if a particular node is alive • Peek/Poke at Ram Symbols • allows a lightweight (low memory footprint) method of switching modes • Thread running script to query Battery and Capacitor Voltage Levels

35. Matlab: Application Environment • Matlab connects to Java SerialForwarder application • Parses packets directly, adds receive-time timestamps, and stores in a matrix • Provides a quick detection visualization environment • Map of Detections • ‘Oscilloscope’ graph of detection values • Multiple Target Tracking Algorithm runs off of data stored in matrix • Decoupled, parallel development of MTT and Detection Infrastructure

36. Incremental Rollout • Indoor Testing on 5 nodes • Ensure code stability in controlled environment: ran 5 days without crashing • Testing on 36 USB power nodes • Unsure of effects of power source on triggering false alarms • Scaled to 72, 108, then 144 nodes • Slow reimaging (even with Deluge) as scaled in size • Bandwidth limitations require us to slow down sample rate • Yield: 6 nodes dead out of 144

37. Lessons Learned • Need tools to quickly visualize faulty areas in the network • Importance of Robust Design and on-the-fly reconfiguration • Loss of Time Synchronization • False Alarms • Scaling is not trivial! • Test at small scale thoroughly first

38. Multi-object Tracking, Pursuit Evasion Games, Data Exfiltration via UAVsP. Chen, M. Manzo, S. Oh, T. Pynn, T. Roosta, S. Schaffert, D. Shim, B. Sinopoli, B. Zhu, S. Sastry Aug 29, 2005

39. DEMOS • Application GUI • PDA visualization tool • Single person tracking • Multiple people tracking • Pursuit Evasion Games • Data exfiltration via UAV • Distributed tracking

40. Multi-person tracking demo GUI

41. PDA Visualization • Sensor networks are moving toward … • Ubiquitous • Distributed systems • Field tool • Deployment • Debugging • Interaction • Portable client tools • PyTOS • Nucleus • Applications • First responders • Guidance system • On-site technicians • Human-in-the-loop pursuit-evasion games • Tracking Demo • Detection relayed to laptop • Tracks generated at laptop • Relayed / Render on PDA

42. Single Person tracking Video Multi-hop Communication Sensing

43. Multiple Person tracking Video Multi-hop Communication Sensing

44. Multiple person tracking 3 Unknown number of evaders enter the field at different times 1 Dynamic estimation of number of evaders 2

45. Multiple person tracking • Evaders will enter the field intersecting their trajectories, then exit the field 2 • Trajectory tracking • Robustness at crossing points • Termination of tracks 3 1

46. Pursuit evasion games Control input Control input Multi-hop Communication

47. Pursuit Evasion Game Some Evaders will re-enter the field 1 2

48. Pursuit Evasion Game As soon as they are tracked, Pursuers will be dispatched P1 1 Pursuit evasion game over sensor network. Pursuer will use real time tracking info to control 3 P2

49. Pursuit evasion with two pursuers, two evaders

50. Three evaders crossing the field