Maurice Chu, Sanjoy Mitter, Alan Willsky SensorWeb MURI Review Meeting September 22, 2003

1. Distributed Multiple Target Tracking and an Information Architecture for Designing Applications on Ad Hoc Sensor Networks Maurice Chu, Sanjoy Mitter, Alan Willsky SensorWeb MURI Review Meeting September 22, 2003

2. MTT on Sensor Networks • Traditional MTT • Multiple Hypothesis Tracking (MHT) D.B.Reid • Joint Probabilistic Data Association Filter (JPDAF) Y. Bar-Shalom • conceived for a centralized system • radar data (time series of position estimates) • Sensor network characteristics • limited energy • unreliable communications • frequent node failures • multiple processors • distributed heterogeneous data • Challenges • Distributed algorithm for tracking multiple targets • Fusion of heterogeneous data (classification data to resolve data association ambiguities) • Minimizing resources expended

3. Focus • A general architecture for generating efficient distributed algorithms for multiple target tracking • Assumptions • Ignoring networking issues: packet collisions, dropped packets, routing, etc. • Simple, exemplary sensing models • Sensor nodes have knowledge of one-hop neighbors. • No track initiation

4. Outline • Motivation and Problem Formulation • Basic Track Update Algorithm (RCA-5&6) • Information Integration Architecture • Application Scenarios and Simulations • Conclusions and Future Work

5. Graphical model formalism Track state to raw sensor measurement model Top-down pass (prior information) track state at time t-1 track state at time t individual contributions to each sensor measurement Bottom-up pass (measurement incorporation) sensor measurement Basic Track Update Algorithm Motion model Single target to single sensor model Combined measurement model Compute approximate posteriors using belief propagation.

6. Top-down pass (prior information) 1 3 6 1 3 6 1 3 6 1 1 1 3 3 3 6 1 2 3 2 3 4 6 Bottom-up pass (measurement incorporation) 1 2 3 4 6 1 2 3 4 6 Distributed Implementation Leader-based approach Agent assignment: assign variable nodes in model to physical sensor nodes Optimal agent assignment according to model of communication cost.

7. Distributed Implementation • Distributed representation of set of current tracks are by a set of track leaders. • Track leaders send individual contributions to sensor nodes within range of track. • Each sensor node computes posterior contribution of all tracks within range and sends this information back to track leaders. • Track leader updates track.

8. Distributed Implementation • Distributed representation of set of current tracks are by a set of track leaders. • Track leaders send individual contributions to sensor nodes within range of track. • Each sensor node computes posterior contribution of all tracks within range and sends this information back to track leaders. • Track leader updates track.

9. Distributed Implementation • Distributed representation of set of current tracks are by a set of track leaders. • Track leaders send individual contributions to sensor nodes within range of track. • Each sensor node computes posterior contribution of all tracks within range and sends this information back to track leaders. • Track leader updates track.

10. Distributed Implementation • Distributed representation of set of current tracks are by a set of track leaders. • Track leaders send individual contributions to sensor nodes within range of track. • Each sensor node computes posterior contribution of all tracks within range and sends this information back to track leaders. • Track leader updates track.

11. Distributed Implementation • Distributed representation of set of current tracks are by a set of track leaders. • Track leaders send individual contributions to sensor nodes within range of track. • Each sensor node computes posterior contribution of all tracks within rangeand sends this information back to track leaders. • Track leader updates track.

12. Information Integration Architecture • For efficiency, selectively update quantities. • Availability of data • Quality of estimates • Context • Define trigger conditions for indicating when and how quantities should be updated. • Model view vs. Process view

13. Measurement Model vs. Information Architecture Process point of view Model point of view Tracks Continuation or Termination Individual Contributions prime frequency Localization for a Specific Target query Sensor Measurements

14. Classification Data Association Position update Track initiation Information Integration Architecture

15. Holistic ViewGlobal View Classification Tracking Localization

16. Holistic ViewLayered Information Processing Architecture • Decompose inference tasks into individual processing tasks. Classification Tracking Localization

17. Holistic View • Distributed algorithms for global tasks. Sensor node 1 Sensor node 2 Sensor node 3 local sensor measurements local sensor measurements local sensor measurements

18. Process 1 Process 2 Inference 2 Inference 1 General Methodology • Three elements • System Architecture – physical characteristics, constraints • Layered Information Processing Architecture – software architecture • Hierarchy Representation – meta-representation for converting processes into distributed algorithms Step 1 Step 2 Step 3 Hierarchy for Process Set of Inference Tasks Algorithms for System Architecture Layers Information Processes Inter-layer Interaction Hierarchy

19. sensor nodes track Virtual single target tracking and classification • Classification information unnecessary during virtual single target tracking. • Classification performed assuming track position known and data available • Ex. Frequency Signature Trigger: several intensity sensors within range, virtual single target

20. Data Association ScenarioDiscriminatory Information Available Immediately • Desired behavior is to use discriminatory information when needed and available.Using classification information to aid multiple target tracking.Ex. Frequency signatures of targets.

21. SimulationDiscriminatory Information Available Immediately Two targets crossing. Different frequency signature. Two targets converging then diverging. Different frequency signature.

22. Data Association ScenarioDiscriminatory Information Delayed or Unavailable • Desired behavior is to detect ambiguity in data association and represent this appropriately. Break tracks into tracklets at times of ambiguous data association. Encode connections between tracklets.

23. SimulationDelayed Discrimination of Data Association Ambiguity Two targets crossing. Same frequency signature. Blue target is metallic. Red target is not. Two targets converging then diverging. Same frequency signature. Blue target is metallic. Red target is not.

24. Conclusions • Application • Distributed algorithm for MTT on sensor networks. • Graphical Models (J. Pearl, F. Kschischang) • Basic inference algorithm for application • Distributed implementation via agent assignment on graphical model • Information integration architecture • Organizational tool to keep complexity of distributed algorithm under control. • Specific update algorithms depend on the context of the scenario. • Ongoing work at Palo Alto Research Center • Change to joint track representation (with J. J. Liu, J. Shin) • Identity management (with J. Shin, L. Guibas) • Programming framework (with J. Liu, J. Reich, F. Zhao)

25. Future Work • General Problem • Bounded Region • Finite number of random points • Fusion center at each point (e.g. Kalman filter) • Neighborhood of i • Point i receives observations (e.g. wireless link) • Hierarchical Collaboration Scheme … … … … … … Issues: convergence, redundancy for link failures, sub-instantiation for efficiency

26. Network fusion: Who does what; who says what; who listens? • A surprisingly insightful simple example • A single binary variable, x, to be inferred • Two sensing nodes, y1, y2conditionally independent given x • Tandem architecture: • One sensor sends a single bit to the other sensor • The other sensor uses this bit plus its own data to choose the estimate of x

27. Simple example II • Questions: • What decision rule does the first sensor use to decide what bit to send? • What decision rule does the second sensor then use to determine its estimate • Answers: These are all likelihood ratio tests, but the thresholds are coupled to achieve best overall team performance • So, which sensor sends first and which makes the final decision • Answer: A very good question, since except in specific cases, there isn’t a canned answer!

28. Where is this taking us? • A simple graphical model with an overlayed communications/computation network Graphical Model Sensor Network

29. Some questions • Which sensor is responsible for estimating which variable? • Which sensor communicates to which? • Do we allow the possibility of no communication or repeat communication (a la SPRTs) • Note that the act of not sending or requesting a message conveys information • This leads to problems in self-organizing, dynamic fusion networks • Tune in next time…

30. Future Work • Switch to tracking in joint target space when targets very close. • Repair of tracklets is based on hard decisions and should be extended to softer schemes. (J. Shin) • Relaxing networking assumptions.

31. Exemplary Sensor Measurement Models • (Signal Intensity) Sound Intensity (MS) • (Binned Signal Intensity) Frequency Bandlimited Intensity (FFT) (refinement) • (Discrete Classification) Magnetometer – information about whether a target is metallic

32. Canonical ScenariosData Association Problem in MTT • Data association is the problem of resolving ambiguity of assigning position measurements to tracks.

33. Overall System minus Track Initiation