Tracking in High Target Densities Using a First-Order Multitarget Moment Density

1. Tracking in High Target Densities Using aFirst-Order Multitarget Moment Density Ronald Mahler, Ph.D. Lockheed Martin NE&SS Tactical Systems Eagan, Minnesota, USA 651-456-4819 / ronald.p.mahler@lmco.com IMA Industrial Seminar Series, University of Minnesota October 4, 2002

2. Problem • It is not always feasible—or necessary—to detect, track, and identify individual targets with accuracy • large-formation tracking: track density is so large that only knowledge of overall geometrical target distribution is feasible • group tracking: detection and tracking of force-level objects (brigades, battalions, etc.) is of greater interest than detection and tracking of their individual targets • cluster tracking: a few Targets of Interest (ToI's) are obscured by a “multi-target background” of low-priority targets, which is of interest only because it might contain a ToI

3. Approach: “Bulk Tracking” • Conventional: detect & track all targets • most feasible: where density is lowest (outskirts of a formation) • least feasible: where inter-track confusion is greatest because density is highest (i.e., where ToI's are most likely to be found) • Bulk tracking: detect & track bulk target groupings first, then sort out individual targets as data permits • estimate what is knowable given current data quantity / quality (at first, only bulk multitarget behavior) instead of attempting to estimate what cannot be known until sufficient, and sufficiently good, data has been collected (i.e., individual target behavior) • resolve individual targetsout of the "multitarget background" as separate identifiable tracks only as additional information about them is accumulated over time

4. Approach (Ctd.): 1st-Order Multitarget Moment Filter Bayes problem formulation Bayes-optimal approach computational strategy state of system = random state vector single-sensor, single-target Bayes filter first-moment filter (e.g.a-b-gfilter) multi-sensor, single-target state of system = random state-set (point process) multi-sensor, multi-target Bayes filter multi-sensor, multi-target first-moment filter (“PHD” filter)

5. Topics 1.    First-order moment filtering • Multitarget first-order moments: the “PHD” • Multitarget first-order moment filtering • Simulations

6. 1st- and 2nd-Order Moment Filters observation space zk+1 random observationsz produced by target zk state space target motion random state- vector,x Xk+1k+1 Xk+1|k Xk|k fk|k(xk|Zk) fk+1|k(xk+1|Zk) fk+1|k+1(xk+1|Zk+1)  optimal Bayes filter time-update step data-update step a-b-gfilter ^ ^ ^ how can we extend this reasoning to multitarget systems? xk|k xk+1|k xk+1|k+1 ^ ^ ^ Pk|k Pk+1|k Pk+1|k+1 Kalman filter

7. Multi-Sensor/Target Problem: Point Process Formulation reformulate multi-object problem as generalized single-object problem multisensor state X*k = {x*1,…,x*s} sensors “meta- multisensor- multitarget observation Z = {z 1,…,z m} sensor” diverse observations random obser- vation- set “meta-  observation” random state- set “meta- X target” multitarget state X = {x1,…,xn} targets

8. Geometric Point Processes (= Random Finite Sets) three equivalent formulations of a (multidimensional) simple point process sum the Dirac deltas concentrated at the elements ofX S dX(x) = Sdy(x) X NX(S) = |X S| yX random object-set random density random counting measure “engineering-friendly” (multi-object systems are modeled as visualizable random images) preferred by physicists preferred by mathematicians

9. Integral and Derivative for Simple Point Processes Set integral: Functional (Gateaux) derivative: physics: “functional derivative” Dirac delta function

10. Probability Law of a Geometric Point Process probability generating functional (p.g.fl.) h = bounded real-valued test function discrete-space notation used only for claritication multitarget posterior distribution (= Janossy densities) X = {x1,…,xn} Frechét functional derivative belief measure probability that all targets are in S plausibility measure (= Choquet functional) probability that some target is in S

11. Multitarget Posterior Density Functions  fk|k(X|Z(k))dX = 1 normality condition fk|k(X|Z(k)) multitarget posterior measurement-stream Z(k)Z ,...,Zk multitarget state fk|k(|Z(k))(no targets) fk|k(x|Z(k)(one target) fk|k(xx2|Z(k))( two targets) … fk|k(xxn|Z(k))(ntargets) multisensor-multitarget measurements: Zk = {zzm(k)} individual measurements collected at timek

12. The Multi-Sensor/Target Recursive Bayes Filter observation space random observation- setsSproduced by targets state space multi-target motion Xk|k Xk+1|k Xk+1|k+1 evolving random state-set recursive Bayes filter  fk|k(X|Z(k),U(k-1)) time-update fk+1|k(X|Z(k),U(k-1)) data-update fk+1|k+1(X|Z(k+1),U(k))  multitarget Markov transition density fk+1|k(Y|X) fk+1(Zk+1|X, X*k+1) multisensor-multitarget likelihood function (target-generated observations & clutter) X*k+1 new sensor state-set (to be determined) p1D(x,x*),…, psD(x,x*) sensor FoVs Zk+1 future observation-set (unknowable)

13. What is a Multitarget First-Order Moment? Naïve concepts of multitarget expected value fail So: we must resort to “indirect” multitarget moments   subset space vector space well-behaved function f f(X)= f({x1 ,…, xn}) X= {x1 ,…, xn} f(X  Y) = f(X) + f(Y) ifX  Y = (disjoint unions are transformed into sums)  “Indirect” multitarget expectation: expected value of random vector f(X) corresponding to random set X E[f(X)]

14. Two Possible Choices for f three different notations for a simple point process sum the Dirac deltas concentrated at the points ofX S f(X) = Sdy(x) = dX(x) f(X) = |X S| X yX random counting measure random state-set random density

15. Indirect Expected Values of a Random State-Set S DX(x)= E[dX(x)] MX(S)= E[|X S|] first-moment density “probability hypothesis density” (PHD) first-moment measure

16. PHD for a Discrete State Space (Picture) probability of the hypothesis: “the multitarget system contains a target with state x0“ DX(x0) = Pr(x0X) = p1 + p6 + p9 + p11 + p16 p17 p16 four-state instantiations {x, x2, x3, x4}of X p15 p14 p13 p12 three-state instantiations {x, x2, x3}of X p11 p10 p9 p8 p7 two-state instantiations {x, x2}of X p6 p5 p4 p3 one-state instantiations {x1}of X p2 p1 x0 state space (discrete)

17. The PHD (Ctd.) PHD magnitude S DX(x)dx= expected number of targets inS DX(x0)= expected target density at x0 If state space is discrete then the PHD is a fuzzy membership function (fuzzy subset of target states) S x0 state space

18. Example of a PHD on 2-D Euclidean Space five peaks (largest target densities) correspond to locations of seven partially resolved, closely spaced targets target density, D(x) cluster X

19. First-Order Multarget Bayes Filtering observation space random observation- setsZproduced by targets state space random state-set random state-set multi-target motion Xk|k Xk+1|k Xk+1|k+1 three targets five targets multitarget Bayes filter  fk|k(X|Z(k)) fk+1|k(X|Z(k)) fk+1|k+1(X|Z(k+1))  1st-moment “PHD” filter  Dk|k(x|Z(k)) Dk+1|k(x|Z(k)) Dk+1|k+1(x|Z(k+1))  time-update step data-update step

20. PHD Functional Derivative Formula

21. PHD Filter Assumptions: Motion Model time-step k time-step k+1 fk+1|k(y|x) motion probability that target will have state yif it had statex x y 1 dk+1|k(x) probability that target will vanish if it had statex death  x X bk+1|k(X|x) spawn probability that target will spawn a target-set X if it had statex x X bk+1|k(X) creation  probability that a target-set X will appear in scene all target motions are assumed statistically independent

22. PHD Filter Assumptions: Sensor Model state space observation space fk(z|x) likelihood probability that target will generate observation zif it has statex x z pD probability that target will not generate an observation (assumed state-independent) misdetection  x ck+1|k(Z) probability that a set Z = {zzm} of clutter observations will be generated; Poisson false alarms: clutter  ck|k(Z) = e-llk|kn ck|k(z1)  ck|k(zm) • observations and clutter are statistically independent • multitarget posteriors are approximately Poisson (need high SNR): fk|k(X|Z(k)) e-N Nk|k sk|k(x1)  sk|k(xn) X = {x1,…,xn}

23. PHD Time-Update Step  Dk+1|k(y|Z(k)) = bk+1|k(y) +Dk+1|k(y|x) Dk|k(x|Z(k))dx time- updated PHD term for spontaneous target births = PHD of bk+1|k(X) Markov transition PHD PHD from previous time-step Dk+1|k(y|x) = dk+1|k(x) fk+1|k(y|x) + bk+1|k(y|x) probability of target survival Markov transi- tion density term for targets spawned by existing targets = PHD of bk+1|k(X|x)  Nk+1|k = Dk+1|k(y|Z(k))dy Nk+1|k = dk+1|k(x) + nk+1|k (x) Dk|k(x|Z(k)) dx  predicted expected number of targets nk+1|k(x) =bk+1|k(y|x)dy  expected number of targets spawned by x

24. PHD Filter Bayes Update Step  Given new scan of data, Zk+1 ={zzm} S pDDk+1(z) Dk+1|k+1(x|Z(k+1))  Dk+1(x|z) + (1-pD) Dk+1|k(x|Z(k+1)) lk+1ck+1(z)+pDDk+1(z) zZk+1 Bayes-updated PHD average no. of false alarms distribution of false alarms predicted PHD (from previous time-step)  Dk+1(z) = fk+1(z|x) Dk+1|k(x|Z(k+1))dx sensor likelihood function single-observation Bayes update of PHD f(z|x) Dk+1|k(x|Z(k+1)) Dk+1(x|z) =  Nk+1|k+1= Dk+1|k+1(x|Z(k+1))dx Dk+1(z) predicted expected number of targets (from previous time-step) S pDDk+1(z) Nk+1|k+1 expected number of targets after new scan + (1-pD)Nk+1|k lk+1ck+1(z)+pDDk+1(z) zZk+1

25. Proof, I: Transform PHD into p.g.fl. Form 1. Data-updated multitarget posterior: 2. p.g.fl. of multitarget posterior: 3. PHD of multitarget posterior:

26. Proof, II: Transform PHD into p.g.fl. Form 4. Define Bayes characterizing functional (B.c.fl.): p.g.fl. of multitarget measurement density 5. PHD in B.c.fl. form:

27. Proof, III: Choice of Likelihood Function 6. Multitarget likelihood function: Poisson false alarm process w/ spatial distribution c(z) each target generates at most one observation with probability pD 7. Simplified B.c.fl.:

28. Proof, IV: Simplification 8. Assume predicted multitarget posterior is Poisson: 9. Final simplified B.c.fl.: • Inductively determine denominator and numerator of • B.c.fl. form of PHD using functional derivatives

29. Special Case  Dk+1|k(y|Z(k)) = fk+1|k(y|x) Dk|k(x|Z(k))dx assuming: no target deaths or births Nk+1|k = Nk|k time-prediction S Dk+1|k+1(x|Z(k+1))  Dk+1(x|z) assuming: no misdetections or false alarms zZk+1 Nk+1|k+1 = |Zk+1| data update Following example is based on above assumptions: Example: bulk tracking of two clusters of three more separated targets

30. Example: Six Targets, Two Clusters  = actual target locations  = noisy observations  = estimated target locations y x Targets are further apart. Increased data quality (relative to filter resolution) allows filter to resolve targets as well as track the two formations.

31. Two Clusters: PHD at 3rd Observation Since the PHD filter estimates the number N of targets, a peak extraction algorithm is used to look for the N tallest peaks PHD value direction of motion x y

32. Two Clusters : PHD at 9th Observation

33. Two Clusters: PHD at 17th Observation

34. Two Clusters: PHD at 27th Observation

35. Two Clusters: PHD at 31st Observation

36. Implementation Based on Particle-System Filters posterior, timek posterior, timek+1 “particles” = samples Delta functions • -Non-restrictive with respect to measurement models • - Very general continuous-state Markov models • e.g. heavy-tail models, non-smooth models • - Very strong, general guaranteed-convergence properties • for every observation sequence, particle distribution converges a.s. to posterior • - Computational order:O(pd)(low-SNR detection),O(p)(low-SNR tracking) • p = no. particles, d = dimensionality, N = pd = no. of unknowns • - LMTS is co-developing these filters with U. Alberta(Prof. M. Kouritzin)

37. Branching Particle-System Filtersfast and rapidly convergent  most-probable particles “spawn” new particles in next time-step  least-probable particles “die” in next time-step

38. Simple Simulation: Multitarget Tracking in Clutter 2002 2003 Dec Jan Feb Mar Apr May May Jun Jul Aug Sept Oct Nov • 200 scans collected • average of 120 • clutter observations • per scan • targets cross at • 40th scan • 26% increase in • RMS localization • accuracy over • target-generated • observations • velocity also • tracked successfully first target second target

39. Summary / Conclusions • First-order moment filtering provides a potential means of tracking clusters of emitters until enough information has been accumulated to begin extracting RVs • Computational power can be shifted from low-interest regions of the PHD to regions that may contain targets of interest, to allow “peaking up” of those targets • Efficient computational implementation requires particle-systems methods