Finite Set Statistics for Sensor Data Fusion

1. Finite Set Statistics for Sensor Data Fusion BahadorKhaleghi Pattern Analysis and Machine Intelligence Lab

2. Outline • FISST Introduction • Bayes Filter (Single Target) • Multi-target Estimation • FISST for Multi-Target Estimation • Pros and Cons • Approximating MT Bayes Filter • Practical Implementations • Demos • Conclusion and Future Work

3. What is FISST? • Random sets theory was first proposed to study integral geometry in 1970s • IDEA: treat sets as random elements within probabilistic framework • FISST is formulated for finite sets • FISST has not been popular for sensor data fusion as it has not been well understood • It comes from applied math community rather than computer science

4. Why FISST? • FISST enables unification of much of data fusion under a single Bayesian umbrella • Major potential contributions • A unified expert systems theory: fusion of imperfect and highly disparate data • A unified level 1 fusion: detection, tracking, and identification as a single problem • A unified performance evaluation framework • A Unified and optimal single-target and multi-target detection and estimation

5. Optimal Bayes Filtering • Relies on Bayes theorem to provide a solution to recursive target estimation problem Likelihood Prior Distribution System State Measurements Normalization Factor

6. Optimal Bayes Filtering • Both Prior and Normalizing Factor involve integrals that generally can not be evaluated analytically • Kalman filter is fortunate exception for linear measurement and motion models • Approximation to Bayes filter • Approximations based on KF (e.g. EKF, UKF) • Approximate the integrals using particles (e.g. SMC) • Multi-target estimation is even more challenging

7. Multi-target Estimation • Objective: estimate, at each time step, the multi-object state from a sequence of noisy and cluttered observation sets • Conventional Stages • Detection => Association => Tracking => Identification

8. Multi-target Estimation • Challenges • Noise • Missed and false detections (clutter) • Target dynamics (linear, nonlinear, manoeuvring) • Imperfect and disparate observations • Non-standard targets and scenarios, e.g. extended and unresolved targets • Varying number of targets, sensors • Data association problem

9. Multi-target Estimation • Conventional approaches • Solve the data association problem (e.g. MHT, JPDA) then apply Bayes (KF) filtering to them • Drawbacks • Measurements and motion models are approximate • Bayes filtering results are approximate • Data association is usually solve only in approximate • Many challenging aspects of problem can not be modeled explicitly in a principled manner

10. FISST for Multi-target Estimation • FISST provides a natural yet rigorous mathematical tool to represent and compute in multi-target estimation systems • IDEA: represent the multi-target state and observations as finite sets instead of vectors • Rationale: vector representation • Does not admit a mathematically consistent notion of estimation error • Cannot represent all occurrences of multi-target state

11. Vector Representation Issues

12. Vector Representation Issues

13. FISST for Multi-target Estimation • Finite set representation • Casts estimation error as well-established concept of set distance • Can represent all possible occurrences • There is no inherent ordering of measurements or targets • Allows explicit modeling of many challenging aspects of (multi)target estimation

14. Multi-target Bayes Filtering • Where • And

15. Multi-target Bayes Filtering • It is not a straightforward generalization • Requires a novel calculus (derivative and integral) for finite random sets, which is a generalization of vector calculus (statistics 101) • Recent book by Mahler provides just this! • Results in closed form solutions for priors and likelihoods, which are not analytical ones as expected

16. Sample FISST Formulations Surviving Targets Spawned Targets Appearing Targets Clutter Measurement Model

17. Advantages vs. Drawbacks • Pros • Obviates data association problem • Formulates detection, tracking, and identification as a unified problem • A natural way of treating challenging effects involved in realistic tracking scenarios • Cons • Mathematically complex and less understood • Generally intractable (combinatorial complexity) • Track continuity problem

18. Approximating MT Bayes Filter • In single-target case successful approximations are based on moment matching • e.g. Kalman filter: match the first two moments (i.e. mean and covariance) • PHD filter: propagates the first moment in MT case, i.e. Probability Hypothesis Density • Cardinalized PHD filter: propagates the PHD as well as pdf for number of targets (better estimation accuracy and adaptation)

19. Approximating (C)PHD Filter • SMC method(s) • Require additional computations, i.e. particle clustering to extract multi-target state estimates • Higher flexibility • Gaussian Mixture • More computationally efficient • More restrictive, i.e. linear and mildly non-linear models • Both compare favourably with MHT and JPDA with less computational load

20. Sample Applications • SMC-PHD Filter on Radar

21. Sample Applications • SMC-PHD Filter on Video

22. Conclusion • FISST is a mathematically principled and natural solution to multi-target estimation • FISST provides a unified framework for explicit modeling of all challenging aspects of problem with promising results • FISST potential to serve as a unifying framework for expect systems in both single-target and multi-target case is yet to be investigated • Many other aspects of data fusion systems may potentially be unified using FISST!

23. Thank YoU!