Methods for Data and Information Fusion

1. Institute for Parallel Processing - Bulgarian Academy of Science Methods for Data and Information Fusion Kiril Alexiev, Iva Nikolova alexiev@bas.bg Tel: 9796620; 0898 898 616 25A, Acad.G.Bonchev Str., Sofia 1113, Bulgaria NATO ARW, Velingrad, Bulgaria, 2006 1

2. Institute for Parallel Processing - Bulgarian Academy of Sciences Correct decision making (taking) in the security sector mainly depends on information, received from multiple sources. Often, the information is insufficient, unreliable and contradictive. Methods for Data and Information Fusion 2

3. Institute for Parallel Processing - Bulgarian Academy of Sciences Architecture of sensor network sensor node Communication routing data sensor data query sensor data sensor node routing data user sensor data Methods for Data and Information Fusion 3

4. Institute for Parallel Processing - Bulgarian Academy of Sciences Definition of Data and Information Fusion • Wikipedia: Sensor fusion is the combining of sensory data such that the resulting information is in some sense better than would be possible when these sources were used individually. Better = more accurate, more complete, or more dependable Methods for Data and Information Fusion 4

5. Institute for Parallel Processing - Bulgarian Academy of Sciences Definition of Data and Information Fusion • Authors remark: In definition: “combination of data” is not very suitable phrase. We have to find better one, for example “simultaneously processed data” Methods for Data and Information Fusion 5

6. Institute for Parallel Processing - Bulgarian Academy of Sciences Benefits from Fusion Process • The first and the most important remark is that fusion process is necessary most of all to reduce (to filter) input information through its integration (merging) and generalization. • Fusion process is necessary to improve accuracy. • Fusion process is necessary to reduce uncertainty. Methods for Data and Information Fusion 6

7. Institute for Parallel Processing - Bulgarian Academy of Sciences Structure of Data and Information Fusion (JDL) Level 0: Preliminary data processing – pixel or signal level data association and characterization. Level 1: Data alignment, association, tracking and identification. Level 2: Situation assessment. Level 3: Threat assessment. Level 4: Process Refinement includes adaptive processing through performance evaluation and decision or resource and mission management. Methods for Data and Information Fusion 7

8. Institute for Parallel Processing - Bulgarian Academy of Sciences Methods for Data and Information Fusion 8

9. Institute for Parallel Processing - Bulgarian Academy of Sciences Methods for Data and Information Fusion 9

10. Institute for Parallel Processing - Bulgarian Academy of Sciences Methods for Data and Information Fusion 10

11. Institute for Parallel Processing - Bulgarian Academy of Sciences Level 1 Temporal data fusion Sensor data fusion Methods for Data and Information Fusion 11

12. Institute for Parallel Processing - Bulgarian Academy of Sciences Data association methods • The Nearest Neighbor method associates the nearest measurement to the track prediction. The more complicated Global Nearest Neighbor minimizes cluster cost function in measurement distribution. • The probabilistic data association filter (PDAF) and its extension to multiple targets – joint PDAF (JPDAF), solve the same task of measurement identification in a simpler way. In the JPDAF hypotheses are built for the measurements and targets only for the current scan. In this way the number of hypotheses is additionally reduced but the chance of combinatorial explosion in dense target and clutter scenarios still remains. Methods for Data and Information Fusion 12

13. Institute for Parallel Processing - Bulgarian Academy of Sciences Data association methods • In Multiple Hypothesis Tracking approach all measurements received at a scan are assigned to initialized targets, new targets or false alarms. A number of hypotheses are generated. Every one supposes a possible assignment scheme between measurements, received in all scans, and the targets - confirmed, new ones or false. Pruning and gating techniques are used to retain the most likely hypotheses and in this way to reduce their number • Finite Set Statistics considers all measurements as measurements from a generalized sensor and all targets as a generalized target of interest. Fusion of information from one and the same sensor but from different moments of time Methods for Data and Information Fusion 13

14. Institute for Parallel Processing - Bulgarian Academy of Sciences Identification Two types of identification: • Structural identification – more difficult Define structure (model), which in the best way corresponds to the observed system (process). • Parametrical identification – a lot of algorithms Find (calculate) values of parameters, which characterize entirely considered system (process). Methods for Data and Information Fusion 14

15. Institute for Parallel Processing - Bulgarian Academy of Sciences Parameter identificationMathematical description Linear dynamic system (Markovian presentation): Kalman filter gives optimal solution for Gaussian noises Methods for Data and Information Fusion 15

16. Institute for Parallel Processing - Bulgarian Academy of Sciences Description Markovian - semi Markovian Linear - non-linear dynamic system Additive - non additive system noise Gaussian - non Gaussian system noise Additive - non additive measurement noise Gaussian - non Gaussian measurement noise Methods for Data and Information Fusion 16

17. Institute for Parallel Processing - Bulgarian Academy of Sciences • The simplest tracking filter, considered in the paper, is alpha-beta filter. It is suitable for tracking of moving with constant velocity targets without steady-state error. The alpha-beta-gamma filter has ability to track even accelerating targets without steady-state error. • Kalman filter is a classical optimal estimating algorithm for dynamical linear system with Gaussian measurement and system noise. The modification of Kalman filter - Extended Kalman filter is developed for non-linear systems. The EKF gives particularly poor performance on highly non-linear functions because only the mean is propagated through the non-linearity. The unscented Kalman filter (UKF) uses a deterministic sampling technique to pick a minimal set of sample points (called sigma points) around the mean. Methods for Data and Information Fusion 17

18. Institute for Parallel Processing - Bulgarian Academy of Sciences • The theoretically most powerful approach for manoeuvring targets tracking is known to be Interacting Multiple Models estimator. Generalized Pseudo-Bayesian (GPB) estimators different orders, Fixed structure IMM, Variable Structure IMM, Probabilistic Data Association IMM are variants. The most important feature is that all these estimators use in parallel several models for modelling of the estimated system. • Particle filters, also known as Sequential Monte Carlo methods (SMC), are sophisticated model estimation techniques based on simulation. Particle filters generate a set of samples that approximate the filtering distribution to some degree of accuracy. Sampling Importance Resampling (SIR) filters with transition prior as importance function are commonly known as bootstrap filter and condensation algorithm. Methods for Data and Information Fusion 18

19. Institute for Parallel Processing - Bulgarian Academy of Sciences Temporal data fusion (Alan Steinberg) NN Nearest Neighbor PF Particle Filter F Alpha-Beta Filter PDAF Probabilistic Data Association Filter KF Kalman Filter JPDAF Joint Probabilistic Data Association Filter EKF Extended Kalman Filter FISST Finite Set Statistics IMM Interacting Multiple Model filter Y/N Good/Poor Choice; Methods for Data and Information Fusion 19

20. Institute for Parallel Processing - Bulgarian Academy of Sciences Alan Steinberg Methods for Data and Information Fusion 20

21. Institute for Parallel Processing - Bulgarian Academy of Sciences Fully Centralized Measurement Fusion Architecture Methods for Data and Information Fusion 21

22. Institute for Parallel Processing - Bulgarian Academy of Sciences Fully Centralized Trajectory Fusion Architecture Methods for Data and Information Fusion 22

23. Institute for Parallel Processing - Bulgarian Academy of Sciences Distributed Decision Fusion Architecture Methods for Data and Information Fusion 23

24. Institute for Parallel Processing - Bulgarian Academy of Sciences Simple Example When both sources are reliable, there is a consensus and it is reasonable to find solution in the cross-section of and - sets of corresponding sources: . If the two sources do not agree, we have . The hypothesis for reliability sources is no longer credible and three other hypotheses appear: 1) First source is correct, the second is incorrect; 2) First source is incorrect, but second is incorrect; 3) Both sources are incorrect. How to find the correct hypothesis? As a precaution, all available information is kept and we hold up . Methods for Data and Information Fusion 24

25. Institute for Parallel Processing - Bulgarian Academy of Sciences Example – continue It is obvious that the first fusion method is the most informative because the information is refined to the intersection of sets given by each source. It is also the most “risky” approach because the real value of is assumed to be inside a smaller set than the two initial sets. The secondfusion method is more reliable since all the information given by the two sources is preserved. The drawback of such an approach is a loss of accuracy since the set assumed to contain , is larger than each of the initial sets. Methods for Data and Information Fusion 25

26. Institute for Parallel Processing - Bulgarian Academy of Sciences Homogeneous sensor fusion • AND Operator. This method transforms the output of the sensors in a binary yes/no consensus operating with logical AND. After that thresholds are applied to find the result. The procedure is very simple, intuitive and fast, if the values of thresholds are determined in advance. The method does not take into account the degree of confidence of each sensor. • Weighted Average. This method takes a weighted average of available sensor data and uses it as the fused value. Usually the weights are proportional to accuracy of sensors or to credibility of sensor information. • Voting. The voting schemes main advantage is computation efficiency. Voting involves the derivation of an output data object from a collection of n input data objects, as prescribed by the requirements and constraints of a voting algorithm. The voting algorithms can be quite complex in terms of content and structure of the input data objects and how they handle the votes (weights) at input and output. Methods for Data and Information Fusion 26

27. Institute for Parallel Processing - Bulgarian Academy of Sciences Sensor fusion • Bayesian Theory. The use of Bayesian inference theory is widely spread for the fusion of redundant information. The most known method is the Kalman Filter, that is optimal in a statistical sense (it presents the least square error). Bayesian theory is also used to establish the weights linking the sensors in a weighted average fusion architecture. Moreover, some reductions of superbayesian methods to probabilistic evidence combination formulas have been provided. Some problems arise in a Bayesian framework: I) it does not distinguish between “lack of evidence” and “disbelief”; ii) practical difficulties in setting the apriori probabilities: noninformative priors can cause a wrong bias of further reasoning; iii) it assumes that the knowledge sources are consistent. Methods for Data and Information Fusion 27

28. Institute for Parallel Processing - Bulgarian Academy of Sciences Sensor Fusion • Information Theory. Mutual information, in the form of the Kullback-Leiber divergence, has been used in [12] as a way of combining probabilistic masses (sensor outputs). This is yet another method of fusing two probabilities, this time with a non-bayesian law, adding some information on average image values (e.g. depending on lighting conditions). The local maximum of the mutual information is then taken as the fused value. Methods for Data and Information Fusion 28

29. Institute for Parallel Processing - Bulgarian Academy of Sciences Sensor Fusion • Belief Theory. Dempster-Shafer evidential reasoning is used to compute the belief of a given event from two or more assessments provided by different knowledge sources at a symbolic level. This theory is based on the premise that each source of information provides only a partial belief about a proposition. Problem – redistribution of conflicts. • Dezert Smarandache Theory(DSmT). DSmT is analogous to Dempster-Shafer evidential reasoning theory but overcomes some drawbacks of this theory Methods for Data and Information Fusion 29

30. Institute for Parallel Processing - Bulgarian Academy of Sciences Sensor fusion • Fuzzy Reasoning. Fuzzy sets and variables are used to deal with real-world models where the usual ideal mathematical assumptions are inappropriate. Under the fuzzy framework, the possibility theory has emerged to represent imprecision in terms of fuzzy sets and to quantify uncertainty through four proposed notions: possibility, necessity, plausibility, and credibility distributions . • Geometric Methods, e.g. using uncertainty ellipsoids. Parametrical identification – if we know model, we can estimate parameters; Methods for Data and Information Fusion 30

31. Institute for Parallel Processing - Bulgarian Academy of Sciences Level 2,3,4 fusion • Belief Propagation Nets • Markov Random Fields • Factor Graphs • Game theory Methods for Data and Information Fusion 31

32. Institute for Parallel Processing - Bulgarian Academy of Sciences New research direction • New tracking filters – may be FISST, may be new one • Increased interest in image fusion methods - improvement of existing, search for new ones. • Increased interest on higher level fusion – not only theoretical but engineering approach • Decision level methods for fusion – like Dezert –Smarandache Theory or new ones. Methods for Data and Information Fusion 32