Generic Framework for Context-Dependent Fusion with Application to Landmine Detection

1. Generic Framework for Context-Dependent Fusion with Application to Landmine Detection Ahmed Chamseddine Ben Abdallah Multimedia Research Lab CECS Department University of Louisville June 2009

2. Outline • Motivational Example • Related Work • Global Fusion • Local Fusion • Contributions • Context-Extraction for Local Fusion (CELF) • CELF with Feature Discrimination (CELF-FD) • Application to Landmine detection • Conclusions and Future Work

3. Motivational Example

4. Global Fusion: Ask the audience or the wisdom of crowds ? Majority Voting

5. Example 1

6. Example 2

7. Example 3 (1)

8. Expert selection: Call a friend Artist CECS Phd Student Economist Musician Doctor News Music Cooking Fashion Music Music Movies Fashion Politics Literature Nurse Basketball player

9. A C Doctor News Music A C A Medical question D B B C C A B Nurse What is the scientific name of the swine Flu virus? A: H1N1 A: H1N1 B: H3N2 C: H2N2 D: H5N1

10. A C Doctor News Music B C A Medical question D B B C C A B Musician Fashion Music What is best-selling music album worldwide? A: Come on Over B: Thriller B: Thriller C: Falling into You D: Daydream

11. Economist A C A A A Medical question D B B C C A B CECS Phd Student Which of the following is not a prime number? A: 19 B: 29 C: 39 C: 39 D: 59

12. Literature Art Cooking Math Medicine Hey… I know about Music! Q3 Q1 Q2 Hey… I know about fashion! Sport Fashion Music

13. Conclusions • Multiple sources better than single source need for fusion. • Local fusion is better than global fusion. • Grouping experts. • Identifying the context/domain. • Combining the experts’ decision.

14. Related Work Global Fusion

15. Global fusion • When combining multiple independent and diversedecisions each of which is at least more accuratethan random guessing, random errors cancel each other out, correct decisions are reinforced.

16. Classifier fusion architecture Combiner … … Classifier 2 Classifier 1 Classifier K Decision Level … Feature Extraction 2 Feature Extraction 1 Feature Extraction K Feature Level … Raw Data 2 Raw Data 1 Raw Data K Data Level

17. Global Fusion approaches • Bayesian Fusion • ANN Fusion • Borda Count Fusion • Dempster-Shafer Fusion • Decision Template Fusion • Fuzzy Integral • …

18. Related Work Local Fusion

19. Local fusion • Divide and Conquer approach Classifier output Feature set Context extraction Decision fusion Two independent tasks Global decision

20. Local Fusion approaches • Category 1: find the neighborhood of the testing sample and create a fusion model in the testing phase • Dynamic classifier by local accuracy,… • time consuming • Category 2: cluster and create fusion models in the training phase • Clustering and selection, Context-Dependent Fusion, …. • Treats the context extraction and the decision fusion components independently.

21. Contributions

22. Contributions • A local fusion approach (CELF) based on a novel objective function that combines • Context identification • Multi-algorithm fusion • CELF with adaptive feature weight assignment (CELF-FD) • Application to landmine detection

23. Contributions 1- Context Extraction for Local Fusion (CELF)

24. Decision vector Ground truth Notations (xj, yj, tj) yj1 yj2 yjK … Classifier 2 Classifier 1 Classifier K Feature vector (xj)1 (xj)2 (xj)K … Feature Extraction 2 Feature Extraction 1 Feature Extraction K Data sample j Ground truth tj

25. Clustering (u1,5,u2,5,u3,5) x6 x7 • FCM Algorithm • Update ci • Update uij • where x1 x4 x8 x x9 c2 x2 x x10 c1 x5 x3 x11 x13 x12 x x14 c3 x15

26. Classification w1 Conf2 w2 Conf1

27. Decision space Proposed approach Confidence space Decision space Decision space

28. Context-Extraction for Local Fusion (CELF) • CELF Combines: • Context identification • Multi-algorithm fusion Train Samples (Features, Algorithm outputs) Context extraction Decision fusion Fusion output

29. Objective Function Classification component Clustering Component: FCM type

30. Update Equations (1) • OptimizingJ w.r.t the centers yields

31. Similarity in the feature space  Compact clusters Deviation from desired output  Clusters with consistent fusion weights Update Equations (2) • OptimizingJ w.r.t the membership yields

32. Update Equation (3) • OptimizingJ w.r.t the aggregation weights yields

33. CELF Algorithm • Initialize U and W. • repeat • Update cluster centers. • Update W. • Update U. • until stopping condition satisfied • return Centers, U, W

34. Illustrative example (1) Decision space Feature space Classifier 2 Classifier 1

35. Illustrative Example (2)

36. Illustrative Example (3) Classifier 1 CELF P(Y) Classifier 2 Y

37. Contributions 2- Context Extraction for Local Fusion with feature discrimination (CELF)

38. Context Extraction for Local Fusion with feature discrimination(CELF-FD) • For high dimensional spaces, standard clustering algorithms cannot generate a meaningful partition. • To alleviate this drawback, we introduce feature weighting aspect. • CELF-FD combines: • Clustering • Feature Discrimination • Selection of local expert classifiers

39. Notations • Vset of features weights. Cluster i Features from classifier l

40. Objective Function Optimized in CELF-FD Feature discrimination

41. CELF-FD Algorithm • Initialize U, V and W. • repeat • Update cluster centers. • Update W. • Update U. • Update V. • until stopping condition satisfied • return Centers, U, V, W

42. Toy data (1)

43. Toy data (2) Classifier 1 Accuracy 69% Classifier 2 Accuracy 81%

44. Experimental results using CELF (1)

45. Experimental results using CELF (2) P(Y) Y

46. Contributions 3- Application to landmine detection

47. Landmine problem • Objective: analyze data collected by multiple sensors and make a decision if there is buried mine. • Different mine types • Soil properties: • Asphalt, gravel, sand… • Varying density • Water held by vegetation roots • Rain, snow • Various minerals

48. Ground Penetration component Autonomous Mine Detection System Vehicle

49. WEMI Data • taken at 21 frequencies (logarithmically spaced from 330 Hz to 90.03 KHz). Magnitude Frequency Frequency NMC Blank Magnitude Frequency Frequency HMC LM mine

50. Landmine detectors • Landmine detectors using GPR • EHD • SCF • HMM • Landmine detector using WEMI