A Generic Framework for Context-Dependent Fusion with Application to Landmine Detection

1. Multimedia Research Lab CECS Department University of Louisville A Generic Framework for Context-Dependent Fusion with Application to Landmine Detection Ahmed Chamseddine Ben Abdallah December 2010

2. Outline • Introduction • Related Work • Contributions • Experimental Results • Conclusions and Future Work

3. 1. Introduction

4. Need for Fusion Fusion Sensor Feature Extraction Classifier decision Data final decision Feature Extraction Classifier Sensor Feature Extraction Classifier Data

5. Landmine Detection Problem Fusion Edge Features Classifier GPR (e.g. AT mines) final decision Frequency Features Classifier (e.g. deep mines) Feature Extraction Classifier WEMI (e.g. HM mines)

6. Motivation for local fusion (1) Performance on site A Performance on site B Performance of the different detectors for the whole data set Detector 1 Detector 2 Detector 3 Detector 4

7. Motivation for local fusion (2) High Anti-Personal High Metal Anti-Tank High Metal Metal Detector Performance Anti-Personal Low Metal Anti-Tank Low Metal Low Low High Ground Penetrating Radar Performance

8. Multiple sources better than single source need for fusion. • Local fusion is better than global fusion. • Grouping experts. • Identifying the context/domain. • Combining the experts’ decision.

9. 2. Related Work

10. Global fusion • “When combining multiple independent and diversedecisions each of which is at least more accuratethan random guessing, random errors cancel each other out, correct decisions are reinforced.”

11. Classifier fusion architecture Combiner … Classifier 2 Classifier 1 Classifier K Decision Level … Feature Extraction 2 Feature Extraction 1 Feature Extraction K Feature Level … Raw Data 2 Raw Data 1 Raw Data K Data Level

12. Global Fusion approaches • Bayesian Fusion • ANN Fusion • Borda Count Fusion • Dempster-Shafer Fusion • Decision Template Fusion • Fuzzy Integral • … Global Fusion Classifier 1 Classifier 2 Fusion Result … Data Sample Classifier K

13. Local Fusion approaches • Category 1: find the neighborhood of the testing sample and create a fusion model in the testing phase • Dynamic classifier by local accuracy,… • time consuming • Category 2: cluster and create fusion models in the training phase • Clustering and selection, Context-Dependent Fusion, …. • Treats the context extraction and the decision fusion components independently.

14. 3. Contributions

15. Context-Extraction for Local Fusion (CELF) • CELF is a generic framework for context-dependent fusion. • Local fusion approach. • Based on optimizing one objective function that aims to: • partition the input feature space into different contexts, and • identify the relevant classifiers for each context simultaneously.

16. Architecture

17. Contributions • Feature Discrimination • assign a relevance weight to each feature within each context • Optimizing the number of clusters • add a regularization term to the objective function. • start by a large C • let the clusters compete for data points until convergence • Multi-class data • Non-linear local fusion using: • Neural Networks • Fuzzy Integrals • Landmine Detection • Using GPR system • Using AMDS system • Semantic video indexing • Image database categorization • Phoneme recognition • The baseline CELF: • formulate the objective function • optimize it • construct an iterative algorithm CELF-NN CELF-FD CELF-CA CELF-M CELF-FI CELF Applications

18. Notations Decision vector Ground truth (xj, yj, tj) y1j y2j yKj … Classifier 2 Classifier 1 Classifier K Feature vector x1j x2j xKj … Feature Extraction 2 Feature Extraction 1 Feature Extraction K Data sample j Ground truth tj

19. CELF – Objective Function Classification component Clustering Component: FCM type

20. Clustering Component (u1,5,u2,5,u3,5) x1 x4 x c1 x2 x5 x3 x6 x7 x c2 x9 x13 x8 x11 x14 x c3 x10 x12 x15

21. Classification Component w1 Conf2 w2 Conf1

22. Proposed Approach Feature space Decision space Decision space Decision space Decision space Decision space x2 y2 x1 y1

23. Similarity in the feature space  Compact clusters Deviation from desired output  Clusters with consistent fusion weights CELF – Update Equations • Centers • Aggregation weights • Memberships

24. CELF - Algorithm • Initialize U and W. • repeat • Update cluster centers. • Update W. • Update U. • until stopping condition satisfied • return Centers, U, W

25. Toy data (1) Data in the feature space

26. Toy data (2) Classifier 1 Accuracy 69% Classifier 2 Accuracy 81%

27. Experimental results using CELF (1) Clusters obtained with CELF

28. Experimental results using CELF (1) Classifier 1 Clusters obtained with CELF Classifier 2

29. Experimental results using CELF (1) Classifier 1 Clusters obtained with CELF Classifier 2

30. Experimental results using CELF (1) Classifier 1 Clusters obtained with CELF Classifier 2

31. Experimental results using CELF (2) Results by Global fusion Distribution of the confidence values assigned by CELF Frequency Results by clustering followed by fusion Confidence

32. Feature Discrimination • We introduce a feature weighting aspect to generate more meaningful clusters. • This allows finding clusters in subspaces of the original sparse and high dimensional feature space. Clustering with equal feature weights Clustering with weighted features Data in the feature space

33. CELF-FD Optimized in CELF-FD Feature discrimination

34. Optimizing the number of clusters ?

35. CELF-CA • To optimize the number of clusters, we extend CELF to support Competitive Agglomeration Optimized in CELF-CA Regularization term

36. CELF-M • CELF was designed for 2-class labeled data • CELF-M is an extension of CELF to support multi-class data

37. CELF-NN (1) • CELF uses a simple linear aggregation to assign fusion weights to the individual classifiers. • CELF-NN uses Neural Networks for local fusion. Decision space y2 y1

38. CELF-NN (2)

39. CELF-NN (3)

40. Choquet Integral • Choquet integral is a non-linear fusion approach that assigns weighs to subsets of classifiers to take into account the interaction between them.

41. Sugeno measure

42. CELF-FI • CELF-FI uses Choquet Integrals for local fusion.

43. Landmine detection GPR system AMDS system Semantic video indexing Image database categorization Phoneme recognition 4. Experimental Results

44. AMDS System: Ground Penetrating Radar Sensor Autonomous Mine Detection System Vehicle

45. AMDS System: WEMI Sensor • taken at 21 frequencies (logarithmically spaced from 330 Hz to 90.03 KHz). Magnitude Frequency Frequency Blank NMC Magnitude Frequency Frequency HMC LM mine

46. Landmine detectors • Landmine detectors using GPR • EHD: Edge based features with k-NN classifier. • SCF: Frequency based features. • HMM: Edge based features with HMM classifier. • Landmine detector using WEMI • MFIT: WEMI features with NN classifier.

47. Data collection • Data Collected from 2 different sites • 864 Alarms: • 308 mines, classified into 4 categories: • Anti-tank with high metal content (ATM) • Anti-tank with low metal content (ATLM) • Anti-personal with high metal content (APM) • Anti-personal with low metal content (APLM) • 556 False Alarms, classified into 3 categories: • High metal clutter (HMC) • Non-metal clutter (NMC) • Blank • Targets buried up to 5 inches deep.

48. Motivation for fusion

49. Fusion results’ (1)

50. Fusion results’ (2)