Person-Specific Domain Adaptation with Applications to Heterogeneous Face Recognition (HFR)

1. 2014.05.02 Oral Presentation: Person-Specific Domain Adaptation with Applications to Heterogeneous Face Recognition (HFR) Presenter: Yao-HungTsai Dept.ofElectricalEngineering,NTU

2. Outline • Face Recognition • Conventional Approach • Heterogeneous Face Recognition • Domain Adaptation Approach • Proposed Approach • Domain-independent Component Analysis • Person-specific Classifier • Combinational Framework • Experiments

3. Outline • Face Recognition • Conventional Approach • Heterogeneous Face Recognition • Domain Adaptation Approach • Proposed Approach • Domain-independent Component Analysis • Person-specific Classifier • Combinational Framework • Experiments

4. Heterogeneous Face Recognition • Face Recognition – Face Identification • Identify the subject from the captured images

5. Heterogeneous Face Recognition • Face Recognition – Face Verification • Verify a specific subject with respect to the captured image

6. Heterogeneous Face Recognition • Face Recognition Application Access Control System Photo auto-tagging Crime investigation ……

7. Outline • Face Recognition • Conventional Approach • Heterogeneous Face Recognition • Domain Adaptation Approach • Proposed Approach • Domain-independent Component Analysis • Person-specific Classifier • Combinational Framework • Experiments

8. Conventional Approach • Direct method • Direct compare two images based on their pixel valuesv.s. • Advantages : • Naïve, simple to implement • Disadvantages • Require lots of computation effort

9. Conventional Approach • Acommonmethod:Eigenface method • Representation: pixel intensity • Collecting several images as the training set: • Then we apply PCA to this set. = … d n

10. Conventional Approach • PCA • PCA projects columns of X from high-dimension ( ) to low dimension ( ). • PCA make projection variance maximized by optimize: • After solving the optimization we will get a set of basis vectors (faces): • We can reconstruct the images by: Ex: 2 dim to 1 dim Note: v1 will capture most data variance

11. Conventional Approach • The combinational coefficients will be the new feature of face: • For recognition, we simply project all images into this k-dimensional space and apply classifiers. Note: Same class cluster together.

12. Conventional Approach • However,thereexistseveralproblems • Traditional pattern recognition problems typically deal with • Training and test data collected from the same feature space • In real word applications, these data are • Collected from different feature domains • Exhibiting distinct feature distributions • We call this cross-domain recognition problems • Also called Heterogeneous Face Recognition (HFR)

13. Outline • Face Recognition • Conventional Approach • Heterogeneous Face Recognition • Domain Adaptation Approach • Proposed Approach • Domain-independent Component Analysis • Person-specific Classifier • Combinational Framework • Experiments

14. Heterogeneous Face Recognition • HFR is an emerging task in biometrics • SketchesinCriminal Cases • NightVisionCamera

15. Heterogeneous Face Recognition • Face recognition conduct on different domains

16. Heterogeneous Face Recognition • When conventional FR meets HFR … • If directly apply PCA on images cross domains (e.x. infra-red v.s. visible spectrum) • We visualize the data distribution of first 3 dimensions : NIR NIR Domain VIS VIS Domain

17. Heterogeneous Face Recognition • Observing the difference between domains : • Instances with same class are far from each others. • Data from same domain close to each others. • That is, • Domain difference dominates the data variance. • So, we need to conduct domain adaptation approach • For comparing images from source and target domain and

18. Outline • Face Recognition • Conventional Approach • Heterogeneous Face Recognition • Domain Adaptation Approach • Proposed Approach • Domain-independent Component Analysis • Person-specific Classifier • Combinational Framework • Experiments

19. Domain Adaptation Approach • There are numerous approaches of domain adaptation • Observing domain invariant features • Local Binary Patterns (LBP) – PAMI 2006 • Projecting images on common feature space • Canonical Correlation Analysis (CCA) • Partial Least Squares (PLS) – CVPR 2011 • Semi-coupled Dictionary Learning (SCDL) – CVPR 2012 • Coupled Dictionary Learning (CDL) – ICCV 2013 • Match distributions between cross domain images • Match marginal distributions (TCA) – TNN 2011 • Match also joint distributions (JDA) – ICCV 2014

20. Domain Adaptation Approach • Illustrate the notation of external data • Take access control system (ACS) as an example • At first, we usually cannot get the user’s images in advance • Thus, we need to use images from other subjects collected in advance to model the system • Let us call it external data External Data … … Note: Images from both domains need to be collected.

21. Domain Adaptation Approach • Most of the approaches requirealargenumberofpairedexternaldata • However,itisverydifficulttocollectthem ! • Collecting external data with no labeled information is much easier • Moreover, direct use of external data might be non-preferable • There’s no guarantee of the same feature distribution among external data and test data • For example, the common feature space observed from the face images of females will not generalize well to those of males.

22. Domain Adaptation Approach • So, I proposed an approach with the following properties • Require no labeled information in external data • Advocate the learning of person-specific domain adaptation model for HFR • DiCA ( Domain-independent Components Analysis) is proposed to build a common feature space

23. Outline • Face Recognition • Conventional Approach • Heterogeneous Face Recognition • Domain Adaptation Approach • Proposed Approach • Domain-independent Component Analysis • Person-specific Classifier • Combinational Framework • Experiments

24. Domain-independent Component Analysis • Review the observations in HFR problems • Domain difference dominates the data variance. • We check differences of projected means (MMD) for every dimensions of PCA space: and : mean of NIR : mean of VIS … MMD: -

25. Domain-independent Component Analysis • Then we can discard the components with high MMD value. • We get the final domain-independent projection matrix: Domain-independent Components Analysis: DiCA … External Data …

26. Domain-independent Component Analysis • So far, we can directly project user’s images to DiCA space and match test images. • However, to address the issue that subjects from external data are different from users and to improve the performance. • I further proposed • Person-specific Classifier (PC)

27. Outline • Face Recognition • Conventional Approach • Heterogeneous Face Recognition • Domain Adaptation Approach • Proposed Approach • Domain-independent Component Analysis • Person-specific Classifier • Combinational Framework • Experiments

28. Person-specific Classifier • Forming a specific classifier for the input test image, for this specific classifier outperforms than the general one • SVM (support vector machine) classifier is chose to be this person-specific classifier • Choose test data as positive instance. • User defined negativeinstancescouldbechosen for different usage. Negative Positive Person-specific classifier

29. Person-specific Classifier • Support Vector Machines (SVM) • Classifier to discriminate two categories data • Training dataset xi ∈ A+ ⇔ yi = 1 & xi ∈ A- ⇔ yi = -1

30. Person-specific Classifier • Goal : Predict the unseen class label for new data • Find a function f : Rn → R by learning from data f(x) ≥ 0 ⇒x ∈ A+ and f(x) < 0 ⇒ x ∈ A- • Simplest functionislinear:f (x) =w⊤x + b

31. Outline • Face Recognition • Conventional Approach • Heterogeneous Face Recognition • Domain Adaptation Approach • Proposed Approach • Domain-independent Component Analysis • Person-specific Classifier • Combinational Framework • Experiments

32. Combinational Framework … Test NIR Form DiCA External Data positive negative VIS Data Similarity Score NIR Data … Subspace User’s images … VIS User’s images

33. Outline • Face Recognition • Conventional Approach • Heterogeneous Face Recognition • Domain Adaptation Approach • Proposed Approach • Domain-independent Component Analysis • Person-specific Classifier • Combinational Framework • Experiments

34. Experiments • Two HFR scenario: • Photo – sketch (CUHK database) • VIS – NIR (CASIA 2.0 database) • Identification Task • For photo-sketch, there are 100 gallery images and 100 test images. • For VIS-NIR, there are 359 gallery images and 6200 test images (with different occlusions)

35. Experiments • Two HFR scenario: • Photo – sketch (CUHK database) • VIS – NIR (CASIA 2.0 database) • Identification Task • For photo-sketch, there are 100 gallery images and 100 test images. • For VIS-NIR, there are 359 gallery images and 6200 test images (with different occlusions)

36. Experiments • Sketch-to-photo Dataset • NIR-to-VIS Dataset

37. The End Thank You!