Cross Domain Distribution Adaptation via Kernel Mapping

1. Cross Domain Distribution Adaptation via Kernel Mapping Erheng Zhong† Wei Fan‡ Jing Peng* Kun Zhang# Jiangtao Ren† Deepak Turaga‡ Olivier Verscheure‡ †Sun Yat-Sen University ‡IBM T. J. Watson Research Center *Montclair State University #Xavier University of Lousiana

2. Standard Supervised Learning training (labeled)‏ test (unlabeled)‏ Classifier 85.5% New York Times New York Times

3. In Reality…… training (labeled)‏ test (unlabeled)‏ Classifier 64.1% Labeled data not available! Reuters New York Times New York Times

4. Domain Difference->Performance Drop train test ideal setting Classifier NYT NYT 85.5% New York Times New York Times realistic setting Classifier NYT Reuters 64.1% Reuters New York Times

5. Synthetic Example

6. Synthetic Example

7. Main Challenge  Motivation • Both the marginal and conditional distributions between target-domain and source-domain could be significantly different in the original space!! Could we remove those useless source-domain data? Could we find other feature spaces? How to get rid of these differences?

8. Main Flow Kernel Discriminant Analysis

9. Kernel Mapping

10. Instances Selection

11. Ensemble

12. Properties It assumes that if two point x1 and x2 are close in the intrinsic geometry of q(x), then the conditionals r(y|x1) and r(y|x2) are similar. Both the target and source domain data follow appropriate Gaussian distributions in a Gaussian RKHS. • Kernel mapping can reduce the difference of marginal distributions between source and target domains. [Theorem 2] • Cluster-based instances selection can select those data from source domain with similar conditional probabilities. [Cluster Assumption] • Error rate of the proposed approach can be bounded; [Theorem 3] • Ensemble can further reduce the transfer risk. [Theorem 4]

13. 20 News groups (Reuters) Target-Domain First fill up the “GAP”, then use knn classifier to do classification comp rec SyskillWebert comp.sys rec.sport Source-Domain Target-Domain Source-Domain Sheep comp.graphics rec.auto Bands-recording First fill up the “GAP”, then use knn classifier to do classification Biomedical Goats Experiment – Data Set • Reuters • 21758 Reuters news articles • 20 News Groups • 20000 newsgroup articles • SyskillWebert • HTML source of web pages plus the ratings of a user on those web pages from 4 different subjects • All of them are high dimension (>1000)!

14. Experiment -- Baseline methods • Non-transfer single classifiers • Transfer learning algorithm TrAdaBoost. • Base classifiers: • K-NN • SVM • NaiveBayes

15. Experiment -- Overall Performance • kMapEnsemble -> 24 win, 3 lose! Dataset 1~9

16. Conclusion • Domain transfer when margin and conditional distributions are different between two domains. • Flow • Step-1 Kernel mapping -- Bring two domains’ marginal distributions closer; • Step-2 Cluster-based instances selection -- Make conditional distribution transferable; • Step-3 Ensemble – Further reduce the transfer risk. • Code and data available from the authors.