Cross Domain Distribution Adaptation via Kernel Mapping

1. Cross Domain Distribution Adaptation via Kernel Mapping Erheng Zhong†Wei Fan‡ Jing Peng* Kun Zhang# Jiangtao Ren† Deepak Turaga‡ Olivier Verscheure‡ †Sun Yat-Sen University ‡IBM T. J. Watson Research Center *Montclair State University #Xavier University of Lousiana

2. Can We?

3. Standard Supervised Learning training (labeled)‏ test (unlabeled)‏ Classifier 85.5% New York Times New York Times

4. In Reality…… training (labeled)‏ test (unlabeled)‏ Classifier 64.1% Labeled data not available! Reuters New York Times New York Times

5. Domain Difference->Performance Drop train test ideal setting Classifier NYT NYT 85.5% New York Times New York Times realistic setting Classifier NYT Reuters 64.1% Reuters New York Times

6. Synthetic Example

7. Synthetic Example

8. Main Challenge  Motivation • Both the marginal and conditional distributions between target-domain and source-domain could be significantly different in the original space!! Could we remove those useless source-domain data? Could we find other feature spaces? How to get rid of these differences?

9. Main Flow Kernel Discriminant Analysis

10. Kernel Mapping

11. Instances Selection

12. Ensemble

13. Properties • Kernel mapping can reduce the difference of marginal distributions between source and target domains. [Theorem 2] • Both source and target domain after kernal mapping are approximately Gaussian. • Cluster-based instances selection can select those data from source domain with similar conditional probabilities. [Cluster Assumption, Theorem 1] • Error rate of the proposed approach can be bounded; [Theorem 3] • Ensemble can further reduce the transfer risk. [Theorem 4]

14. 20 News groups (Reuters) Target-Domain First fill up the “GAP”, then use knn classifier to do classification comp rec SyskillWebert comp.sys rec.sport Source-Domain Target-Domain Source-Domain Sheep comp.graphics rec.auto Bands-recording First fill up the “GAP”, then use knn classifier to do classification Biomedical Goats Experiment – Data Set • Reuters • 21758 Reuters news articles • 20 News Groups • 20000 newsgroup articles • SyskillWebert • HTML source of web pages plus the ratings of a user on those web pages from 4 different subjects • All of them are high dimension (>1000)!

15. Experiment -- Baseline methods • Non-transfer single classifiers • Transfer learning algorithm TrAdaBoost. • Base classifiers: • K-NN • SVM • NaiveBayes

16. Experiment -- Overall Performance • kMapEnsemble -> 24 win, 3 lose! Dataset 1~9

17. Conclusion • Domain transfer when margin and conditional distributions are different between two domains. • Flow • Step-1 Kernel mapping -- Bring two domains’ marginal distributions closer; • Step-2 Cluster-based instances selection -- Make conditional distribution transferable; • Step-3 Ensemble – Further reduce the transfer risk. • Code and data available from the authors.