1 / 17

Knowledge Transfer via Multiple Model Local Structure Mapping

Knowledge Transfer via Multiple Model Local Structure Mapping. Jing Gao† Wei Fan‡ Jing Jiang†Jiawei Han† †University of Illinois at Urbana-Champaign ‡IBM T. J. Watson Research Center. Standard Supervised Learning. training (labeled). test (unlabeled). Classifier. 85.5%. New York Times.

rossa
Download Presentation

Knowledge Transfer via Multiple Model Local Structure Mapping

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Knowledge Transfer via Multiple Model Local Structure Mapping Jing Gao† Wei Fan‡ Jing Jiang†Jiawei Han† †University of Illinois at Urbana-Champaign ‡IBM T. J. Watson Research Center

  2. Standard Supervised Learning training (labeled) test (unlabeled) Classifier 85.5% New York Times New York Times

  3. In Reality…… training (labeled) test (unlabeled) Classifier 64.1% Labeled data not available! Reuters New York Times New York Times

  4. Domain Difference  Performance Drop train test ideal setting Classifier NYT NYT 85.5% New York Times New York Times realistic setting Classifier NYT Reuters 64.1% Reuters New York Times

  5. Other Examples • Spam filtering • Public email collection  personal inboxes • Intrusion detection • Existing types of intrusions  unknown types of intrusions • Sentiment analysis • Expert review articles blog review articles • The aim • To design learning methods that are aware of the training and test domain difference • Transfer learning • Adapt the classifiers learnt from the source domain to the new domain

  6. All Sources of Labeled Information test (completely unlabeled) training (labeled) Reuters Classifier ? …… New York Times Newsgroup

  7. A Synthetic Example Training (have conflicting concepts) Test Partially overlapping

  8. Goal Source Domain Source Domain Target Domain Source Domain • To unify knowledge that are consistent with the test domain from multiple source domains

  9. Summary of Contributions • Transfer from multiple source domains • Target domain has no labeled examples • Do not need to re-train • Rely on base models trained from each domain • The base models are not necessarily developed for transfer learning applications

  10. Locally Weighted Ensemble Training set 1 C1 X-feature value y-class label Training set 2 C2 Test example x …… …… Training set k Ck

  11. Optimal Local Weights Higher Weight 0.9 0.1 C1 Test example x 0.8 0.2 0.4 0.6 C2 • Optimal weights • Solution to a regression problem • Impossible to get since f is unknown!

  12. Graph-based Heuristics Higher Weight • Graph-based weights approximation • Map the structures of a model onto the structures of the test domain • Weight of a model is proportional to the similarity between its neighborhood graph and the clustering structure around x.

  13. Experiments Setup • Data Sets • Synthetic data sets • Spam filtering: public email collection  personal inboxes (u01, u02, u03) (ECML/PKDD 2006) • Text classification: same top-level classification problems with different sub-fields in the training and test sets (Newsgroup, Reuters) • Intrusion detection data: different types of intrusions in training and test sets. • Baseline Methods • One source domain: single models (WNN, LR, SVM) • Multiple source domains: SVM on each of the domains • Merge all source domains into one: ALL • Simple averaging ensemble: SMA • Locally weighted ensemble: LWE

  14. Experiments on Synthetic Data

  15. Experiments on Real Data

  16. Conclusions • Locally weighted ensemble framework • transfer useful knowledge from multiple source domains • Graph-based heuristics to compute weights • Make the framework practical and effective

  17. Thanks! • Any questions? Welcome to our poster tonight! http://www.ews.uiuc.edu/~jinggao3/kdd08transfer.htm jinggao3@illinois.edu

More Related