1 / 13

Shuang-Hong Yang, Hongyuan Zha, Bao-Gang Hu NIPS2009

Dirichlet-Bernoulli Alignment: A Generative Model for Multi-Class Multi-Label Multi-Instance Corpora. Shuang-Hong Yang, Hongyuan Zha, Bao-Gang Hu NIPS2009. Presented by Haojun Chen. Some contents are from author’s paper and poster. Outline. Introduction

oralee
Download Presentation

Shuang-Hong Yang, Hongyuan Zha, Bao-Gang Hu NIPS2009

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Dirichlet-Bernoulli Alignment: A Generative Model for Multi-Class Multi-Label Multi-Instance Corpora Shuang-Hong Yang, Hongyuan Zha, Bao-Gang Hu NIPS2009 Presented by Haojun Chen Some contents are from author’s paper and poster

  2. Outline • Introduction • Dirichelet-Bernoulli Alignment (DBA) Model • Model Inference and Prediction • Experiments • Conclusion

  3. Pattern Class Instance Feature Introduction • In this paper, multi-class, multi-label and multi-instance classification (M3C) problem is considered. • Goal: infer class label for both pattern and its instances (e.g. document) (e.g. topic) (e.g. paragraph) (e.g. word) Figure is adopted from author’s poster

  4. : set of input patterns : corresponding labels : dictionary features : set of instances in pattern n : a bag of discrete features : class label Problem Formalization • For a multi-class, multi-label multi-instance corpus , we define

  5. Basic Assumption • Assumption 1 [Exchangeability]: A corpus is a bag of patterns, and each pattern is a bag of instances. • Assumption 2 [Distinguishablity]: Each pattern can belong to several classes, but each instance belongs to a single class. Tree Structure Assumption

  6. Dirichelet-Bernoulli Alignment (DBA) Model (1) DBA generative process: • Sample pattern-level class mixture

  7. Dirichelet-Bernoulli Alignment (DBA) Model (2) • For each of the M instances in X • Choose instance-level class label • Generate the instance according to observation model

  8. Dirichelet-Bernoulli Alignment (DBA) Model (3) • Generate pattern-level label where

  9. Model Inference and Prediction • Parameter Estimation (MLE) • Variational Approximation • Prediction • Pattern Classification: • Instance Disambiguation:

  10. Why The Name? • Lower bound Fourth term

  11. Experiments 1 • Text classification • ModApte split of the Reuters-21578 text collection, 10788 documents, 10 classes • Each paragraph of a document is represented with Vector-Space-Model • Eliminate docs with empty label sets, length<20. Remaining 1879 docs, 721 docs (38.4%) with multiple labels • Compared with Multinomial-event-model-based Naive-Bayes (MNB) and two state-of-art multi-instance multi-label classifiers (MIMLSVM and MIMLBOOST)

  12. Experiments 2 • Named entity disambiguation • Yahoo! Answer query log crawled in 2008,101 classes, 216563questions • 300 entities for training and 100 for test • Compared with Multinomial Naive Bayes with TF (MNBTF) or TFIDF (MNBTFIDF) attributes, as well as linear SVM classifier with TF (SVMTF) or TFIDF (SVMTFIDF) attributes.

  13. Conclusion • A Dirichlet-Bernoulli Alignment model is proposed and proved to be useful for both pattern classification and instance disambiguation

More Related