Modeling Semantic Similarities in Multiple Maps

1. Modeling Semantic Similarities inMultiple Maps Presenter : Wei-Hao Huang Authors : Laurens van der Maaten, Geoffrey Hinton EWI-ICT TR, 2009

2. Outlines • Motivation • Objectives • Methodology • Experiments • Conclusions • Comments

3. Motivation • Semantic space models cannot faithfully represent intransitive pairwise similarities or the similarities of words that have multiple meanings. • Triangle inequality • Nearest neighbor is limited • Similarities are symmetric suit dog dog tuxedo Animal tie dog rope knot dog dog North Korea China

4. Objectives • To propose multiple map SNE to solve fundamental limitations of metric spaces suit tuxedo tie suit tuxedo tie rope knot rope tie knot

5. Methodology Data Data Mixing proportion (importance) SNE Multiple maps SNE Map3 Map2 Map Map1 Stochastic neighbor embedding Multiple maps SNE

6. Methodology suit tuxedo tie rope knot Stochastic neighbor embedding

7. Methodology dog dog dog dog suit rope animal animal tie tie dog dog Multiple map SNE

8. Methodology A*C=1*1/2=1/2 B*C=1*1/2=1/2 Multiple map SNE

9. Experiments • Visualization Experiments • Florida State University word association dataset • Selecting 5019 words • Generalization Experiments • To evidence their model for semantic representation • Training data: 80% • Validation data: 10% • Test data: 10%

10. Experiments Visualization

11. Experiments Statue of Liberty Sport Cheerleader Clothing monarchy Tie Tie Cheerleader monarchy

12. Experiments Generalization

13. Experiments • Comparing multiple maps SNE with other method. • Semantic space models • Semantic networks • Topic models

14. Conclusions The multiple maps SNE alleviates the fundamental limitations of metric spaces. Multiple map model has characteristics that are similar to those of topic models.

15. Comments • Advantages • Multiple maps SNE alleviates the fundamental limitations of metric spaces • Applications • Data visualization • Semantic similarities