Combining Contexts in Lexicon Learning for Semantic Parsing

1. Combining Contexts in Lexicon Learning for Semantic Parsing May 25, 2007 NODALIDA 2007, Tartu, Estonia Richard Socher Saarland University Germany Chris Biemann University of Leipzig Germany Rainer Osswald FernUniversität Hagen Germany

2. Outline • Motivation: lexicon extension for semantic parsing • The semantic lexicon HaGenLex • Binary features and complex sorts • Method: bootstrapping via syntactic contexts • Results • Discussion

3. Motivation • Semantic parsing aims at finding a semantic representation for a sentence • Semantic parsing needs as a prerequisite semantic features of words. • Semantic features are obtained by manually creating lexicon entries (expensive in terms of time and money) • Given a certain amount of manually created lexicon entries, it might be possible to train a classifier in order to find more entries • Objective is Precision, Recall is secondary

4. HaGenLex: Semantic Lexicon for German size: 22,700 entries of these: 13,000 nouns, 6,700 verbs complex sort WORD SEMANTIC CLASS Aggressivität nonment-dyn-abs-situation Agonie nonment-stat-abs-situation Agrarprodukt nat-discrete Ägypter human-object Ahn human-object Ahndung nonment-dyn-abs-situation Ähnlichkeit relation Airbag nonax-mov-art-discrete Airbus mov-nonanimate-con-potag Airport art-con-geogr Ajatollah human-object Akademiker human-object Akademisierung nonment-dyn-abs-situation Akkordeon nonax-mov-art-discrete Akkreditierung nonment-dyn-abs-situation Akku ax-mov-art-discrete Akquisition nonment-dyn-abs-situation Akrobat human-object ... ...

5. Characteristics of complex sorts in HaGenLex In total, 50 complex sorts for nouns are constructed from allowed combinations of: • 16 semantic features (binary), e.g. HUMAN+, ARTIFICIAL- • 17 sorts (binary), e.g. concrete, abstract-situation... sort (hierarchy) semantic features complex sorts

6. Application: WOCADI-Parser „Welche Bücher von Peter Jackson über Expertensysteme wurden bei Addison-Wesley seit 1985 veröffentlicht?“

7. General Methodology Distributional Hypothesis projected on syntactic-semantic contexts for nouns: nouns of similar complex sort are found in similar contexts We use three kinds of context elements • Adjective Modifier • Verb-Subject (deep) • Verb-Object (deep) as assigned by the WOCADI parser for training 33 binary classifiers.

8. Data Corpus: • 3,068,945 sentences obtained from the Leipzig Corpora Collection • parser coverage: 42% • verb-deep-subject relations: 430,916 • verb-deep-object relations: 408,699 • adjective-noun relations: 450,184 Lexicon • 11,100 noun entries • lexicon extension: 10-fold cross validation on known nouns • Also unknown nouns will be classified

9. Bootstrapping Mechanism Algorithm: Initialize the training set; As long as new nouns get classified { calculate class probabilities for each context element; for all yet unclassified nouns n { Multiply class probs of context elements class-wise; Assign the class with highest probabilities to noun n; } } Class probabilities per context element: a) count number of per class b) normalize on total number of class wrt. noun classes c) normalize to row sum=1 A threshold  regulates the minimum number of different context elements a noun co-occurs with in order to be classified

10. From binary classes to complex sorts • Binary classifiers for single features for all three context element types are combined into one feature assignment: • Lenient: voting • Strict: all classifiers for different context types agree • Combining the outcome: safe choices ANIMAL +/- ANIMATE +/- Selection: compatible complex sorts that are minimal w.r.t hierarchy and unambiguous. ARTIF +/- AXIAL +/- result classorreject ... (16 features) ab +/- abs +/- ad +/- as +/- ... (17 sorts)

11. Results: binary classes for different context types most of the binary features are highly biased =5 =1

12. Combination of context types =1

13. Results for complex sorts =5 =1 Complex sorts with highest training frequency

14. Typical mistakes Pflanze (plant) animal-object instead of plant-object zart, fleischfressend, fressend, verändert, genmanipuliert, transgen, exotisch, selten, giftig, stinkend, wachsend... Nachwuchs (offspring)human-object instead of animal-object wissenschaftlich, qualifiziert, akademisch, eigen, talentiert, weiblich, hoffnungsvoll, geeignet, begabt, journalistisch... Café (café)art-con-geogr instead of nonmov-art-discrete(cf. Restaurant) Wiener, klein, türkisch, kurdisch, romanisch, cyber, philosophisch, besucht, traditionsreich, schnieke, gutbesucht, ... Neger (negro)animal-object instead of human-object weiß, dreckig, gefangen, faul, alt, schwarz, nackt, lieb, gut, brav but: Skinhead (skinhead)human-object (ok) {16,17,18,19,20,21,22,23,30}ährig, gleichaltrig, zusammengeprügelt, rechtsradikal, brutal In most cases the wrong class is semantically close. Evaluation metrics did not account for that.

15. Discussion of Results Binary features: • Precision >98% for most binary features • Assigning the smaller class is hard for bias>0.9 Context types • verb-subject and verb-object are better than adjective • verb-subject is best single context for complex sorts • combination always helps for binary features Complex sorts • Todo: more lenient combination procedure to increase recall

16. Conclusion • Method for semantic lexicon extension • High precision for binary semantic features • Unknown nouns: • For 3,755 nouns not in the lexicon, a total of 125,491 binary features was assigned. • For 1,041 unknown nouns, a complex sort was assigned • Combination to complex sorts yet to be improved • Combination of different context types improves results

17. Any Questions? Thank you very much!