Acquisition of Semantic Classes for Adjectives from Distributional Evidence

1. Acquisition of Semantic Classes for Adjectives from Distributional Evidence Gemma Boleda Universitat Pompeu Fabra Barcelona

2. general picture • automatic classification of adjectives • Catalan • according to broad semantic characteristics • clustering • syntactic evidence

3. motivation • Lexical Acquisition • infer properties of words • lexical bottleneck • both symbolic and statistical approaches • adjectives • determining NP reference • the French general • establishing properties of entities • this maimai is round and sweet

4. motivation • initial motivation: POS-tagging • 55% remaining ambiguity involves adjectives general francès:‘French general’ or ‘general French’? • observations • general tendencies in syntactic behaviour of adjectives • ... which correspond to broad semantic properties • generalisation: best at semantic level • low-level tasks (POS-tagging) • initial schema for lexical semantic representation

5. approach • no general, well established semantic classification • have to build and test ours! • clustering: unsupervised technique • groups objects according to feature distribution • does not depend on pre-classification • provides insight into the nature of the data • shallow approach to syntax: n-grams • limited syntactic distribution • local relationship to arguments => test feasibility

6. Boleda, Badia, Batlle (2004) outline • adjective syntax and semantic classification • methodology • experiment 1 • experiment 2 • partial conclusions • outlook: rest of the thesis

7. outline • adjective syntax and semantic classification • methodology • experiment 1 • experiment 2 • partial conclusions • outlook: rest of the thesis

8. adjective syntax • default function: noun modifier (92%) • right of the noun (default position: 72%) • some to the left (‘epithets’: 28%) • predicative uses unfrequent (7%), but significant

9. two-way classification • number of arguments • unary: pilota vermella ‘red ball’ • binary: professor gelós de la Maria ‘teacher jealous of Maria’ • ontological kind (Ontological Semantics) • basic:vermell ‘red’ • object: malaltia pulmonar ‘pulmonary disease’ (=> lung) • event: propietat constitutiva ‘constitutive property’ (=> constitutes)

10. Ontological Semantics • coverage (ordinary cases) • machine tractability • explicit model of world: ontology • vermell => attribute::colour::red(x) • pulmonar => related-to::lung(x) • constitutiu => event::benef::constitute(x) • however: no commitment to particular framework

11. rationale • observation: syntactic preferences correspond to semantic properties • hypothesis: we can use syntactic features to infer semantic classes

12. outline • adjective syntax and semantic classification • methodology • experiment 1 • experiment 2 • conclusions and future work

13. data and procedure • 2283 adjectives >50 times in 16 million word Catalan corpus • lemma and morphological info • cluster the whole set • perform different tasks on different subsets • tuning subset: choose features • Gold Standard: evaluation and analysis

14. features and feature selection • features: • empirically chosen from blind distribution • double bigram, simplified POS-representation • tuning subset: 100 adjectives • choose features (distribution)

15. Fig. A: Feature selection

16. analysis • Gold Standard • 80 adjectives • annotated by 3 human judges, acceptable agreement (92 and 84%, .72 and .74 kappa)

17. outline • adjective syntax and semantic classification • methodology • experiment 1 • experiment 2 • partial conclusions • outlook: rest of the thesis

18. experiment 1: unary / binary • final evaluation:10 features, raw percentage • clustering algorithm: k-means (cosine) • predictions: • binary adjectives cooccur with prepositions more frequently than unary ones • unary adjectives are more flexible

19. unary (yellow) binary (red) Fig. B: Clusters vs. unary/binary unary / binary: results • agreement with Gold Standard: • 97%, kappa = 0.87 • comparable to humans • features:

20. outline • adjective syntax and semantic classification • methodology • experiment 1 • experiment 2 • partial conclusions • outlook: rest of the thesis

21. experiment 2: basic / object / event • final evaluation: 32 features, normalisation • clustering algorithm: k-means (cosine) • predictions: • basic adjectives are flexible, work as epithets, occur in predicative contexts, appear further from the noun • object adjectives appear rigidly after the noun • event adjectives tend to occur in predicative positions and do not act as epithets

22. Fig C: Clusters vs. basic/event/object basic / object / event: results object (yellow) • agreement with Gold Standard: • 73%, kappa = 0.56 • lower than humans • features: event (orange) basic (red)

23. Fig C: Clusters vs. basic/event/object Fig D: Clusters vs. unary/binary basic/object/event: error analysis • something has gone wrong! • characterisation of event adjectives basic adjectives with an object reading (polysemy) binary! unary event adjectives binary event adjectives

24. outline • adjective syntax and semantic classification • methodology • experiment 1 • experiment 2 • partial conclusions • outlook: rest of the thesis

25. partial conclusions • overall, results seem to back up: • use of syntax-semantics interface for adjectives • linguistic predictions as to relevant features and differences across classes • shallow approach • unary / binary: piece of cake • few binary adjectives, but worth spotting (denote relationships)

26. partial conclusions • basic / object / event: need reworking • object adjectives seem to be the most robust class • variation in basic adjectives (default class), polysemy • event adjectives: seem to behave much like basic adjectives with respect to features chosen => redefine class?

27. outline • adjective syntax and semantic classification • methodology • experiment 1 • experiment 2 • partial conclusions • outlook: rest of the thesis

28. outlook: rest of the thesis • rethink classification • redefine features in light of results • integrate polysemy judgments into the experiment and analysis • perform experiments with other corpora

29. classification • what to do with event adjectives? cp.: • constitutiu ‘constitutive’ (“active”) • legible ‘readable’ (“passive”) • reproductor ‘reproducing’ (“active, habituality”) • yet another parameter: gradability • important for adjectives • should be easy to induce

30. better blind distribution or self-defined features? • n-grams: sparseness, selection • other features? • account for different levels of description

31. polysemy • crucial aspect, explains much of results • difficult to integrate! • meaningless kappa values • alternatives? • clearer definition of polysemy within task • specific tests • other resources: dictionary?

32. other resources • CUCWeb (208 million word) http://www.catedratelefonica.upf.es • test whether “more data is better data” (Mercer and Church 1993: 18-19) • advantages and challenges of Web corpora • current results: for verb subcategorisation experiment, results 12 points lower than using smaller, balanced, controled corpus

33. Acquisition of Semantic Classes for Adjectives from Distributional Evidence Gemma Boleda Universitat Pompeu Fabra Barcelona