1 / 61

Extraction of Coocurrence Data

Extraction of Coocurrence Data. Basic Assumptions. The collocates of a collocation cooccur more frequently within text than arbitrary word combinations. (Recurrence) Stricter control of cooccurrence data leads to more meaningful results in collocation extraction. Word (Co)occurrence.

aadi
Download Presentation

Extraction of Coocurrence Data

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Extraction of Coocurrence Data

  2. Basic Assumptions • The collocates of a collocation cooccur more frequently within text than arbitrary word combinations. (Recurrence) • Stricter control of cooccurrence data leads to more meaningful results in collocation extraction.

  3. Word (Co)occurrence • Distribution of words and word combinations in text approximately described by Zipf’s law. • Distribution of combinations is more “extreme” than that of individual words.

  4. Word (Co)occurrence • Zipf 's law: • nm is the number of different words occurring m times • i.e., there is a large number of low-frequency words, and few high-frequency ones

  5. Word (Co)occurrence • Collocations will be preferably found among highly recurrent word combinations extracted from text. • Large amounts of text need to be processed to obtain sufficient number of high-frequency combinations.

  6. Control of Candidate Data • Extract collocations from relational bigrams • Syntactic homogeneity of candidate data • (Grammatical) cleanness of candidates e.g. N+V pairs: Subject+V vs. Object+V • Text type, domain, and size of source corpus influence the outcome of collocation extraction

  7. Terminology • Extraction corpustokenized, pos-tagged or syntactically analysed text • Base datalist of bigrams found in corpus • Cooccurrence databigrams with contingency tables • Collocation candidatesranked bigrams

  8. Types and Tokens • Frequency counts (from corpora) • identify labelled units (tokens),e.g. words, NPs, Adj-N pairs • set of different labels (types) • type frequency = number of tokens labelled with this type • example:... whatthe blackbox does ...

  9. box Types and Tokens • Frequency counts (from corpora) • identify labelled units (tokens),e.g. words, NPs, Adj-N pairs • set of different labels (types) • type frequency = number of tokens labelled with this type • example:... whatthe blackbox does ...

  10. Types and Tokens • Counting cooccurrences • bigram tokens = pairs of word tokens • bigram types = pairs of word types • contingency table = four-way classification of bigram tokens according to their components

  11. Contingency Tables contingency table for pair type (u,v)

  12. Collocation Extraction: Processing Steps • Corpus preprocessing • tokenization (orthographic words) • pos-tagging • morphological analysis / lemmatization • partial parsing • (full parsing)

  13. Collocation Extraction: Processing Steps • Extraction of base data from corpus • adjacent word pairs • Adj-N pairs from NP chunks • Object-V & Subject-V from parse trees • Calculation of cooccurrence data • compute contingency table for each pair type (u,v)

  14. Collocation Extraction: Processing Steps • Ranking of cooccurrence data by "association scores" • measure statistical association between types u and v • true collocations should obtain high scores • using association measures (AM) • N-best list = listing of N highest-ranked collocation candidates

  15. Base Data:How to get? • Adj-N • adjacency data • numerical span • NP chunking • (lemmatized)

  16. Base Data:How to get? • V-N • adjacency data • sentence window • (partial) parsing • identification of grammatical relations • (lemmatized)

  17. Base Data:How to get? • PP-V • adjacency data • PP chunking • separable verb particles(in German) • (full syntactic analysis) • (lemmatization?)

  18. Adj-N In the first place, the ‘less genes, more behavioural flexibility’ argument is a total red herring. In/PRP the/ART first/ORD place/N ,/$, the/ART ‘/$’ less/ADJ genes/N ,/$, more/ADJ behavioural/ADJ flexibility/N ’/$’ argument/N is/V a/ART total/ADJ red/ADJ herring/N ./$.

  19. span size 1 (adjacency)wj, j= -1 first/ORD place/N less/ADJ genes/N behavioural/ADJ flexibility/N ’/$’ argument/N red/ADJ herring/N Adj-N:poswi = N

  20. more/ADJ flexibility/N ’/$’ argument/N flexibility/N argument/N red/ADJ herring/N total/ADJ herring/N span size 2wj, j = -2, -1 first/ORD place/N the/ART place/N less/ADJ genes/N ‘/$’ genes/N behavioural/ADJ flexibility/N Adj-N:poswi = N

  21. more/ADJ flexibility/N ’/$’ argument/N flexibility/N argument/N red/ADJ herring/N total/ADJ herring/N span size 2wj, j = -2, -1 first/ORD place/N the/ART place/N less/ADJ genes/N ‘/$’ genes/N behavioural/ADJ flexibility/N Adj-N:poswj = ADJ, poswi = N

  22. Adj-N (S (PP In/PRP (NP the/ART first/ORD place/N ) ) ,/$, (NP the/ART ‘/$’ less/ADJ genes/N ,/$, more/ADJ behavioural/ADJ flexibility/N ’/$’ argument/N ) (VP is/V (NP a/ART total/ADJ red/ADJ herring/N ) ) ) ./$.

  23. Adj-N (S (PP-mod In/PRP (NP the/ART first/ORD place/N ) ) ,/$, (NP-subj the/ART ‘/$’ less/ADJ genes/N ,/$, more/ADJ behavioural/ADJ flexibility/N ’/$’ argument/N ) (VP-copula is/V (NP a/ART total/ADJ red/ADJ herring/N ) ) ) ./$.

  24. Adj-N:NP chunks NP chunks • (NP the/ART first/ORD place/N ) • (NP the/ART ‘/$’ less/ADJ genes/N ,/$, more/ADJ behavioural/ADJ flexibility/N ’/$’ argument/N) • (NP a/ART total/ADJ red/ADJ herring/N ) Adj-N Pairs • less/ADJ genes/N • more/ADJ flexibility/N • behavioural/ADJ flexibility/N • more/ADJ argument/N • behavioural/ADJ argument/N • total/ADJ herring/N • red/ADJ herring/N

  25. N-V: Object-VERB • spill the beans • Good for you for guessing the puzzle but from the beans Mike spilled to me, I think those kind of twists are more maddening than fun. • bury the hatchet • Paul McCartney has buried the hatchet with Yoko Ono after a dispute over the songwriting credits of some of the best-known Beatles songs.

  26. N-V: Object-Mod-VERB • keep <one‘s> nose to the grindstone • I'm very impressed with you for having keptyour nose to the grindstone, I'd like to offer you a managerial position. • We’ve learned from experience and keptour nose to the grindstone to make sure our future remains a bright one. • She keepsher nose to the grindstone.

  27. N-V: Object-Mod-VERB • keep <one‘s> nose to the grindstone (VP {kept, keeps, ...} {(NP-obj your nose), (NP-obj our nose), (NP-obj her nose), ... } (PP-mod to the grindstone) )

  28. PN-V: P-Object-VERB • zur Verfügung stellen (make available)Peter stellt sein Auto Maria zur Verfügung (Peter makes his car available to Maria) • in Frage stellen (question) Peter stellt Marias Loyalität in Frage (Peter questions Maria’s loyalty) • in Verbindung setzen (to contact) Peter setzt sich mit Maria in Verbindung (Peter contacts Maria)

  29. Contingency Tablesfor Relational Cooccurrences (big, dog) (black, box) (black, dog) (small, cat) (small, box) (black, box) (old, box) (tabby, cat) pair type: (u,v) = (black, box)

  30. Contingency Tablesfor Relational Cooccurrences (big, dog) (black, box) (black, dog) (small, cat) (small, box) (black, box) (old, box) (tabby, cat) pair type: (u,v) = (black, box)

  31. Contingency Tablesfor Relational Cooccurrences (big, dog) (black, box) (black, dog) (small, cat) (small, box) (black, box) (old, box) (tabby, cat) pair type: (u,v) = (black, box)

  32. Contingency Tablesfor Relational Cooccurrences (big, dog) (black, box) (black, dog) (small, cat) (small, box) (black, box) (old, box) (tabby, cat) pair type: (u,v) = (black, box)

  33. Contingency Tablesfor Relational Cooccurrences (big, dog) (black, box) (black, dog) (small, cat) (small, box) (black, box) (old, box) (tabby, cat) pair type: (u,v) = (black, box)

  34. Contingency Tablesfor Relational Cooccurrences (big, dog) (black, box) (black, dog) (small, cat) (small, box) (black, box) (old, box) (tabby, cat) pair type: (u,v) = (black, box)

  35. Contingency Tablesfor Relational Cooccurrences (big, dog) (black, box) (black, dog) (small, cat) (small, box) (black, box) (old, box) (tabby, cat) f(u,v) = 2 f1(u) = 3 f2(v) = 4 N = 8

  36. Contingency Tablesfor Relational Cooccurrences f(u,v) = 2 f1(u) = 3 f2(v) = 4 N = 8

  37. Contingency Tablesfor Relational Cooccurrences f(u,v) = 2 f1(u) = 3 f2(v) = 4 N = 8

  38. Contingency Tablesfor Relational Cooccurrences real data from the BNC(adjacent adj-noun pairs, lemmatised)

  39. Contingency Tables in Perl %F = (); %F1 = (); %F2 = (); $N = 0; while (($u, $v)=get_pair()) { $F{"$u,$v"}++; $F1{$u}++; $F2{$v}++; $N++; }

  40. Contingency Tables in Perl foreach $pair (keys %F) { ($u,$v) = split /,/, $pair; $f = $F{$pair}; $f1 = $F1{$u}; $f2 = $F2{$v}; $O11 = $f; $O12 = $f1 - $f; $O21 = $f2 - $f; $O22 = $N - $f1 - $f2 - $f; # ... }

  41. Reminder: Contingency Tablewith Row and Column Sums

  42. Why are Positional Cooccurrences Different? • adjectives and nous cooccurring within sentences • "I saw a black dog" (black, dog)f(black, dog)=1, f1(black)=1, f2(dog)=1 • "The old man with the silly brown hat saw a black dog" (old, dog), (silly, dog), (brown, dog), (black, dog), ... , (black, man), (black, hat)f(black, dog)=1, f1(black)=3, f2(dog)=4

  43. Why are PositionalCooccurrences Different? • "wrong" combinations could be considered as extraction noise( association measures distinguish noise from recurrent combinations) • but: very large amount of noise • statistical models assume that noise is completely random • but: marginal frequencies often increase in large steps

  44. Contingency Tables for Segment-Based Cooccurrences • within pre-determined segments (e.g. sentences) • components of cooccurring pairs may be syntactically restricted(e.g. adj-noun, nounSg-verb3.Sg) • for given pair type (u,v),set of all sentences is classified into four categories

  45. Contingency Tables for Segment-Based Cooccurrences • u  S = at least one occurrence of u in sentence S • u  S = no occurrences of u in sentence S • v  S = at least one occurrence of v in sentence S • v  S = no occurrences of v in sentence S

  46. Contingency Tables for Segment-Based Cooccurrences • fS(u,v) = number of sentences containing both u and v • fS(u) = number of sentences containing u • fS(v) = number of sentences containing v • NS= total number of sentences

  47. Frequency Counts for Segment-Based Cooccurrences • adjectives and nous cooccurring within sentences • "I saw a black dog" (black, dog)fS(black, dog)=1, fS(black)=1, fS(dog)=1 • "The old man with the silly brown hat saw a black dog" (old, dog), (silly, dog), (brown, dog), (black, dog), ... , (black, man), (black, hat)fS(black, dog)=1, fS(black)=1, fS(dog)=1

  48. Segment-Based Cooccurrences in Perl foreach $S (@sentences) { %words = map {$_ => 1} words($S); %pairs = map {$_ => 1} pairs($S); foreach $w (keys %words) { $FS_w{$w}++; } foreach $p (keys %pairs) { $FS_p{$p}++; } $NS++; }

  49. Contingency Tables for Distance-Based Cooccurrences • problems are similar to segment-based cooccurrence data • but: no pre-defined segments • accurate counting is difficult • here: sketched for special case • all orthographic words • numerical span: nL left, nR right • no stop word lists

  50. Contingency Tables for Distance-Based Cooccurrences • nL = 3, nR = 2

More Related