1 / 32

Word classes and part of speech tagging Chapter 5

Word classes and part of speech tagging Chapter 5. Outline. Why part of speech tagging? Word classes Tag sets and problem definition Automatic approaches 1: rule-based tagging Automatic approaches 2: stochastic tagging On Part 2: finish stochastic tagging, and continue on to:

mkenny
Download Presentation

Word classes and part of speech tagging Chapter 5

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Word classes and part of speech tagging Chapter 5

  2. Outline • Why part of speech tagging? • Word classes • Tag sets and problem definition • Automatic approaches 1: rule-based tagging • Automatic approaches 2: stochastic tagging • On Part 2: finish stochastic tagging, and continue on to: • Automatic approaches 3: transformation-based tagging • Other issues: tagging unknown words, evaluation

  3. WORDS TAGS the girl kissed the boy on the cheek N V P DET Definition “The process of assigning a part-of-speech or other lexical class marker to each word in a corpus” (Jurafsky and Martin)

  4. An Example WORD LEMMA TAG the girl kissed the boy on the cheek the girl kiss the boy on the cheek +DET +NOUN +VPAST +DET +NOUN +PREP +DET +NOUN

  5. Motivation • Speech synthesis — pronunciation • Speech recognition — class-based N-grams • Information retrieval — stemming, selection high-content words • Word-sense disambiguation • Corpus analysis of language & lexicography

  6. Outline • Why part of speech tagging? • Word classes • Tag sets and problem definition • Automatic approaches 1: rule-based tagging • Automatic approaches 2: stochastic tagging • Automatic approaches 3: transformation-based tagging • Other issues: tagging unknown words, evaluation

  7. Word Classes • Basic word classes: Noun, Verb, Adjective, Adverb, Preposition, … • Open vs. Closed classes • Open: • Nouns, Verbs, Adjectives, Adverbs. • Why “open”? • Closed: • determiners: a, an, the • pronouns: she, he, I • prepositions: on, under, over, near, by, …

  8. Open Class Words • Every known human language has nouns and verbs • Nouns: people, places, things • Classes of nouns • proper vs. common • count vs. mass • Verbs: actions and processes • Adjectives: properties, qualities • Adverbs: hodgepodge! • Unfortunately, John walked home extremely slowly yesterday • Numerals: one, two, three, third, …

  9. Closed Class Words • Differ more from language to language than open class words • Examples: • prepositions: on, under, over, … • particles: up, down, on, off, … • determiners: a, an, the, … • pronouns: she, who, I, .. • conjunctions: and, but, or, … • auxiliary verbs: can, may should, …

  10. Prepositions from CELEX

  11. English Single-Word Particles

  12. Pronouns in CELEX

  13. Conjunctions

  14. Auxiliaries

  15. Outline • Why part of speech tagging? • Word classes • Tag sets and problem definition • Automatic approaches 1: rule-based tagging • Automatic approaches 2: stochastic tagging • Automatic approaches 3: transformation-based tagging • Other issues: tagging unknown words, evaluation

  16. Word Classes: Tag Sets • Vary in number of tags: a dozen to over 200 • Size of tag sets depends on language, objectives and purpose • Some tagging approaches (e.g., constraint grammar based) make fewer distinctions e.g., conflating prepositions, conjunctions, particles • Simple morphology = more ambiguity = fewer tags

  17. Word Classes: Tag set example PRP PRP$

  18. Example of Penn Treebank Tagging of Brown Corpus Sentence • The/DT grand/JJ jury/NN commented/VBD on/IN a/DT number/NN of/IN other/JJ topics/NNS ./. • VB DT NN .Book that flight . • VBZ DT NN VB NN ?Does that flight serve dinner ? • See http://www.infogistics.com/posdemo.htm • Buffalo buffalo Buffalo buffalo buffalo buffalo Buffalo buffalo

  19. The Problem • Words often have more than one word class: this • This is a nice day = PRP • This day is nice = DT • You can go this far = RB

  20. Word Class Ambiguity(in the Brown Corpus) • Unambiguous (1 tag): 35,340 • Ambiguous (2-7 tags): 4,100 (Derose, 1988)

  21. Part-of-Speech Tagging • Rule-Based Tagger: ENGTWOL (ENGlish TWO Level analysis) • Stochastic Tagger: HMM-based • Transformation-Based Tagger (Brill)

  22. Outline • Why part of speech tagging? • Word classes • Tag sets and problem definition • Automatic approaches 1: rule-based tagging • Automatic approaches 2: stochastic tagging • Automatic approaches 3: transformation-based tagging • Other issues: tagging unknown words, evaluation

  23. Rule-Based Tagging • Basic Idea: • Assign all possible tags to words • Remove tags according to set of rules of type: if word+1 is an adj, adv, or quantifier and the following is a sentence boundary and word-1 is not a verb like “consider” then eliminate non-adv else eliminate adv. • Typically more than 1000 hand-written rules

  24. Sample ENGTWOL Lexicon • Demo: http://www2.lingsoft.fi/cgi-bin/engtwol

  25. Stage 1 of ENGTWOL Tagging • First Stage: Run words through a morphological analyzer to get all parts of speech. • Example: Pavlov had shown that salivation …Pavlov PAVLOV N NOM SG PROPERhad HAVE V PAST VFIN SVO HAVE PCP2 SVOshown SHOW PCP2 SVOO SVO SVthat ADV PRON DEM SG DET CENTRAL DEM SGCSsalivation N NOM SG

  26. Stage 2 of ENGTWOL Tagging • Second Stage: Apply constraints. • Constraints used in negative way. • Example: Adverbial “that” ruleGiven input: “that”If (+1 A/ADV/QUANT) (+2 SENT-LIM) (NOT -1 SVOC/A)Then eliminate non-ADV tagsElse eliminate ADV

  27. Outline • Why part of speech tagging? • Word classes • Tag sets and problem definition • Automatic approaches 1: rule-based tagging • Automatic approaches 2: stochastic tagging • Automatic approaches 3: transformation-based tagging • Other issues: tagging unknown words, evaluation

  28. Stochastic Tagging • Based on probability of certain tag occurring given various possibilities • Requires a training corpus • No probabilities for words not in corpus. • Training corpus may be different from test corpus.

  29. Stochastic Tagging (cont.) • Simple Method: Choose most frequent tag in training text for each word! • Result: 90% accuracy • Baseline • Others will do better • HMM is an example

  30. HMM Tagger • Intuition: Pick the most likely tag for this word. • HMM Taggers choose tag sequence that maximizes this formula: • P(word|tag) × P(tag|previous n tags) • Let T = t1,t2,…,tnLet W = w1,w2,…,wn • Find POS tags that generate a sequence of words, i.e., look for most probable sequence of tags T underlying the observed words W.

  31. Start with Bigram-HMM Tagger • argmaxT P(T|W) • argmaxTP(T)P(W|T) • argmaxtP(t1…tn)P(w1…wn|t1…tn) • argmaxt[P(t1)P(t2|t1)…P(tn|tn-1)][P(w1|t1)P(w2|t2)…P(wn|tn)] • To tag a single word: ti = argmaxj P(tj|ti-1)P(wi|tj) • How do we compute P(ti|ti-1)? • c(ti-1ti)/c(ti-1) • How do we compute P(wi|ti)? • c(wi,ti)/c(ti) • How do we compute the most probable tag sequence? • Viterbi

  32. An Example • Secretariat/NNP is/VBZ expected/VBN to/TO race/VB tomorrow/NN • People/NNS continue/VBP to/TO inquire/VB the DT reason/NN for/IN the/DT race/NN for/IN outer/JJ space/NN • to/TO race/???

More Related