Introduction to Computational Linguistics

1. Introduction to Computational Linguistics Martha Palmer April 19, 2006 NLP

2. Natural Language Processing • Machine Translation • Predicate argument structures • Syntactic parses • Producing semantic representations • Ambiguities in sentence interpretation NLP

3. Machine Translation • One of the first applications for computers • bilingual dictionary > word-word translation • Good translation requires understanding! • War and Peace, The Sound and The Fury? • What can we do? Sublanguages. • technical domains, static vocabulary • Meteo in Canada, Caterpillar Tractor Manuals, Botanical descriptions, Military Messages NLP

4. Example translation NLP

5. Translation Issues: Korean to English - Word order - Dropped arguments - Lexical ambiguities - Structure vs morphology NLP

6. Common Thread • Predicate-argument structure • Basic constituents of the sentence and how they are related to each other • Constituents • John, Mary, the dog, pleasure, the store. • Relations • Loves, feeds, go, to, bring NLP

7. Abstracting away from surface structure NLP

8. Transfer lexicons NLP

9. Machine Translation Lexical Choice- Word Sense Disambiguation • Iraq lost the battle. • Ilakuka centwey ciessta. • [Iraq ] [battle] [lost]. • John lost his computer. • John-i computer-lul ilepelyessta. • [John] [computer] [misplaced]. NLP

10. Natural Language Processing • Syntax • Grammars, parsers, parse trees, dependency structures • Semantics • Subcategorization frames, semantic classes, ontologies, formal semantics • Pragmatics • Pronouns, reference resolution, discourse models NLP

11. Syntactic Categories • Nouns, pronouns, Proper nouns • Verbs, intransitive verbs, transitive verbs, ditransitive verbs (subcategorization frames) • Modifiers, Adjectives, Adverbs • Prepositions • Conjunctions NLP

12. Syntactic Parsing • The cat sat on the mat. Det Noun Verb Prep Det Noun • Time flies like an arrow. Noun Verb Prep Det Noun • Fruit flies like a banana. Noun Noun Verb Det Noun NLP

13. Context Free Grammar • S -> NP VP • NP -> det (adj) N • NP -> Proper N • NP -> N • VP -> V, VP -> V PP • VP -> V NP • VP -> V NP PP, PP -> Prep NP • VP -> V NP NP NLP

14. Parses The cat sat on the mat S NP VP Det PP N V the cat sat NP Prep N on Det mat the NLP

15. Parses Time flies like an arrow. S NP VP N time V PP flies Prep NP like Det N arrow an NLP

16. Parses Time flies like an arrow. S NP VP N time V NP N like flies N Det arrow an NLP

17. Features • C for Case, Subjective/Objective • She visited her. • P for Person agreement, (1st, 2nd, 3rd) • I like him, You like him, He likes him, • N for Number agreement, Subject/Verb • He likes him, They like him. • G for Gender agreement, Subject/Verb • English, reflexive pronouns He washed himself. • Romance languages, det/noun • T for Tense, • auxiliaries, sentential complements, etc. • * will finished is bad NLP

18. Probabilistic Context Free Grammars • Adding probabilities • Lexicalizing the probabilities NLP

19. Simple Context Free Grammar in BNF S → NP VP NP → Pronoun | Noun | Det Adj Noun |NP PP PP → Prep NP V → Verb | Aux Verb VP → V | V NP | V NP NP | V NP PP | VP PP NLP

20. Simple Probabilistic CFG S → NP VP NP → Pronoun [0.10] | Noun [0.20] | Det Adj Noun [0.50] |NP PP [0.20] PP → Prep NP [1.00] V → Verb [0.33] | Aux Verb [0.67] VP → V [0.10] | V NP [0.40] | V NP NP [0.10] | V NP PP [0.20] | VP PP [0.20] NLP

21. Simple Probabilistic Lexicalized CFG S → NP VP NP → Pronoun [0.10] | Noun [0.20] | Det Adj Noun [0.50] |NP PP [0.20] PP → Prep NP [1.00] V → Verb [0.33] | Aux Verb [0.67] VP → V [0.87] {sleep, cry, laugh} | V NP [0.03] | V NP NP [0.00] | V NP PP [0.00] | VP PP [0.10] NLP

22. Simple Probabilistic Lexicalized CFG VP → V [0.30] | V NP [0.60] {break,split,crack..} | V NP NP [0.00] | V NP PP [0.00] | VP PP [0.10] VP → V [0.10] what about | V NP [0.40] leave? | V NP NP [0.10] leave1, leave2? | V NP PP [0.20] | VP PP [0.20] NLP

23. Language to Logic • John went to the book store. Johnstore1, go(John, store1) • John bought a book. buy(John,book1) • John gave the book to Mary. give(John,book1,Mary) • Mary put the book on the table. put(Mary,book1,table1) NLP

24. SemanticsSame event - different sentences • John broke the window with a hammer. • John broke the window with the crack. • The hammer broke the window. • The window broke. NLP

25. Same event - different syntactic frames • John broke the window with a hammer. • SUBJ VERB OBJ MODIFIER • John broke the window with the crack. • SUBJ VERB OBJ MODIFIER • The hammer broke the window. • SUBJ VERB OBJ • The window broke. • SUBJ VERB NLP

26. Semantics -predicate arguments • break(AGENT, INSTRUMENT, PATIENT) • AGENT PATIENT INSTRUMENT • John broke the window with a hammer. • INSTRUMENT PATIENT • The hammer broke the window. • PATIENT • The window broke. • Fillmore 68 - The case for case NLP

27. AGENT PATIENT INSTRUMENT • John broke the window with a hammer. • SUBJ OBJ MODIFIER • INSTRUMENT PATIENT • The hammer broke the window. • SUBJ OBJ • PATIENT • The window broke. • SUBJ NLP

28. Canonical Representation • break (Agent: animate, • Instrument: tool, • Patient: physical-object) • Agent <=> subj • Instrument <=> subj, with-pp • Patient <=> obj, subj NLP

29. Syntax/semantics interaction • Parsers will produce syntactically valid parses for semantically anomalous sentences • Lexical semantics can be used to rule them out NLP

30. Headlines • Police Begin Campaign To Run Down Jaywalkers • Iraqi Head Seeks Arms • Teacher Strikes Idle Kids • Miners Refuse To Work After Death • Juvenile Court To Try Shooting Defendant NLP