1 / 45

NLP 1 An Introduction to Pragmatics in NLP

NLP 1 An Introduction to Pragmatics in NLP. GSLT, Göteborg, March 2006. Barbara Gawronska, Högskolan i Skövde. Reading list:. Jurafsky & Martin, part IV

ivanbritt
Download Presentation

NLP 1 An Introduction to Pragmatics in NLP

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. NLP 1An Introduction to Pragmatics in NLP GSLT, Göteborg, March 2006 Barbara Gawronska, Högskolan i Skövde

  2. Reading list: • Jurafsky & Martin, part IV • Mitkov, R. 2000. "Towards a more consistent and comprehensive evaluation of anaphora resolution algorithms and systems." Proceedings of the Discourse Anaphora and Anaphora Resolution Colloquium (DAARC-2000), 96-107, Lancaster, UK (pdf) http://clg.wlv.ac.uk/papers/Lancaster2000.PDF • Mitkov, R. and Barbu, C. 2002. "Using corpora to improve pronoun resolution." Languages in context, 4(1). (pdf ) http://clg.wlv.ac.uk/papers/mitkov02.pdf • Hutchins, J. 2003. "Has Machine Translation improved?" An expanded version [PDF, 288KB] of a paper presented at MT Summit IX: Proceedings of the Ninth Machine Translation Summit, New Orleans, USA, September 23-27, 2003, 181-188. [East Stroudsburg, PA: AMTA.] [PDF, 191KB] http://ourworld.compuserve.com/homepages/WJHutchins/HasMTimproved-exp.pdf

  3. Outline: • The notion ”Pragmatics” • Pragmatics vs. Semantics • Pragmatics and NLP: Discourse Processing • Anaphora resolution • NL Generation • Information Extraction and Text Summarization • Machine Translation • CALL • Future directions

  4. Pragmatics vs. Semantics (1) • Austin 1962 – Pragmatics = the study of "how to do things with words“ • Leech & Weisser 2003 – Pragmatics = “the branch of linguistics which seeks to explain the meaning of linguistics messages in terms of their context of use” , while Semantics investigates meaning ”as part of the language system irrespective of wider context”

  5. Pragmatics vs. Semantics (2) • Classical work on pragmatics (Austin 1962, Searle 1969, Grice 1975); problems as: • Discourse referents; what entities does a given message refer to? • What background knowledge is needed to understand a given message? • How do the beliefs of speaker and hearer interact in the interpretation of a message? • What is a relevant answer to a given question?

  6. Pragmatics vs. Semantics (3) This implies that the study object of semantics and pragmatics comprisesinteractions between entities on different levels of the linguistic structure. Pragmatics takes even interactions between the linguistic and the non-linguistic reality into account. E.g. identification of discourse referents (entities referred to) in spoken language requires an interplay between phonetic/phonological, morphological, syntactic, and semantic factors as well as the use of extralinguistic knowledge.

  7. Problems with reference in spoken language processing – an example (from “August”, KTH) User and system have different background knowledge: • User: Finns det en bra restaurang i närheten? (Is there a good restaurant nearby?) • System: Du måste ange gatan (You have to name the street) The system gives an answer that is true, but not relevant: • User: Var är vi? (Where are we?) • System: Vi är ju här. (We are here.)

  8. Pragmatics in NLP Discourse processing for • Dialogue systems • Natural Language Generation • Reading Comprehension (e.g. in Q/A systems, in summarization systems) • Machine Translation • Multifunctional NLP systems • Computer Assisted Language Learning (CALL)

  9. Discourse processing (1) • Discourse level – beyond the sentence level • Traditional distinctions: • Spoken/written discourse • Monologue/dialogue • New discourse types related to new ways of communicating: SMS, chatting, e-mail...

  10. Discourse processing (2) • The main aspects: • Anaphora resolution • Cohesion and coherence • Discourse structure

  11. Anaphora resolution (1) • Theoretical work: Karttunen 1976, Kamp 1979, 1981, Grosz and Sidner 1986, Hobbs 1978, 1982, Dagan & Itai 1990, Lappin & Leass 1944, Mitkov and Barbu 2000, 2002...) • Basic notions: • Anaphora • Antecedent • Discourse referent • Coreference chain

  12. Anaphora resolution (2) • Sources of knowledge: • Syntactic and morphosyntactic constraints (boundedness, gender, number, grammatical roles): Mary met John. He/She/They decided... She helped her/herself • Semantic features, selectional restrictions I bought a bottle of wine, sat down on a stone, and drankit

  13. Anaphora resolution (3) • Ontological knowledge, domain knowledge in interaction with semantic and grammatical constraints My friends have a greyhound. They are really huge beasts They prohibited them from demonstrating because they feared violence They prohibited them from demonstrating because they advocated violence (Winograd 1972:33)

  14. Algorithms for anaphora resolution (1) • Based on parse trees (naïve): Left-to right, breadth-first search, starting with the sentence containing the pronoun • Based on syntactic roles: The centering algorithm (Grosz et al 1995, Lappin and Leass 1994) • Based on lexical and collocational indicators: Mitkov’s knowledge poor approach (Mitkov 1998) • Based on so-kalled ”pragmatic functions”: the Mental Space model (Fauconnier 1985,1998)

  15. Algorithms for anaphora resolution (2): The centering algorithm Backward lookning center (CB) - the entity currently in focus Forward looking centers (CF) - an ordered list of entities: Subject > Predicative NP > direct object >oblique > PP ”Preferred center” (CP) - the highest ranked forward looking element A ranked set of transitions: • Continue: CB = CP = CB of the previous utterance • Retain: CB\= CP; CB = CB of the previous utterance • Smooth –shift: CB = CP; CB \= CB of the previous utterance • Rough-shift: CB \= CP; CB \= CB of the previous utterance

  16. Algorithms for anaphora resolution (3): The knowledge-poor approach (Mitkov 1998, 2000) • Input: a text processed by a POS-tagger and an NP extractor • Locate all NPs which precede the anaphor within a distance of 3 sentences • Check number and gender agreement, filter out NP:s that do not fulfil agreement conditions • Apply boosting and impeding indicators to the remaining NP:s

  17. Algorithms for anaphora resolution (4): The knowledge-poor approach (Mitkov 1998) • Boosting indicators (some examples): • First NP in a sentence • Lexical Iteration (NP:s repeated twice or more in the papagraph before the pronoun) • Section Heading Preference • Collocation Pattern Preference (Press the key down and turn the volume up. Press it again…)* • Term preference (terms characteristic for the genre * For an interesting approach to collocations (collostructional analysis), see Gries, Hampe, & Schönefeld 2005

  18. Algorithms for anaphora resolution (4): The knowledge-poor approach (Mitkov 1998) • Impeding indicators (some examples): • Indefiniteness • Complement of a preposition • Referential distance Evaluation: Success rate: Number of sucessfully resolved anaphors/Number of all anaphors (Different variants paying atention to trivial and non-trivial anaphors)

  19. The Theory of Mental Spaces (Fauconnier1985, Fauconnier and Sweetser 1996 – focus on beliefs and attitudes)

  20. The Theory of Mental Spaces (2) (Fauconnier 1985, Sweetser & Fauconnier 1996, Sanders & Redeker 1996)

  21. Natural Language Generation (1) • Discourse planning • Templates – partially pre-defined text frames • Algorithms based on discourse theories (e.g. Rhetorical Strucure Theory (RST) – Mann & Thompson • Sentence planning (sentence aggregation, generation of referring expressions, lexical selection) • Surface realization (word order and agreement control, graphic realization)

  22. Natural Language Generation (2) • Main issues: cohesion and coherence • Cohesion: establishing anaphoric connections (the ”reverse” of anaphora resolution) • Coherence: ”nucleus-satellite” relations (RST) e.g. result, cause, elaboration, contrast, parallel...

  23. Natural Language Generation (3) • Some examples of coherence relations: John bought a dog. His wife went furious (result) John hid Bill’s car keys. Bill had drunk too much (explanation) John bought a Mercedes. Bill leased a BMW (parallel) • An insufficiently cohesive/coherent text: I saw a little dog. Dogs like bones. Bones are white. White is my favourite colour...

  24. Sentence aggregation – an example (visit http://www.iccs.informatics.ed.ac.uk/~jbos/anna/ for more demos) • TEXT WITHOUT AGGREGATION:IcelandIceland is situated in the north Atlantic. Iceland has a coastline of 5 000 km. Iceland has an area of 103 000 sq km. The highest point is 2 119 m. The highest point is Hvannadalshnukur. Iceland has a temperate climate. Iceland has mild, windy winters and cool, damp summers. Iceland has 280 000 inhabitants. The population density is 3 people/sq km. The life expectancy is 79 years. The fertility rate is 2 children. The official language is Icelandic. Icelandic is a germanic language. Iceland is a constitutional republic. The capital is Reykjavik. Reykjavik has 107 000 inhabitants. The national holiday is June 17.

  25. TEXT WITH AGGREGATION:IcelandIceland is situated in the north Atlantic. Iceland has a coastline of 5 000 km and an area of 103 000 sq km. The highest point, Hvannadalshnukur, is 2 119 m. Iceland has a temperate climate with mild, windy winters and cool, damp summers. Iceland has 280 000 inhabitants and the population density is 3 people/sq km. The life expectancy is 79 years and the fertility rate is 2 children. The official language is Icelandic, a germanic language. Iceland is a constitutional republic. The capital, Reykjavik, has 107 000 inhabitants.The national holiday is June 17.

  26. Document and Text Retrieval vs. Information Extraction

  27. Text Summarization: Types of summaries • With respect to content: • Indicative: provide an idea what the text is about • Informative: shortened versions of the text • With respect to the way of creating: • Extracts: reused portions of the text (text retrieval) • Abstracts: re-generated text reflecting the important content (information extraction and text generation) • Compressed texts: (Knight & Marcu 2000): compressing parse trees in order to get a shorter text • Dialogue summarization: selecting successful ’dialog transactions’ • Non-verbal ”summaries” (e.g. graphical representations)

  28. Abstract creation: • Template-based summarization (templates, “sketchy frames”, “extraction patterns”: frames containing slots with constraints and variables; relay on prior domain knowledge); some examples: • DeJong 1982: FRUMP (Fast Reading Understanding and Memory Program) • Rilloff 1996: CIRCUS (terrorism domain) • McKeown and Radev 1999: SUMMONS (SUMMarizing Online NewS articles) • “Plot units” (selecting causal relations; Lehnert 1981)

  29. Template based summarization- general architecture

  30. The outline of the summarization process in Newspeak (Gawronska et al. 2004)

  31. Some principles for selection of claims to be rendered: • 1) Informatives: • Neutral, the sender is not marked for high status: officials said, the news agency reported, reportedly…A claim p introduced by a neutral informative is rendered in the summary; the source is omitted if there are no denials or confirmations of p in the text and if the source is not marked for high status, like ‘President’ • Neutral, the sender marked for high status, and ‘declarations’: the President said…the government condemned…The source is rendered if it is marked for high status • Affirmative; confirmations of explicit claims: Israeli sources confirmed that…Confirmations of previous explicit claims are omitted in the summary • Affirmative; confirmations of claims that are not explicitly mentioned:Both the information source and the claim, including the type of the speech act phrase, are rendered in the summary, if the speech act is a confirmation of a claim not present in the news report

  32. Some principles for selection of claims to be rendered: • 1) Informatives: • Negative, or neutral followed by denied claims:The president denied, The Israeli source said that it is not true…Both the initial claim and its denial are rendered in the summary together with the information about the senders

  33. Some principles for selection of claims to be rendered (2): 2) Utterance refusal, negated speech act phrases, hypotheses, commissives, interpretations: The Israeli sources neither denied or confirmed, the minister did not say, if…, the defense secretary declined to say…, the government had no immediate comments… Utterance refusals or negated speech act phrases related to an explicit claim are omitted If a source refuses to confirm/deny a claim that has not been explicitly mentioned in the previous part of the text, the whole speech act is rendered, inclusive the type of the speech act Hypotheses and commissives are rendered together with their sources and marked for unsure epistemic status

  34. Some principles for selection of claims to be rendered (3): 3) Epistemic spaces: e. g. no one knows if the device was planted deliberately or if it was leftover from New Year’s Eve If two claims would exclude each other in the same mental space, and if no source in the text takes responsibility for any of these claims, both claims are to be rendered as hypotheses

  35. Some challenges… RAMALLAH, West Bank -- Palestinian leader Yasser Arafat said Thursday that elections as part of a reform of the Palestinian Authority will be held this winter, whether or not Israeli forces withdraw from the Palestinian territories. That represented a change of course from Arafat, who said last week that no elections would be held until the Israelis pulled back. Shortly after Arafat's announcement, a committee he had appointed to set up elections resigned, according to Israel Radio, because Arafat would not agree to a specific date for the elections. Other Palestinian leaders said the resignations were a procedural matter. Arafat also condemned Wednesday's suicide bombing in the Israeli town of Rishon Letzion . Two Israelis were killed and at least 37 others wounded when the bomber detonated explosives in the center of a crowded pedestrian district. The terror attack marked the second time in two weeks a suicide bombing directed at civilians has rocked Rishon Letzion, a town about 15 miles southeast of Tel Aviv. On May 8, a suicide attack at a pool hall killed 15 people and wounded dozens of others. "Suddenly there was an explosion," 16-year-old Shmuel Voller told The Associated Press on Wednesday. The bombing occurred on Rothschild Street in the heart of the town around 9:15 p.m. (2:15 p.m. ET).

  36. An example of non-verbal “summaries”: extracting pathway maps similar to those in Kyoto Encyclopaedia of Genes and Genomes (KEGG) from biomedical literature

  37. A sample comprimed tree expressing a biological process

  38. Machine Translation – combining NL Understanding and NL Generation (1) • 1940... – the first attempts: direkt word-to-word translation + some morphosyntactic processing (e.g. case recognition in Russian) • 1970...-syntax-based approaches: interlingua and transfer • 1990 – Brown et al. – foundation of stochastic MT (computing translation probabilities on the basis of parallel corpora)

  39. Machine Translation (2) • Knowledge Based Machine Translation – KBMT – Nirenburg et al., Hobbs, Wilks mm - knowledge stored in lexicons, onomasticons, and ontologies • rule-based parsing and semantico-pragmatic analysis aimed at conceptuel representations • Example Based MT – EBMT - translation in analogy with best match in the corpus of previously translated texts • Hybrid systems (e.g. Verbmobil – Wahlster et al 2000)

  40. The multi engine architecture of the MT system Verbmobil (a simplified version of Figure 11, p. 17 in [33])

  41. MT evaluation – some useful links • Hutchins, John (2000) The IAMT Certification initiative and defining translation system categories. (Presented at EAMT Workshop, Ljubljana, May 2000) http://ourworld.compuserve.com/homepages/WJHutchins/IAMTcert.htmhttp://ourworld.compuserve.com/homepages/WJHutchins/Compendium-4.pdf • http://www.issco.unige.ch/projects/isle/femti/

  42. MT, current trends: • Towards hybrid systems: integration of rule-based approaches and stochastic approaches • Spoken language translation • Sign language translation • Combined MT and Intormation Extraction • Computer aided translation

  43. Computer Assisted Language Learning (CALL): focus on communicative competence

  44. Conclusions; Future? Pragmatics – still a challenge for NLP Research needed on: • General vs. domain-specific resources and algorithms • User models (beliefs, attitudes, etc.) • The interplay between prosody, syntax, and semantics • New means of communication, new types of discourse • Synergy between rule-based and stochastic approaches

More Related