Semantics of Paragraphs

1. Semantics of Paragraphs Wlodek Zadrozny, Karen Jensen Computational Linguistics 17 (2) pp171-209 (1991) (as (mis-)interpreted by Peter Clark)

2. Why study paragraphs? • Paragraph: “unit of thought” • Can study: • Coherence • Anaphora resolution • Discourse structure, (e.g. use of connectives like “but”, “however”, “therefore”, …) • Proposes three-level logical theory • Object level: a model of the paragraph itself • Reference level: background knowledge • E.g., from dictionaries and encyclopedias • Metalevel: constraints on the object level representation

3. Example paragraph “In the summer of 1347 a merchant ship returning from the Black Sea entered the Sicilian port of Messina bringing with it the horrifying disease that came to be known as the Black Death. It struck rapidly. Within twenty-four hours of infection and the appearance of the first small black pustule came an agonizing death. The effect of the Black Death was appalling. In less than twenty years half the population of Europe had been killed, the countryside devastated, and a period of optimism and growing economic welfare had been brought to a sudden and catastrophic end.”

4. But few grammatical devices for paragraph cohesion! - no connectives - only two anaphoric pronouns (“it”) “In the summer of 1347 a merchant ship returning from the Black Sea entered the Sicilian port of Messina bringing with it the horrifying disease that came to be known as the Black Death. It struck rapidly. Within twenty-four hours of infection and the appearance of the first small black pustule came an agonizing death. The effect of the Black Death was appalling. In less than twenty years half the population of Europe had been killed, the countryside devastated, and a period of optimism and growing economic welfare had been brought to a sudden and catastrophic end.”

5. Against knowledge-poor methods… “In the summer of 1347 a merchant ship returning from the Black Sea entered the Sicilian port of Messina bringing with it….” • “it” = the ship or the port? • Surface syntax & selectional restrictions don’t suffice – need background knowledge • Dictionary can supply this • Also: want to establish shared-word relationships between “disease”, “Black Death”, “infection”, “death”, “killed”, “end”. • Dictionary can supply these connections too

6. For example… “A ship entered the port of Messina bringing with it….” = ? Longman’s Dictionary: Bring: To convey, lead, carry or cause to come along… Ship: A large boat for carrying people or goods… Port: 1. Harbour… 2. An opening in the side of a ship… - There’s a “short connection” between “bring” and “ship” (PS reminiscent of Thesaurus-based search) - Suggests “it” = “ship” is most coherent interpretation

7. The Structure of Background Knowledge • A set of logical “theories” • One for each sense of each word • Theories ordered according to “preference” • Theories for different word senses can’t exist together (i.e., are inconsistent) • These can be obtained automatically from dictionary definitions. sh1: “Ship: A large boat for carrying people or goods at sea.” ship(x)  large-boat(x); y carry(x,y) & (people(y)  goods(y))

8. Background Knowledge: Example e1: “Enter: To come into a place.” enter(x,y)  come-in(x,y); place(y); … e2: “Enter: To join a group, typically of professionals.” enter(x,y)  join(x,y) & group(y); professionals(y); … sh1: “Ship: A large boat for carrying people or goods at sea.” ship(x)  large-boat(x); y carry(x,y) & (people(y)  goods(y)) sh2: “Ship: A large aircraft or space vehicle.” ship(x)  large-aircraft(x)  space-vehicle(x) NB use of “ambiguous predicates” e.g. ship(x)! Partial ordering of interpretation preference: e1 >enter e2 > unknown sh1 >ship sh2 > unknown

9. How to Use Background Knowledge 1. Sentence  logical form (“object level theory”) “Entering the port, a ship brought a disease”  enter(x1,x2) & ship(x1) & port(x2) & bring(x3,x4) & disease(x4) “ambiguous predicates”

10. How to Use Background Knowledge 2. For each predicate (was word), add in one background theory (sense) for that word  one interpretation of the sentence An example of an interpretation: enter(x1,x2) & ship(x1) & port(x2) & bring(x3,x4) & disease(x4) ;;; e1: “enter: come into a place.” enter(x,y)  come-in(x,y); place(y); … ;;; sh1: “Ship: A large boat for carrying people or goods at sea.” ship(x)  large-boat(x); y carry(x,y) & (people(y)  goods(y))

11. How to Use Background Knowledge 3. Search the space of interpretations to find the “best”: - discount interpretations which are inconsistent e.g. x = boat & x = aircraft - dominance: prefer the more plausible theories - coherence: prefer interpretations with “coherence” - coreference: maximize coreference links - metalevel constraints: should be satisfied

12. 1. Dominance “Entering the port, a ship brought a disease”  enter(x1,x2) & ship(x1) & port(x2) & bring(x3,x4) & disease(x4) e1 sh1 b1 d1 p1 a a e2 sh2 b2 “Ship: large boat” a a a “Ship: aircraft or space vehicle” “Enter: to come in” “Enter: to join”

13. 1. Dominance “Entering the port, a ship brought a disease”  enter(x1,x2) & ship(x1) & port(x2) & bring(x3,x4) & disease(x4) Interpretation 1: e1 sh1 b1 d1 p1 a a e2 sh2 b2 “Ship: large boat” a a a “Ship: aircraft or space vehicle” “Enter: to come in” “Enter: to join”

14. 1. Dominance “Entering the port, a ship brought a disease”  enter(x1,x2) & ship(x1) & port(x2) & bring(x3,x4) & disease(x4) Interpretation 2: e1 sh1 b1 d1 p1 a a e2 sh2 b2 “Ship: large boat” a a a “Ship: aircraft or space vehicle” “Enter: to come in” “Enter: to join”

15. 1. Dominance “Entering the port, a ship brought a disease”  enter(x1,x2) & ship(x1) & port(x2) & bring(x3,x4) & disease(x4) All interpretations = all (consistent) paths e1 sh1 b1 d1 p1 a a e2 sh2 b2 “Ship: large boat” a a a “Ship: aircraft or space vehicle” “Enter: to come in” “Enter: to join”

16. Or to put it another way... “Entering the port, a ship brought a disease” Possible interpretations are all permutations of: {Coming into/Joining} the {harbor}, a {ship/large aircraft} {carried/caused} a {illness}.

17. 2. Coherence • Look for “coherence links” (shared predicates) between (but not within) theories • More c-links  more coherent • Consider: Sentence: “Entering the port, a ship brought a disaster.” b2: “Bring: to cause” C-link! d1: “Disaster: causesa harm” (compared with: b1: “Bring: To carry”)

18. 2. Coherence (cont) • word senses which “connect together” are preferred • the coherence of words in a sentence is based on the coherence of their meaning Sentence: “Entering the port, a ship brought a disaster.” b2: “Bring: to cause” d1: “Disaster: causesa harm”

19. 2. Coherence (cont) • Do we iterate, to look at definitions of terms in definitions etc.? • Zadrozny and Jensen: No, psychologically implausible.

20. Paragraphs! The Topic of a Paragraph ;;; “Entering the port, a ship brought a disease” enter(x1,x2) & ship(x1) & port(x2) & bring(x3,x4) & disease(x4) • A topic is one or more predicates, e.g. ship(), port(),.. • That for each sentences in the paragraph, the sentence • mentions the topic • or, uses a word whose definition mentions the topic • or, where the topic definition mentions the word • or, refers back to a sentence which mentioned the topic • Qn: Can one odd sentence “destroy” a topic?

21. For example... Topic: the Black Death disease, i.e., disease(), as “In the summer of 1347 a merchant ship returning from the Black Sea entered the Sicilian port of Messina bringing with it the horrifying disease that came to be known as the Black Death. It struck rapidly. Within twenty-four hours of infection and the appearance of the first small black pustule came an agonizing death…. Disease: illness causing infection cf. incoherent: “John took a train from Paris to Istanbul. He likes spinach.”

22. 3. Maximize coreference/anaphora resolution • Maximize the number of equalities that can be “plausibly inferred”… = ?? • For example: • “bringing with it the disease” disease(d) • “Disease: An illness caused by an infection.” disease(d)  infection(i) • “Within twenty four hours of infection…” infection(i) • Zadrozny & Jensen: This equality i=i’ cannot be proven, but may be reasonably assumed as disease  illness -causes infection. • Really means “connect using some background knowledge” • NB: Here equating constants, not relating predicates • Too open-ended? Restrict to just synonyms

23. Putting it all together:The (best) p-model for the paragraph = coreferences = coherence links (c-links) “Enter: something comes into a place” “Port: A harbor” “A ship entered the port bringing with it the disease known as the Black Death” “Bring: To carry” “Disease: An illness caused by an infection” “It struck rapidly” “Strike (example): They were struck by an illness”

24. 4. Other Metalevel Contraints Again help decide which possible theory to prefer • Avoid redundancy (penalize theories with redundant rules in) • “the captain is worried because the officer can open his safe” • Redundant if “his” = “the officer”, so penalize this interpretation • Gricean Cooperative Principles: • Quantity (don’t say too much or too little) • Quality (be true) • Relation (be relevant) • Manner (be orderly) • Connectives: “but” implies exceptions will follow • “the yacht is cheap, but it is elegant” • Here prefer a definition/implication which is violated by the second part of the sentence