An Extended Model of Natural Logic

1. An Extended Model of Natural Logic Bill MacCartney and Christopher D. Manning NLP Group Stanford University 8 January 2009

2. Some Some no Introduction • Semantic Relations • Joins • Lexical Relations • Projectivity • Implicatives • Inference • Evaluation • Conclusion Natural language inference (NLI) • Aka recognizing textual entailment (RTE) • Does premise P justify an inference to hypothesis H? • An informal, intuitive notion of inference: not strict logic • Emphasis on variability of linguistic expression P Every firm polled saw costs grow more than expected,even after adjusting for inflation. H Every big company in the poll reported cost increases. yes • Necessary to goal of natural language understanding (NLU) • Can also enable semantic search, question answering, …

3. robust,but shallow deep,but brittle lexical/semanticoverlap Jijkoun & de Rijke 2005 FOL &theoremproving Bos & Markert 2006 patternedrelationextraction Romano et al. 2006 semantic graph matching MacCartney et al. 2006 Hickl et al. 2006 Introduction • Semantic Relations • Joins • Lexical Relations • Projectivity • Implicatives • Inference • Evaluation • Conclusion naturallogic (this work) NLI: a spectrum of approaches Solution? Problem:hard to translate NL to FOL idioms, anaphora, ellipsis, intensionality, tense, aspect, vagueness, modals, indexicals, reciprocals, propositional attitudes, scope ambiguities, anaphoric adjectives, non-intersective adjectives, temporal & causal relations, unselective quantifiers, adverbs of quantification, donkey sentences, generic determiners, comparatives, phrasal verbs, … Problem:imprecise  easily confounded by negation, quantifiers, conditionals, factive & implicative verbs, etc.

4. Introduction • Semantic Relations • Joins • Lexical Relations • Projectivity • Implicatives • Inference • Evaluation • Conclusion What is natural logic? ( natural deduction) • Characterizes valid patterns of inference via surface forms • precise, yet sidesteps difficulties of translating to FOL • A long history • traditional logic: Aristotle’s syllogisms, scholastics, Leibniz, … • modern natural logic begins with Lakoff (1970) • van Benthem & Sánchez Valencia (1986-91): monotonicity calculus • Nairn et al. (2006): an account of implicatives & factives • We introduce a new theory of natural logic • extends monotonicity calculus to account for negation & exclusion • incorporates elements of Nairn et al.’s model of implicatives

5. Introduction • Semantic Relations • Joins • Lexical Relations • Projectivity • Implicatives • Inference • Evaluation • Conclusion 16 elementary set relations Assign sets x, y to one of 16 relations, depending on emptiness or non-emptiness of each of four partitions y y x x empty non-empty

6. x ^ y x‿y x⊐y xy Introduction • Semantic Relations • Joins • Lexical Relations • Projectivity • Implicatives • Inference • Evaluation • Conclusion x⊏y x | y x # y 16 elementary set relations But 9 of 16 are degenerate: either x or y is either empty or universal. I.e., they correspond to semantically vacuous expressions, which are rare outside logic textbooks. We therefore focus on the remaining seven relations.

7. Introduction • Semantic Relations • Joins • Lexical Relations • Projectivity • Implicatives • Inference • Evaluation • Conclusion The set of 7 basic semantic relations Relations are defined for all semantic types: tiny⊏small, hover⊏fly, kick⊏strike,this morning⊏today, in Beijing⊏in China, everyone⊏someone, all⊏most⊏some

8. ? R S ? fish human nonhuman Introduction • Semantic Relations • Joins • Lexical Relations • Projectivity • Implicatives • Inference • Evaluation • Conclusion Joining semantic relations y x y z ⊏ | ^

9. What is | | ? ⋈ Introduction • Semantic Relations • Joins • Lexical Relations • Projectivity • Implicatives • Inference • Evaluation • Conclusion | |  {, ⊏, ⊐, |, #} ⋈ Some joins yield unions of relations!

10. Introduction • Semantic Relations • Joins • Lexical Relations • Projectivity • Implicatives • Inference • Evaluation • Conclusion The complete join table Of 49 join pairs, 32 yield relations in ; 17 yield unions Larger unions convey less information — limits power of inference In practice, any union which contains # can be approximated by #

11. atomic edit: DEL, INS, SUB compound expression semantic relation Introduction • Semantic Relations • Joins • Lexical Relations • Projectivity • Implicatives • Inference • Evaluation • Conclusion Lexical semantic relations x e(x) •  will depend on: • the lexical semantic relation generated by e: (e) • other properties of the context x in which e is applied (, ) • Example: suppose x is red car • If e is SUB(car, convertible), then (e) is ⊐ • If e is DEL(red), then (e) is ⊏ • Crucially, (e) depends solely on lexical items in e, independent of context x • But how are lexical semantic relations determined?

12. Introduction • Semantic Relations • Joins • Lexical Relations • Projectivity • Implicatives • Inference • Evaluation • Conclusion Lexical semantic relations: SUBs (SUB(x, y)) = (x, y) For open-class terms, use lexical resource (e.g. WordNet)  for synonyms: sofa couch, forbid  prohibit ⊏ for hypo-/hypernyms: crow⊏bird, frigid⊏cold, soar⊏rise | for antonyms and coordinate terms: hot| cold, cat | dog  or | for proper nouns:USA United States, JFK | FDR # for most other pairs:hungry# hippo Closed-class terms may require special handling Quantifiers: all⊏some, some^no, no | all, at least 4‿at most 6 See paper for discussion of pronouns, prepositions, …

13. Introduction • Semantic Relations • Joins • Lexical Relations • Projectivity • Implicatives • Inference • Evaluation • Conclusion Lexical semantic relations: DELs & INSs Generic (default) case: (DEL(•)) = ⊏, (INS(•)) = ⊐ • Examples: red car⊏car, sing⊐sing off-key • Even quite long phrases: car parked outside since last week⊏car • Applies to intersective modifiers, conjuncts, independent clauses, … • This heuristic underlies most approaches to RTE! • Does P subsume H? Deletions OK; insertions penalized. Special cases • Negation: didn’t sleep ^ did sleep • Implicatives & factives (e.g. refuse to, admit that): discussed later • Non-intersective adjectives: former spy | spy, alleged spy # spy • Auxiliaries etc.: is sleepingsleeps, did sleepslept

14. How is (x, y) projected by f?   @ means fn application @ @ x y f f  ? Introduction • Semantic Relations • Joins • Lexical Relations • Projectivity • Implicatives • Inference • Evaluation • Conclusion The impact of semantic composition How are semantic relations affected by semantic composition? The monotonicity calculus provides a partial answer If f has monotonicity… But how are other relations (|, ^, ‿) projected?

15. Each projectivity signature is a map ↦ Introduction • Semantic Relations • Joins • Lexical Relations • Projectivity • Implicatives • Inference • Evaluation • Conclusion A typology of projectivity Projectivity signatures: a generalization of monotonicity classes In principle, 77 possible signatures, but few actually realized

16. ↦ Introduction • Semantic Relations • Joins • Lexical Relations • Projectivity • Implicatives • Inference • Evaluation • Conclusion A typology of projectivity Projectivity signatures: a generalization of monotonicity classes Each projectivity signature is a map In principle, 77 possible signatures, but few actually realized See paper for projectivity of various quantifiers, verbs

17. @ @ ⊐ ⊐ ⊐ @ @ ⊏ ⊏ @ @ Introduction • Semantic Relations • Joins • Lexical Relations • Projectivity • Implicatives • Inference • Evaluation • Conclusion @ @ nobody nobody can can without without clothes a shirt enter enter Projecting through multiple levels Propagate semantic relation between atoms upward, according to projectivity class of each node on path to root nobody can enter with a shirt⊏nobody can enter with clothes

18. Introduction • Semantic Relations • Joins • Lexical Relations • Projectivity • Implicatives • Inference • Evaluation • Conclusion Implicatives & factives [Nairn et al. 06] 9 signatures, per implications (+, –, or o) in positive and negative contexts

19. Introduction • Semantic Relations • Joins • Lexical Relations • Projectivity • Implicatives • Inference • Evaluation • Conclusion Implicatives & factives We can specify relation generated by DEL or INS of each signature Room for variation w.r.t. infinitives, complementizers, passivation, etc. Factives not fully explained: he didn’t admit that he knew | he didn’t know Some more intuitive when negated: he didn’t hesitate to ask | he didn’t ask

20. Introduction • Semantic Relations • Joins • Lexical Relations • Projectivity • Implicatives • Inference • Evaluation • Conclusion Putting it all together • Find a sequence of edits e1, …, en which transforms p into h. Define x0 = p, xn = h, and xi = ei(xi–1) for i [1, n]. • For each atomic edit ei: • Determine the lexical semantic relation (ei). • Project (ei) upward through the semantic composition tree of expression xi–1 to find the atomic semantic relation (xi–1, xi) • Join atomic semantic relations across the sequence of edits:(p, h) = (x0, xn) = (x0, x1) ⋈ … ⋈ (xi–1, xi) ⋈ … ⋈ (xn–1, xn) Limitations: need to find appropriate edit sequence connecting p and h;tendency of ⋈ operation toward less-informative semantic relations; lack of general mechanism for combining multiple premises Less deductive power than FOL. Can’t handle e.g. de Morgan’s Laws.

21. Introduction • Semantic Relations • Joins • Lexical Relations • Projectivity • Implicatives • Inference • Evaluation • Conclusion An example P The doctor didn’t hesitate to recommend Prozac. H The doctor recommended medication. yes ‿ | | ^ ^ ⊏ ⊏ ⊏ ⊏ yes

22. Introduction • Semantic Relations • Joins • Lexical Relations • Projectivity • Implicatives • Inference • Evaluation • Conclusion Different edit orders? Intermediate steps may vary; final result is typically (though not necessarily) the same

23. Introduction • Semantic Relations • Joins • Lexical Relations • Projectivity • Implicatives • Inference • Evaluation • Conclusion Implementation & evaluation The NatLog system: an implementation of this model in code For implementation details, see [MacCartney & Manning 2008] Evaluation on FraCaS test suite 183 NLI problems, nine sections, three-way classification Accuracy 70% overall; 87% on “relevant” sections (60% coverage) Precision 89% overall: rarely predicts entailment wrongly Evaluation on RTE3 test suite Longer, more natural premises; greater diversity of inference types NatLog alone has mediocre accuracy (59%) but good precision Hybridization with broad-coverage RTE system yields gains of 4%

24. Introduction • Semantic Relations • Joins • Lexical Relations • Projectivity • Implicatives • Inference • Evaluation • Conclusion :-) Thanks! Questions? Conclusion Natural logic is not a universal solution for NLI Many types of inference not amenable to natural logic approach Our inference method faces many limitations on deductive power More work to be done in fleshing out our account Establishing projectivity signatures for more quantifiers, verbs, etc. Better incorporating presuppositions But, our model of natural logic fills an important niche Precise reasoning on negation, antonymy, quantifiers, implicatives, … Sidesteps the myriad difficulties of full semantic interpretation Practical value demonstrated on FraCaS and RTE3 test suites Natural logic is not a universal solution for NLI Many types of inference not amenable to natural logic approach Our inference method faces many limitations on deductive power More work to be done in fleshing out our account Establishing projectivity signatures for more quantifiers, verbs, etc. Better incorporating presuppositions But, our model of natural logic fills an important niche Precise reasoning on negation, antonymy, quantifiers, implicatives, … Sidesteps the myriad difficulties of full semantic interpretation Practical value demonstrated on FraCaS and RTE3 test suites