1 / 52

Focused Entailment Graphs for Open IE Propositions

Focused Entailment Graphs for Open IE Propositions. Omer Levy Ido Dagan Jacob Goldberger Bar- Ilan University, Israel. Open IE. Extracts propositions from text “…which makes aspirin relieve headaches.” No supervision No pre-defined schema. What’s missing in Open IE?. Structure

teva
Download Presentation

Focused Entailment Graphs for Open IE Propositions

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Focused Entailment Graphs for Open IE Propositions Omer Levy Ido Dagan Jacob Goldberger Bar-Ilan University, Israel

  2. Open IE • Extracts propositions from text “…which makes aspirin relieve headaches.” • No supervision • No pre-defined schema

  3. What’s missing in Open IE? • Structure • Open IE does not consolidate natural language expressions relieveheadachetreatheadache

  4. Adding Structure to Open IE Which structure? • Build a graph of Open IE propositions and their semantic relations

  5. Adding Structure to Open IE Which structure? • Build a graph of Open IE propositions and their entailment relations Why entailment? • Merges paraphrases into mutual entailment cliques aspirin relievesheadacheaspirin treatsheadache • Organizes information hierarchically from specific to general aspirinrelievesheadachepainkiller relieves headache

  6. aspirin, eliminate, headache aspirin, cure, headache Original Open IE Output coffee, help, headache drug, relieve, headache headache, control with, aspirin drug, treat, headache tea, soothe, headache analgesic, banish, headache headache, respond to, painkiller headache, treat with, caffeine

  7. drug, relieve, headache drug, treat, headache Consolidated Open IE Output headache, respond to, painkiller headache, treat with, caffeine analgesic, banish, headache tea, soothe, headache headache, control with, aspirin aspirin, cure, headache aspirin, eliminate, headache coffee, help, headache

  8. Semantic Applications • Example: Structured Queries • “What relieves headaches?”

  9. Semantic Applications • Example: Structured Queries • “What relieves headaches?”

  10. drug, relieve, headache drug, treat, headache Structured Query: headache, respond to, painkiller headache, treat with, caffeine analgesic, banish, headache tea, soothe, headache headache, control with, aspirin aspirin, cure, headache aspirin, eliminate, headache coffee, help, headache

  11. drug, relieve, headache drug, treat, headache Structured Query: headache, respond to, painkiller headache, treat with, caffeine analgesic, banish, headache tea, soothe, headache headache, control with, aspirin aspirin, cure, headache aspirin, eliminate, headache coffee, help, headache

  12. drug Structured Query: painkiller caffeine analgesic tea aspirin coffee

  13. Our Contributions • Structuring Open IE withProposition Entailment Graphs • Dataset: 30 gold-standard graphs, 1.5 million entailment annotations • Algorithm for constructing Focused Proposition Entailment Graphs • Analysis: Predicate entailment is not quite what we thought

  14. Proposition Entailment Graphs

  15. Related Work: Predicate Entailment Graphs • Berant et al. (2010,2011,2012) • We extend Berant et al.’s work from predicates to propositions

  16. Focused Proposition Entailment Graphs • Nodes: Open IE propositions • Edges: Textual Entailment

  17. Focused Proposition Entailment Graphs • Assumptions: Binary Propositions and Common Topic • Binary Propositions • Focused on a common topic

  18. Focused Proposition Entailment Graphs • Assumptions: Binary Propositions and Common Topic • Binary Propositions • Focused on a common topic

  19. drug, relieve, headache drug, treat, headache headache, respond to, painkiller headache, treat with, caffeine analgesic, banish, headache tea, soothe, headache headache, control with, aspirin aspirin, cure, headache aspirin, eliminate, headache coffee, help, headache

  20. drug, relieve, headache drug, treat, headache headache, respond to, painkiller headache, treat with, caffeine analgesic, banish, headache tea, soothe, headache headache, control with, aspirin aspirin, cure, headache aspirin, eliminate, headache coffee, help, headache

  21. Focused Proposition Entailment Graphs • Edges: Textual Entailment Proposition Entailment • Simpler than sentence-level entailment • More complicated than lexical entailment • Enables investigation of inference phenomena in an isolated manner

  22. Constructing Proposition Entailment Graphs Task Definition: Given a set of propositions , find all their entailment edges.

  23. Dataset

  24. Dataset: High-Quality Open IE Propositions • Google’s Syntactic N-grams • Based on millions of books • Filter for subject-verb-object • Including prepositional objects and passive • Result: 68 million high-quality propositions

  25. Dataset: Annotating Entailment Graphs • Select 30 healthcare topics • antibiotic, caffeine, insomnia, scurvy, … • Collect a set of propositions focused on each topic • Manually clean noisy extractions • Retaining 200 propositions per graph (average) • Efficiently annotate entailment • 1.5 million entailment judgments

  26. Algorithm

  27. How do we recognize proposition entailment? . ?

  28. How do we recognize proposition entailment? . Observation: propositions entail their lexical components entail

  29. How do we recognize proposition entailment? . Observation: propositions entail their lexical components entail

  30. How do we recognize proposition entailment? . Proposition entailment is reduced to lexical entailment in context

  31. Lexical Entailment Lexical Entailment Features Lexical Entailment(Logistic)

  32. Lexical Entailment Lexical Entailment Features Features • WordNet Relations • UMLS • Distributional Similarity • String Edit Distance Lexical Entailment(Logistic) Supervision

  33. From Lexical to Proposition Entailment Lexical Entailment Features Lexical Entailment(Logistic) Supervision

  34. From Lexical to Proposition Entailment Predicate Entailment Features Argument Entailment Features Predicate Entailment(Logistic) Argument Entailment(Logistic) Supervision Supervision

  35. From Lexical to Proposition Entailment Predicate Entailment Features Argument Entailment Features Predicate Entailment(Logistic) Argument Entailment(Logistic) Supervision Supervision Proposition Entailment(Conjunction)

  36. Distant Supervision (WordNet)? Predicate Entailment Features Argument Entailment Features Following Snow (2005), Berant (2012) Predicate Entailment(Logistic) Argument Entailment(Logistic) WordNet WordNet Proposition Entailment(Conjunction)

  37. Direct Supervision (30 Annotated Graphs) Predicate Entailment Features Argument Entailment Features Predicate Entailment(Logistic) Argument Entailment(Logistic) Proposition Entailment(Conjunction) Annotated Graphs

  38. Direct Supervision (30 Annotated Graphs) Predicate Entailment Features Argument Entailment Features Hidden Layer Proposition Entailment(Conjunction) Annotated Graphs

  39. FlatModel Predicate Entailment Features Argument Entailment Features Proposition Entailment(Logistic) Annotated Graphs

  40. Compared Methods • Component-Level Distant Supervision (WordNet) • Predicates & Arguments • Predicates Only • Arguments Only • Proposition-Level Direct Supervision (30 Annotated Graphs) • Hierarchical (our method) • Flat • All methods used Berant et al.’s Global Optimization method

  41. Results

  42. Direct Supervision: Flat vs Hierarchical • Hierarchal model performs better than flat model • Better to model predicate and argument entailment separately

  43. Distant vs Direct Supervision • Direct supervision is better • Although WordNet provides more training examples

  44. Predicate Entailment with Distant Supervision • Ignoring predicates improves distant supervision baselines

  45. Are WordNet relations capturing real-world predicate entailments?

  46. Predicate Entailment vs WordNet Relations Over a predicate inference subset, how many predicate entailments are covered by WordNet? • Positive indicators • synonyms, hypernyms, entailment

  47. Predicate Entailment vs WordNet Relations Why isn’t WordNet capturing predicate entailment? Over a predicate inference subset, how many predicate entailments are covered by WordNet? • Positive indicators • synonyms, hypernyms, entailment • Negative Indicators • antonyms, hyponyms, cohyponyms

  48. Predicate Entailment is Context-Sensitive The words do not necessarily entail, but the situations do.

  49. Predicate Entailment is Context-Sensitive The words do not necessarily entail, but the situations do.

  50. Investigating Context-Sensitive Entailment • Recent work on context-sensitive lexical inference • e.g. (Melamud et al., 2013) • Previous datasets • Lexical substitution (McCarthy and Navigli, 2007) • Predicate inference (Zeichner et al., 2012) • We offer a new dataset of real-world lexical entailments in context! • Sample:synthetic vs naturally occurring • Size:several thousands vs 1.5 million

More Related