1 / 38

QuASI: Question Answering using Statistics, Semantics, and Inference

QuASI: Question Answering using Statistics, Semantics, and Inference. Marti Hearst, Jerry Feldman, Chris Manning, Srini Narayanan Univ. of California-Berkeley / ICSI / Stanford Univ. Outline. Project Overview Three topics: Assigning semantic relations via lexical hierarchies

heidi
Download Presentation

QuASI: Question Answering using Statistics, Semantics, and Inference

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. QuASI:Question Answering using Statistics, Semantics, and Inference Marti Hearst, Jerry Feldman, Chris Manning, Srini Narayanan Univ. of California-Berkeley / ICSI / Stanford Univ

  2. Outline • Project Overview • Three topics: • Assigning semantic relations via lexical hierarchies • From sentences to meanings via syntax • From text analysis to inference using conceptual schemas

  3. Main Goals Support Question-Answering and NLP in general by: • Deepening our understanding of concepts that underlie all languages • Creating empirical approaches to identifying semantic relations from free text • Developing probabilistic inferencing algorithms

  4. Two Main Thrusts • Text-based: • Use empirical corpus-based techniques to extract simple semantic relations • Combine these relations to perform simple inferences • “statistical semantic grammar” • Concept-based: • Determine language-universal conceptual principles • Determine how inferences are made among these

  5. Assigning Semantic Relations Using a Lexical Hierarchy

  6. Noun Compounds (NCs) • Any sequence of nouns that itself functions as a noun • asthma hospitalizations • asthma hospitalization rates • health care personnel hand wash • Technical text is rich with NCs Open-labeled long-term study of the subcutaneous sumatriptan efficacy and tolerability in acute migraine treatment.

  7. NCs: 3 computational tasks(Lauer & Dras ’94) • Identification • Syntactic analysis (attachments) • [Baseline [headache frequency]] • [[Tension headache] patient] • Semantic analysis • Headache treatment treatment for headache • Corticosteroid treatment treatment that uses corticosteroid

  8. The lexical Hierarchy: MeSH Tree Structures 1. Anatomy [A] 2. Organisms [B] 3. Diseases [C] 4. Chemicals and Drugs [D] 5. Analytical, Diagnostic and Therapeutic Techniques and Equipment [E] 6. Psychiatry and Psychology [F] 7. Biological Sciences [G] 8. Physical Sciences [H] 9. Anthropology, Education, Sociology and Social Phenomena [I] 10. Technology and Food and Beverages [J] 11. Humanities [K] 12. Information Science [L] 13. Persons [M] 14. Health Care [N] 15. Geographic Locations [Z]

  9. 1. Anatomy [A] Body Regions [A01] + Musculoskeletal System [A02] Digestive System [A03] + Respiratory System [A04] + Urogenital System [A05] + Endocrine System [A06] + Cardiovascular System [A07] + Nervous System [A08] + Sense Organs [A09] + Tissues [A10] + Cells [A11] + Fluids and Secretions [A12] + Animal Structures [A13] + Stomatognathic System [A14] (…..) Body Regions [A01] Abdomen [A01.047] Groin [A01.047.365] Inguinal Canal [A01.047.412] Peritoneum [A01.047.596] + Umbilicus [A01.047.849] Axilla [A01.133] Back [A01.176] + Breast [A01.236] + Buttocks [A01.258] Extremities [A01.378] + Head [A01.456] + Neck [A01.598] (….) MeSH Tree Structures

  10. Mapping Nouns to MeSH Concepts • headache recurrence • C23.888.592.612.441 C23.550.291.937 • headache pain • C23.888.592.612.441 G11.561.796.444 • breast cancer cells • A01.236 C04 A11

  11. Descent of Hierarchy • Idea: • Use the top levels of the lexical hierarchy to identify semantic relations • Hypothesis: • A particular semantic relations holds between all 2-word NCs that can be categorized by a category pair.

  12. Linguistic Motivation • Can cast NC into head-modifier relation, and assume head noun has an argument and qualia structure. • (used-in): kitchen knife • (made-of): steel knife • (instrument-for): carving knife • (used-on): putty knife • (used-by): butcther’s knife

  13. Distribution of Category Pairs

  14. Classification decisions • A01 N02 • A01 N02.360 • A01 N02.278 • A01 N02.421 • A01 N03 (W) • J01 A01 (W) • A02 F01 • C04 H01 • C04 H01.548 • C04 H01.671 • C04 H01.770

  15. Levels of the classification decision • Anatomy: 250 CPs • 187 (75%) remain first level • 56 (22%) descend one level • 7 (3%) descend two levels • Natural Science (H01): 21 CPs • 1 (4%) remain first level • 8 (39%) descend one level • 12 (57%) descend two level • Neoplasm (C04) 3 CPs: • 3 (100%) descend one level

  16. Evaluation • Test decisions on “testing” set • Count how many NCs that fall in the groups defined in the classification “rules” are similar with each other • Accuracy: • Anatomy: 91% accurate • Natural Science: 79% • Diseases: 100% • Total: • 89.6% via intra-category averaging • 90.8% via extra-category averaging

  17. Future work • Analyze full spectrum of hierarchy • NCs with > 2 terms • [[growth hormone] deficiency] • (purpose + defect) • Other syntactic structures • Non-biomedical words • Other ontologies (e.g.,WordNet)?

  18. From sentences to meanings via syntax A* Parsing & Stochastic HPSG

  19. 1. A* Parsing • Goal: develop parsers that are • Accurate – produce good parses • Exact – find the models’ best parses • Fast – seconds to parse long sentences • Exhaustive Parsing – Slow but Exact • e.g., chart Parsing, [Earley 70, Kay 80] • Approximate Parsing – Fast but Inexact • Beam Parsing, [Collins 97, Charniak 01] • Best-First Parsing [Charniak et al. 98, etc.] • Technology exists to get any two, but not all three of these goals

  20. A* Search for Parsing Score =  • Problem with uniform-cost parse search • Even unlikely small edges have high score. • We end up processing every small edge! • Solution: A* Estimates • Small edges have to fit into a full parse. • The smaller the edge, the more the full parse will cost!    Score =  +

  21. Add left tag: Score = -13.9 Add right tag: Score = -15.1 Fix outside size: Score = -11.3 Entire context gives the exact best parse. Score = -18.1 The Estimate Trade-off • The more we specify, the better estimate of  we get…

  22. A* Savings: Penn Treebank (SX-F filters more than Caraballo and Charniak 1998 while guaranteeing optimality, but less than Charniak et al. 1998)

  23. 2. Stochastic HPSG / Redwoods Treebank • The Redwoods treebank is being built at Stanford as a resource for for deep NLP • Provides full HPSG (Head-driven Phrase Structure Grammar) analyses, including semantic logical forms • Current corpus is spoken dialog data (Verbmobil) parsed by robust broad coverage HPSG grammar • Information at different levels of detail can be extracted from the treebank • Precise deep grammatical analyses can be combined with probabilistic models • Procedures are being developed for automatically updating the treebank

  24. SUBJH I HCOMP BE_C_AM SORRY_A1 “I” “am” “sorry” Basic Representation Levels • Derivation tree of lexical items and constructions • Phrase Structure Tree (S (NP “I”)(VP (V “am”) (ADJ_P “sorry”))) • Underspecified MRS meaning representation <e1:BOOL:INDICATIVE*:PRESENT*:STRICT_NONPRF, {h2: pron_rel(x3:-*:STD_1SG:1SG:GENDER), h4: def_rel(x3,h5,h6,v7:BOOL),h8:_sorry_rel(e1,x3,v9:BOOL,v10:BOOL),h11: prpstn_rel(h12)}, {h5 QEQ h2, h12 QEQ h8}> • Full HPSG signs for sentences are available

  25. Initial exploration: PCFG Results Models with increasing parent node annotation Generative PCFG vs. loglinear modeling Accuracy Log lin ear PC FG Complete match (parse selection) accuracy

  26. _need2_rel _need2_rel _meet_v_rel pron_rel _meet_v_rel pron_rel again_rel with_rel with_rel again_rel pron_rel pron_rel In progress work: Using semantic forms • Example: “I need to meet with you again”. • I need to [[meet with you] again] (preferred) • (ii) I [need to [meet with you] again] • People use semantic information to disambiguate • Building random field models over relations

  27. Concept-based Analysis From text analysis to inference using conceptual schemas

  28. Hypothesis: Linguistic input is converted into a mental simulation based on bodily-grounded structures. Components: Semantic schemas image schemas and executing schemas are abstractions over neurally grounded perceptual and motor representations Linguistic units lexical and phrasal construction representations invoke schemas, in part through metaphor Inferencelinks these structures and provides parameters for a simulation engine Inference and Conceptual Schemas

  29. Conceptual Schemas • Much is known about conceptual schemas, particularly image schemas • However, this understanding has not yet been formalized • We will develop such a formalism • They have also not been checked extensively against other languages • We will examine Chinese, Russian, and other languages in addition to English

  30. Schema Formalism SCHEMA <name> SUBCASE OF <schema> EVOKES <schema> AS <local name> ROLES < self role name>: <role restriction> < self role name> <-> <role name> CONSTRAINTS <role name> <- <value> <role name> <-> <role name> <setting name> :: <role name> <-> <role name> <setting name> :: <predicate> | <predicate>

  31. A Simple Example SCHEMA hypotenuse SUBCASE OF line-segment EVOKES right-triangle AS rt ROLES Comment inherited from line-segment CONSTRAINTS SELF <-> rt.long-side

  32. Source-Path-Goal SCHEMA: spg ROLES: source: Place path: Directed Curve goal: Place trajector: Entity

  33. Translational Motion SCHEMA translational motion SUBCASE OF motion EVOKES spg AS s ROLES mover <-> s.trajector source <-> s.source goal <-> s.goal CONSTRAINTS before:: mover.location <-> source after:: mover.location <-> goal

  34. Extending Inferential Capabilities • Given the formalization of the conceptual schemas • How to use them for inferencing? • Earlier pilot systems • Used metaphor and Bayesian belief networks • Successfully construed certain inferences • But don’t scale • New approach • Probabilistic relational models • Support an open ontology

  35. A Common Representation • Representation should support • Uncertainty, probability • Conflicts, contradictions • Current plan • Probabilistic Relational Models (Koller et al.) • DAML + OIL

  36. An Open Ontology for Conceptual Relations • Build a formal markup language for conceptual schemas • We propose to use DAML+OIL as the base. • Advantages of the approach • Common framework for extending and reuse • Closer ties to other efforts within AQUAINT as well as the larger research community on the Semantic Web. • Some Issues • Expressiveness of DAML+OIL • Representing Probabilistic Information • Extension to MetaNet, capture abstract concepts

  37. DAML-I: An Image Schema Markup Language <daml:Class rdf:ID="SPG"> <rdf s:comment> A basic type of schema </rdfs:comment> <rdfs:subClassOf rdf:resource="#Schema"/> </daml:Class> <daml:objectProperty rdf:ID="source"> <daml:subPropertyOf rdf:resource="&conc-rel;#role"/ <daml:domain rdf:resource="#SPG"/> <daml:range rdf:resource="&daml;#Thing"/> </daml:objectProperty>

  38. Putting it all Together • We have proposed two different types of semantics • Universal conceptual schemas • Semantic relations • In Phase I they will remain separate • However, we are exploring using PRMs as a common representational format • In later Phases they will be combined

More Related