AQUA: AQUAINT Question Answering System Project Progress Report

1. AQUA: AQUAINT QuestionAnswering SystemProject Progress Report SAIC, San Diego KSL, Stanford AQUA Question-Answering System

2. Project Summary AQUA Question-Answering System

3. Collaboration with NMSU Team • This spring, NMSU has been added as an AQUAINT contractor • Synergies between the NMSU Team and the SAIC Team have led to a collaborative effort for AQUAINT AQUA Question-Answering System

4. SAIC-KSL-NMSU Collaborative System SAIC Interlingua KIF Translator NMSU Query Processor QUESTION Interlingua Query NL Query KIF Query Interlingua Answer NL Answer KIF Answer NMSU NL Generator SAIC KIF  Interlingua Translator ANSWER KSL Java Theorem Prover AQUA Question-Answering System

5. Key Tasks—KSL • Providing knowledge tools for team • Ontolingua Knowledge Server (OKS) • Background ontologies and lexicons • JTP deductive answer determination system • Developing new methods for • Answer determination from large, complex KBs • Using interoperating special-purpose reasoners  • KB partitioning so that most reasoning occurs within partitions • Using Semantic Web representation languages (e.g., DAML+OIL) • Providing understandable explanations for deduced answers • Reasoning steps, data sources, reasoning method, data conversions, … • Use-specific and user-specific customization • Detecting and resolving conflicts in KBs • Proactive background testing for both likely and provable conflicts • Interactive tools for helping analyst correct or annotate conflicts AQUA Question-Answering System

6. Progress to Date—KSL • Developing JTP – An answer determining reasoner • Developing a suite of special-purpose reasoners E.g., for determining time-dependent answers • Reasoner for QA from ontological knowledge (in DAML+OIL) • Produces and caches reasoning results during KB loading • Will be able to accept a series of queries, without waiting for answers • Usability improvements to JTP • For KB development and query-answering testing and debugging • Support for rapid reloading and changing of developing KBs E.g., Checkpointing and “untell” • Hierarchical presentation of reasoning explanations • DQL – Standard query language and QA protocol for DAML+OIL • Basic framework for client/server deductive query answering • Answers generated a batch at a time • Formal semantics • Applicable to other representation languages • Being developed jointly with the DAML language design committee • Will be implemented for JTP AQUA Question-Answering System

7. Key Tasks—SAIC • Ontolingua Translator • Automatic translation of information from NMSU team into KIF knowledge representations • Includes dynamic semantic alignment of TMR Ontology and Ontolingua Ontology • Includes inferring relations that span multiple sentences • Includes flagging relations with document contexts (time, location) • Pre-reasoning to pre-answer known useful questions • Modular, interchangeable KB systems to determine specific types of relations that are likely to be highly relevant and useful • Current system is event-based, but architecture valid for other types of representations • OKS Interface • Transfer, load, and store KIF representations into Ontolingua KBs • Maintain DB of translated documents with info about contents of each • Defines what documents are relevant to specific queries • Reduces burden of KB partitioning • Each document has a separate KB file • Query TMR matched to contents of known documents to determine relevant KBs • Only KBs for relevant documents loaded into JTP to determine the answer • Source Credibility Assessment • Develop historical records of source reliability for sources • Assessment of estimated source credibility for incoming knowledge • Develop methodologies for dynamic changes to credibility ratings AQUA Question-Answering System

8. SAIC-KSL-NMSU Collaborative System SAIC Interlingua KIF Translator NMSU Query Processor QUESTION Interlingua Query NL Query KIF Query Interlingua Answer NL Answer KIF Answer NMSU NL Generator SAIC KIF  Interlingua Translator ANSWER KSL Java Theorem Prover AQUA Question-Answering System

9. Converting Semantic Info to Knowledge • Ontologies (and ontological philosophies) differ between Ontolingua and TMR • SAIC dynamically aligns the two ontologies as part of the knowledge formation process • Must be dynamic process since both TMR and Ontolingua ontologies are actively changing during system development • An automated mechanism for translating information in TMR into KIF relations is needed. • TMR breaks text into smallest pieces • SAIC must re-unify the pieces to produce more meaningful knowledge representations • Next slide shows a typical example of this problem AQUA Question-Answering System

10. Re-Unification of Knowledge SAMPLE TEXT: About 36 US Special Forces troops started a month of anti-terrorism training… • TMR includes separate references for this sentence fragment for: • united-states-soldier • soldier-human-adult • united-states-human-adult • We re-unify this into a single definition that captures that these are all the same and plural. • (defobject united-states-soldier (instance-of united-states-soldier person) (has-country united-states-soldier united-states) (member-of united-states-soldier us-army-special-forces)) • (defobject group-of-united-states-soldier (group group-of-united-states-soldier united-states-soldier) (cardinality group-of-united-states-soldier 36)) AQUA Question-Answering System

11. KIF Formulations from Narratives • Often, useful text sources are narratives rather than unrelated compilations of facts • Story comprehension must extend across sentences to the entire text of the narrative • SAIC’s experience developing knowledge representations from narratives in HPKB and RKF programs provides unique and powerful capability in this area • Our “event descriptor templates” provide relations needed to generate a series of event descriptors for a narrative • These allow us to answer highly complex questions about complicated, real-world situations • Developed and proven successful in HPKB Program • Not trying to generate all possible relations from the text—only those relations that are in the event descriptors (i.e., known useful relations) AQUA Question-Answering System

12. Why Event Templates? • Using event templates dramatically decreases the work load on JTP for each query • Distributes the analysis across multiple small reasoners specialized to answer specific types of questions • Pre-query analysis anticipates common questions that may be asked about this document and pre-determines the answers automatically • JTP’s set of multiple reasoners includes forward chaining, which may add other relations at load time rather than waiting for a specific query against the events • Result should be to dramatically improve answer response time for many queries • Also provides extensibility of the system because each set of relations is handled by a separate, modular KB reasoner • Support for other types of inputs than narratives may replace the mini-reasoners but doesn’t change the architecture AQUA Question-Answering System

13. Progress to Date • Application of general-purpose rules that apply across a broad spectrum of instances • Initial processing of event basics in place • Identification of type of event, agent performing event, basic identification of object/agent acted on in event • Subject-verb-object • Automatic definition of specific objects from text • Places, people, groups • Including cardinality of groups if available in original text • Location and time of event • Determination (from raw text dateline, etc.) of event context location and event context time • All events in this context are tagged relative to event-context-location and event-context-time AQUA Question-Answering System

14. Automatic Knowledge Representation Status (cont.) • More sophisticated event processing • Interests relations • General rules about actions supporting interests of agents performing them • Citizenship relations • Action-object events • Dealing with verbs that are objects of actions in action-object-is relations • Example: “pretending to do something” action-object-is “doing something” • Representation is an event for “doing-something-event” and a separate event for “pretending-the-doing-something-event” • Object re-unification • Recognition of previously referred to objects (from other sentences or within longer sentences) as the same object • Pronoun dereferencing • Multiple phrasings of the same object AQUA Question-Answering System

15. Status of Knowledge Representation System • Preliminary demo of automatic translation from TMR to KIF is up and running • Our demonstration system is limited at the moment • Limited ontology • Limited relations • Limited lexical terms • The Demo System is improving quickly in its capabilities; in the meantime… AQUA Question-Answering System

16. Demonstration System AQUA Question-Answering System