1 / 29

Hapnes Toba, Mirna Adriani, and Ruli Manurung Faculty of Computer Science Universitas Indonesia

Predicting Answer Location Using Shallow Semantic Analogical Reasoning in a Factoid Question Answering System. Hapnes Toba, Mirna Adriani, and Ruli Manurung Faculty of Computer Science Universitas Indonesia. What is QAS. Question answering sustem (QAS): Input: a natural language question

zihna
Download Presentation

Hapnes Toba, Mirna Adriani, and Ruli Manurung Faculty of Computer Science Universitas Indonesia

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Predicting Answer Location Using Shallow Semantic Analogical Reasoningin a Factoid Question Answering System Hapnes Toba, Mirna Adriani, and Ruli Manurung Faculty of Computer Science Universitas Indonesia

  2. What is QAS • Question answering sustem (QAS): • Input: a natural language question • Output: single answer

  3. What is Factoid QAS • Factoid QAS: • Input: an open-domin fact-based question • Output: answer • E.q: • Question: • “Where was an Oviraptor fossil sitting on a nest discovered?” • Answer: • ‘Mongolia’s Gobi Desert’

  4. A Typical pipeline architecture Factoid QAS Question analysis Query formulation Information retrieval Answer selection

  5. A Typical pipeline architecture Factoid QAS Question analysis • Determine the type of a given question, which in turn provides the expected answer type (EAT) • E.q. : person, organization, location. • named-entity recognizer (NER) is usually judged EAT

  6. Semantic Analogical Reasoning (SAR) • SAR predict the location of the final answer in a textual passage by employing the analogical reasoning (AR) framework from Silva et al. (2010). • Author hypothesize that similar questions give similar answers.

  7. Figure 1: Idea of Semantic Analogical Reasoning

  8. SAR System Architecture

  9. Semantic Analogical Reasoning (SAR)

  10. Analogical Reasoning (AR) • AR focus on the similarity between functions that map pairs to links.

  11. Analogical Reasoning (AR) • Lij∈ {0, 1} : • indicator of the existence of a relation between two related objects i and j. • Consider then that we also have K-dimensional vectors, each consisting of features which relates the objects i and j : = Θ [Θ 1 . . . Θ k]T . • This vector will represent the presence or absence of relation between two particular objects.

  12. Analogical Reasoning (AR) • Given the vectors of features Θ, the strength of the relation between two objects i and j is computed by performing logistic regression estimation as follows: P(Lij |xij , Θ) = logistic(Θ TXij) where logistic(x) is defined as: 1 / (1 + e-x)

  13. Analogical Reasoning (AR) • During AR training phase, the framework learns the weight (prior) for each feature by performing the following equation:

  14. Analogical Reasoning (AR) • During the AR retrieval phase, a final score that indicates the rank of predicted relations between two new objects i and j (query) and the related objects that have been learnt in a given set S is compute as follows:

  15. Analogical Reasoning (AR)

  16. Experiments and Evaluation objectives of experiments • find the importance level of the feature set • evaluate the potential of our approach to locate factoid answers in snippets and document retrieval scenarios without using any NER-tool • For this objective we run two kinds of experiments.

  17. Experiments and Evaluation • use the question answer pairs from CLEF 1 English monolingual of the year 2006, 2007 and 2008.

  18. Experiments and Evaluation Importance of feature

  19. Experiments and Evaluation Gold Standard Snippets • Assume: • IR process performed perfectly and returns the best snippet which covers the final answer.

  20. Experiments and Evaluation Gold Standard Snippets

  21. Experiments and Evaluation Gold Standard Snippets: • improve TIMEand MEASURE • TIME: • dd/mm/yy, • dd-mmmyy, • a single year number • hh:mm a.m./p.m. • sometimes the chunker recognizes variations as numbers or as nouns. • MEASURE : • A measurement can be written as numbers (for example: “40”) or as text (“forty”)

  22. Experiments and Evaluation Gold Standard Snippets • ADVP = Adverb phrase • NP = Noun phrase • PP = Prepositional phrase • O = Begin/End of a sentence or a coordinating conjinction

  23. Experiments and Evaluation Indri Document Retrieval • In the real situation, we will not have any information about the semantic chunk of the final answer. • We assume that the best pair (i.e. the top-1 pair after the re-ranking process) of the AR answer features will supply us with that information.

  24. Experiments and Evaluation Indri Document Retrieval • performed IR process by using Indri Search Engine to retrieve the top-5 documents and pass them on to Open Ephyra and our system. • Use same AR feature set as in the first experiment • only use the question feature set • Due to the lack of the answer features, we need to adjust the way of the re-ranking process.

  25. Experiments and Evaluation Indri Document Retrieval

  26. Experiments and Evaluation Indri Document Retrieval • ADVP = Adverb phrase • NP = Noun phrase • PP = Prepositional phrase • O = Begin/End of a sentence or a coordinating conjinction

  27. Experiments and Evaluation Indri Document Retrieval

  28. Experiments and Evaluation Indri Document Retrieval

  29. Conclusion • In this paper we have shown that by learning analogical linkages of question-answer pairs we can predict the location of factoid answers of a given snippet or document. • Author approach achieves a very good accuracy in the OTHER answer-type

More Related