Is Question Answering an Acquired Skill?

1. Is Question Answering an Acquired Skill? Ramakrishnan, Chakrabarti, Paranjpe, Bhattacharyya Paper presentation: Vinay Goel

2. Introduction • Question Answering (QA) system • Most QA systems are substantial team efforts • Difficult to reproduce a well tuned QA system from scratch, gauge the benefit of new algorithmic ideas, new corpora and new languages • QA systems • Complex building blocks like taggers and parsers • Lashed together with customized “glue” • Crucial knobs best preset by QA specialists

3. Goal • Decompose the QA task cleanly into discovering features and learning to score answer snippets • QA system • Performs fast, shallow processing of corpus • Structures the scoring task using features and learners • Trains its scoring algorithm from a past history of questions and vetted answers • Can include side information (Wordnet etc.) • Reuses expertise accumulated from one corpus to a new corpus

4. Noisy simulation perspective • In a structured database with a suitable schema and a structured query language, information needs can be expressed clearly • See QA as a transformation of this process by adding natural language from an unknown generative process, for both query and data • Given a question, discover structured fragments in it • Extract selectors which will appear almost unchanged in an answer passage • Extract atype clues, which tell what else to look for in a passage that satisfied all selectors

5. Typical connections between a question and answer

6. Atype • Minimal subclass of entities which will answer a question • Two representations important to factoid QA • Atype as synset • Atype as surface patterns

7. Atype as synset • Q: Name an animal that sleeps upright • A: horse • Wordnet helps recognize that horse is an instance of animal • Most answers which are common nouns are assisted by this representation

8. Atype as surface patterns • Infinite or very large domains such as numbers, person names, place names etc. cannot be covered by Wordnet • Logically augment Wordnet to add connections from synsets to pattern matchers such as “at DD:DD” or “Xx+ said” etc.

9. From the question to an atype • Set of common “wh-words” • Questions starting with when, where and who immediately reveal their expected atypes • Word after how is almost always a clue • Questions using using what and which mention atype directly

10. Shallow parsing to extract atype • Shallow parsing involves finding noun phrases, modifiers and attachments between phrases • Purely based on POS tags • Strategy for locating atype clues from “what” and “which” questions: • Head of NP appearing before the auxiliary / main verb if it is not a wh-word • Otherwise, head of NP appearing after the verb

11. Learning to map atype • When, where, who and how do not directly use a term that describes a synset • Augmented synsets based on surface patterns (DDDD) may come handy • Devised a learning module to help compile mappings between short token sequences in questions to atypes

12. Selectors • Second blank in the SQL query select…where… • In QA, simply a set of words in the question that are expected to appear unchanged in the answer passage

13. Identifying selectors • Choice of features • POS • POS assigned to left and right neighbors • Whether the word starts with an uppercase letter • Whether the word is a stopword • Some version of IDF • How many senses the word has in isolation • For a given sense, how many other words describe the sense

14. How to use selectors • Two places • Pad the initial keyword query • Rerank the candidate phases • Experiments insist that the response by the word search engine (Lucene) • Contains all selectors • Use OR over other question words

15. Learning to score passages • If (q,r) is a positive instance, it is expected that • All selectors match between q and r • r has an answer zone a which does not contain selectors • The linear distance between a and matched selectors in r, tend to be small • a has strong Wordnet-based similarity with the atype of q

16. Overall architecture

17. Experiments • TREC QA track • Picked sliding windows of three sentences as passages • Questions and passages were tokenized using GATE • For learning tasks, used J48 decision tree and the logistic regression packages in WEKA

18. Extracting atypes from shallow parses

19. Spotting selectors in questions

20. Passage Reranking performance

21. MRR improvement via reranking

22. Training on corpus of another year

23. Conclusion • QA system built by wrapping logic around text indexers, taggers, shallow parsers and classifiers • Simple assembly of building blocks • Future work involves improving performance of different blocks