1 / 17

Deliverable 3 Abdullah Alotayq , Dong Wang, Ed Pham

A Basic Q/A System: Passage Retrieval. Deliverable 3 Abdullah Alotayq , Dong Wang, Ed Pham. Outline. Query Expansion Document Ranking Passage Retrieval Passage Re-ranking . Query Expansion. Two different methods: Target Concatenation

senwe
Download Presentation

Deliverable 3 Abdullah Alotayq , Dong Wang, Ed Pham

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. A Basic Q/A System: Passage Retrieval Deliverable 3Abdullah Alotayq, Dong Wang, Ed Pham

  2. Outline • Query Expansion • Document Ranking • Passage Retrieval • Passage Re-ranking

  3. Query Expansion • Two different methods: • Target Concatenation • Add the target for each question to the end of the question. • Deletion/Addition • Deletion of wh-words + function words • Addition of synonyms and hypernyms (via WordNet)

  4. Query Expansion • Deletion

  5. Query Expansion • Addition • Synonyms • Hypernyms • First Ancestor • Morphological variants • WordNetas thesaurus: wordnet.morphy

  6. Document Retrieval • Using Indri/Lemur • Ran both query reformulation/expansion approaches through the software. • Took the top 50 documents per query.

  7. Passage Retrieval • Used Indri/Lemur • Took the top passage from each of the top 50 documents for each query. • Query grammar • #combine[passageWIDTH:INC] • Default for system: 120 terms, 1000 terms window

  8. Passage Re-ranking • Modified the window size • 500, 1000 terms • Modified the number of top passages taken from the top 50 documents: • 1, 5, 10, 20, 25 passages

  9. Evaluation • Document ranking Note: All results based on TREC-2004

  10. Evaluation • Passage Retrieval

  11. Evaluation • Passage re-ranking: Top N passages

  12. Evaluation • Passage Re-ranking: Window Size

  13. Conclusions • “Less is Better”… for the most part. • Query Expansion was not beneficial in improving passage retrieval. • Smaller window size contributed to higher scores. • Not the case for the top N passages though • Less passages resulted in lower scores • Mainly because of less passages to work with

  14. Issues and Future Improvements • Run times • Poor performance times for “addition/subtraction” query expansion approach • Too broad of a query • Reduce the number of hypernyms/synonyms • Limited documents • Only did 50, could have done more • Same with passages

  15. Issues and Future Improvements • Query Grammar • Change it to assist in passage re-ranking • Examples • #score • passage length • different weights for different terms

  16. Readings • Query Expansion/Reformulation • Kwok, Etzioni, and Weld, 2001 • Lin, 2007 • Fang, 2008 • Aktolga et al, 2011 • Passage Retrieval • Tiedemann et al, 2008 • Indri/Lemur documentation

  17. Explorations • CELEX • English, Dutch, German Lexical resource • Beneficial for adding Derivational variants • Sepia • MIT developed Symantec system • Semantic Parsing for Named Entities • Both not available online • Query Expansion Techniques for Question Answering, byMatthew W. Bilotti

More Related