1 / 21

Quality-Aware Collaborative Question Answering: Methods and Evaluation

This paper introduces a quality-aware QA framework that considers answer relevance and quality. It presents expertise-based methods and evaluates their performance. The experiments show that quality-aware methods improve answer quality and overall performance.

vernonwashington
Download Presentation

Quality-Aware Collaborative Question Answering: Methods and Evaluation

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Quality-Aware Collaborative Question Answering: Methods and Evaluation Maggy Anastasia Suryanto, Ee-Peng Lim, Aixin Sun, and Roger H. L. Chiang. In Proceedings of the Second ACM International Conference on Web Search and Data Mining (Barcelona, Spain, February 9-12, 2009). Prepared and Presented by Baichuan Li January 2, 2020

  2. Outline • Introduction • Quality-Aware Framework • Expertise Based Methods • Experiments • Conclusion Paper Presentation 2/21

  3. Introduction • Community-Based Question-Answering (CQA) Services Paper Presentation 3/21

  4. Diverse Answer Qualities good poor fair Paper Presentation 4/21

  5. Objective • Automatically find good answers for a user given questions from a community QA portal • answer features • user expertise of answers Paper Presentation 5/21

  6. Quality-Aware Framework Paper Presentation 6/21

  7. Expertise Based Methods Relevance score Quality score Paper Presentation 7/21

  8. Expertise Based Methods Paper Presentation 8/21

  9. Question Independent Expertise • EXHITS uses qscore_exhits(a) as the quality score of an answer a given in below equation: authority hub Paper Presentation 9/21

  10. Question Dependent Expertise Paper Presentation 10/21

  11. Question Dependent Expertise • EX_QD • EX_QD’ Paper Presentation 11/21

  12. Answer Relevance Models • Answer ranking by Yahoo! Answers • Query likelihood retrieval model all answers and questions in the dataset Paper Presentation 12/21

  13. Experiments • Methods Compared • BasicYA • BasicYA(subject + content) • BasicYA(subject + content + best answers) • BasicQL • Adopts query likelihood retrieval model to score the relevance of an answer • NT (classification based on non-textual answer features) • maximum entropy approach • 9 features Paper Presentation 13/21

  14. Dataset Paper Presentation 14/21

  15. Evaluation • The top 20 of the ranked answers of each methods were manually judged in terms of their relevance and quality. • The following evaluation metrics are used to evaluate the accuracy of the methods: Paper Presentation 15/21

  16. Results Paper Presentation 16/21

  17. Results Paper Presentation 17/21

  18. Results Paper Presentation 18/21

  19. Conclusion • Introduce a quality-aware QA framework that considers both answer relevance and quality in selecting answers to be returned. • Develop several QA methods (namely, EXHITS, EXHITS QD, EX QD and EX QD') that consider answerer expertise to determine answer quality. • Conducted extensive experiments and these experiments showed that quality-aware methods can improve both quality and overall performance. Among them, the methods EX QD and EX QD' using question dependent answerer expertise have the best performance. Paper Presentation 19/21

  20. Ideas Paper Presentation 20/21

  21. Q&A Paper Presentation 21/21

More Related