Predicting information seeker satisfaction in community question answering

1. SIGIR 2008 Predicting information seeker satisfaction incommunity question answering Yandong Liu, Jiang Bian, Eugene Agichteinfrom Emory & Georgia Tech University

2. CQA -Community Question Answering問答社群近來成為一個尋找資訊的有效方法

3. IS ASKER IN QA COMMUNITY SATISFIED?

4. IS ASKER IN QA COMMUNITY SATISFIED? 發問者是否得到滿意的回答 主宰著問答社群的發展或沒落

5. Asker use case Lifecycle of a question in CQA

6. Asker use case Lifecycle of a question in CQA

7. Definition • An asker in a CQA • is considered satisfied iff: • The asker personally has closed the question, selected the best answer, and provided a rating of at least 3 stars for best answer quality.

8. Definition • The Asker Satisfaction Problem: • Given a question submitted by an asker is CQA, predict whether the user will be satisfied with the answers contributed by the community.

9. Features • Question • Question-Answer Relationship • Asker User History • Answerer User History • Category Features • Textual Features

11. Formally a two-class classification problembutprimarily focus on the satisfied class.

12. Human judgment • Amazon’s paid rater service • Mechanical Turk

13. Datasets • Random 5000 from top 10000 • Skewed distribution • Satisfaction varies by category

14. Setting • Methods • Human • Heuristic • Baseline • Evaluation Metrics • Precision • Recall • F1 • Accuracy • ASP • ASP_SVM • ASP_RandomForest • ASP_C4.5 • ASP_Boosting • ASP_NaiveBayes

15. Result

16. Selected features

17. Analysis • Online vs. Offline

18. Analysis • Feature Ablation

19. Analysis • Textual Features

20. Analysis • Past Experience

21. Conclusion • First to quantify and predict asker satisfaction • Shown importance of asker history is this • Our system outperform human assessors