1 / 47

Liangjie Hong and Brian D. Davison Computer Science and Engineering Lehigh University Bethlehem, PA USA

A Classification-based Approach to Question Answering in Discussion Boards. Liangjie Hong and Brian D. Davison Computer Science and Engineering Lehigh University Bethlehem, PA USA. Liangjie Hong and Brian D. Davison. Outline. Motivation Problem Definition Features Experiments

anoki
Download Presentation

Liangjie Hong and Brian D. Davison Computer Science and Engineering Lehigh University Bethlehem, PA USA

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. A Classification-based Approach to Question Answering in Discussion Boards Liangjie Hong andBrian D. Davison Computer Science and Engineering Lehigh University Bethlehem, PA USA

  2. Liangjie Hong and Brian D. Davison Outline • Motivation • Problem Definition • Features • Experiments • Conclusion SIGIR July 2009

  3. Liangjie Hong and Brian D. Davison MotivationProblem Definition Features Experiments Conclusion How do you find answers on the Web? SIGIR July 2009

  4. Liangjie Hong and Brian D. Davison MotivationProblem Definition Features Experiments Conclusion Go to search engines… SIGIR July 2009

  5. Liangjie Hong and Brian D. Davison MotivationProblem Definition Features Experiments Conclusion Go to Question Answering Portals SIGIR July 2009

  6. Liangjie Hong and Brian D. Davison MotivationProblem Definition Features Experiments Conclusion Go to discussion boards SIGIR July 2009

  7. Liangjie Hong and Brian D. Davison MotivationProblem Definition Features Experiments Conclusion Why non-trivial? SIGIR July 2009

  8. Liangjie Hong and Brian D. Davison MotivationProblem Definition Features Experiments Conclusion Why non-trivial? Comments, news, tutorials personal experiences SIGIR July 2009

  9. Liangjie Hong and Brian D. Davison MotivationProblem Definition Features Experiments Conclusion Why non-trivial? SIGIR July 2009

  10. Liangjie Hong and Brian D. Davison MotivationProblem Definition Features Experiments Conclusion Why non-trivial? Hello, I have a problem with my GUI not loading, it used to load and then i tried installing nvidia-glx from synpatics and then rebooted it removed a load of files but for some reason it wont go back to the gui so i remove nvidia-glx and installed nvidia-glx-180 with envyng -t when i do startx it goes to a black screen prior to this icouldnt even get to install glx-180 as it said missing files in the modules and everything but now i got past that, installed the driver, but still icant get into the gui xserver is installed too as well as core SIGIR July 2009

  11. Liangjie Hong and Brian D. Davison MotivationProblem Definition Features Experiments Conclusion Why non-trivial? Hello, I have a problem with my GUI not loading, it used to load and then i tried installing nvidia-glx from synpatics and then rebooted it removed a load of files but for some reason it wont go back to the gui so i remove nvidia-glx and installed nvidia-glx-180 with envyng -t when i do startx it goes to a black screen prior to this icouldnt even get to install glx-180 as it said missing files in the modules and everything but now i got past that, installed the driver, but still icant get into the gui xserver is installed too as well as core No punctuation Spelling errors Mixed content … SIGIR July 2009

  12. Liangjie Hong and Brian D. Davison MotivationProblem Definition Features Experiments Conclusion How does Google deal with forums? SIGIR July 2009

  13. Liangjie Hong and Brian D. Davison MotivationProblem Definition Features Experiments Conclusion SIGIR July 2009

  14. Liangjie Hong and Brian D. Davison MotivationProblem Definition Features Experiments Conclusion SIGIR July 2009

  15. Liangjie Hong and Brian D. Davison MotivationProblem Definition Features Experiments Conclusion How about SIGIR July 2009

  16. Liangjie Hong and Brian D. Davison MotivationProblem Definition Features Experiments Conclusion SIGIR July 2009

  17. Liangjie Hong and Brian D. Davison MotivationProblem Definition Features Experiments Conclusion Not an answer SIGIR July 2009

  18. Liangjie Hong and Brian D. Davison MotivationProblem Definition Features Experiments Conclusion Not an answer Not an answer Not an answer Not an answer SIGIR July 2009

  19. Liangjie Hong and Brian D. Davison MotivationProblem Definition Features Experiments Conclusion What are “Questions” ? SIGIR July 2009

  20. Liangjie Hong and Brian D. Davison MotivationProblem Definition Features Experiments Conclusion What are “Questions” ? • a sentence • a paragraph • several paragraphs • a post • … SIGIR July 2009

  21. Liangjie Hong and Brian D. Davison MotivationProblem Definition Features Experiments Conclusion What are “Questions” ? • a sentence • a paragraph • several paragraphs • a post • … SIGIR July 2009

  22. Liangjie Hong and Brian D. Davison MotivationProblem Definition Features Experiments Conclusion What are “Answers” ? SIGIR July 2009

  23. Liangjie Hong and Brian D. Davison MotivationProblem Definition Features Experiments Conclusion What are “Answers” ? Hello, My problem is I can't find documentation on setting up a shared folder purely through the terminal. I know how to setup shared folders when there is a Gnome desktop on Ubuntu but my searches on this forum and Google haven't come up with instructions on how to do this purely through terminal commands. How do I add permissions to these folders for each user again purely through terminal commands? I plan on using Hardy since it's supported to 2013, but are there advantages to using Jaunty server instead that I'm not aware off? If someone can point me in the right direction on where I can find the information to set this up, it would be greatly appreciated. Thanks in advance for your help! SIGIR July 2009

  24. Liangjie Hong and Brian D. Davison MotivationProblem Definition Features Experiments Conclusion What are “Answers” ? Hello, My problem is I can't find documentation on setting up a shared folder purely through the terminal. I know how to setup shared folders when there is a Gnome desktop on Ubuntu but my searches on this forum and Google haven't come up with instructions on how to do this purely through terminal commands. How do I add permissions to these folders for each user again purely through terminal commands? I plan on using Hardy since it's supported to 2013, but are there advantages to using Jaunty server instead that I'm not aware off? If someone can point me in the right direction on where I can find the information to set this up, it would be greatly appreciated. Thanks in advance for your help! if you are using a ubuntu server and accessing it through windows workstations, you can simply install SAMBA, and then edit the smb.conf (the configuration file) through the terminal to set up file shares and permissions etc. as for users, you can simply create samba users and passwords (which could match your xp logins for simplicity) SIGIR July 2009

  25. Liangjie Hong and Brian D. Davison MotivationProblem Definition Features Experiments Conclusion What are “Answers” ? Hello, My problem is I can't find documentation on setting up a shared folder purely through the terminal. I know how to setup shared folders when there is a Gnome desktop on Ubuntu but my searches on this forum and Google haven't come up with instructions on how to do this purely through terminal commands. How do I add permissions to these folders for each user again purely through terminal commands? I plan on using Hardy since it's supported to 2013, but are there advantages to using Jaunty server instead that I'm not aware off? If someone can point me in the right direction on where I can find the information to set this up, it would be greatly appreciated. Thanks in advance for your help! if you are using a ubuntu server and accessing it through windows workstations, you can simply install SAMBA, and then edit the smb.conf (the configuration file) through the terminal to set up file shares and permissions etc. as for users, you can simply create samba users and passwords (which could match your xp logins for simplicity) Hello renzokuken..thanks for the quick response! So I could just install my server and setup my user id so I can login and administer the server. I don't need to setup userid's for each employee on the server. Instead I create samba users (which could be the userid of their windows PC's). I didn't know you could do this? I thought each user who used the share needed to have an Ubuntu login id? SIGIR July 2009

  26. Liangjie Hong and Brian D. Davison MotivationProblem Definition Features Experiments Conclusion What are “Answers” ? Hello, My problem is I can't find documentation on setting up a shared folder purely through the terminal. I know how to setup shared folders when there is a Gnome desktop on Ubuntu but my searches on this forum and Google haven't come up with instructions on how to do this purely through terminal commands. How do I add permissions to these folders for each user again purely through terminal commands? I plan on using Hardy since it's supported to 2013, but are there advantages to using Jaunty server instead that I'm not aware off? If someone can point me in the right direction on where I can find the information to set this up, it would be greatly appreciated. Thanks in advance for your help! if you are using a ubuntu server and accessing it through windows workstations, you can simply install SAMBA, and then edit the smb.conf (the configuration file) through the terminal to set up file shares and permissions etc. as for users, you can simply create samba users and passwords (which could match your xp logins for simplicity) A couple of things to note. One is that Samba has a way to handle individual user (home dir) shares. The share is called [homes] and it is for all users. The second thing is where you are creating your mount point. Although you can create the mount point anywhere in the file system, I find that it makes more sense to keep the Samba shares under its own mount point. I use /smb Hello renzokuken..thanks for the quick response! So I could just install my server and setup my user id so I can login and administer the server. I don't need to setup userid's for each employee on the server. Instead I create samba users (which could be the userid of their windows PC's). I didn't know you could do this? I thought each user who used the share needed to have an Ubuntu login id? SIGIR July 2009

  27. Liangjie Hong and Brian D. Davison MotivationProblem Definition Features Experiments Conclusion What are “Answers” ? Hello, My problem is I can't find documentation on setting up a shared folder purely through the terminal. I know how to setup shared folders when there is a Gnome desktop on Ubuntu but my searches on this forum and Google haven't come up with instructions on how to do this purely through terminal commands. How do I add permissions to these folders for each user again purely through terminal commands? I plan on using Hardy since it's supported to 2013, but are there advantages to using Jaunty server instead that I'm not aware off? If someone can point me in the right direction on where I can find the information to set this up, it would be greatly appreciated. Thanks in advance for your help! A couple of things to note. One is that Samba has a way to handle individual user (home dir) shares. The share is called [homes] and it is for all users. The second thing is where you are creating your mount point. Although you can create the mount point anywhere in the file system, I find that it makes more sense to keep the Samba shares under its own mount point. I use /smb Hello renzokuken..thanks for the quick response! So I could just install my server and setup my user id so I can login and administer the server. I don't need to setup userid's for each employee on the server. Instead I create samba users (which could be the userid of their windows PC's). I didn't know you could do this? I thought each user who used the share needed to have an Ubuntu login id? if you are using a ubuntu server and accessing it through windows workstations, you can simply install SAMBA, and then edit the smb.conf (the configuration file) through the terminal to set up file shares and permissions etc. as for users, you can simply create samba users and passwords (which could match your xp logins for simplicity) SIGIR July 2009

  28. Liangjie Hong and Brian D. Davison MotivationProblem Definition Features Experiments Conclusion What are “Answers” ? Hello, My problem is I can't find documentation on setting up a shared folder purely through the terminal. I know how to setup shared folders when there is a Gnome desktop on Ubuntu but my searches on this forum and Google haven't come up with instructions on how to do this purely through terminal commands. How do I add permissions to these folders for each user again purely through terminal commands? I plan on using Hardy since it's supported to 2013, but are there advantages to using Jaunty server instead that I'm not aware off? If someone can point me in the right direction on where I can find the information to set this up, it would be greatly appreciated. Thanks in advance for your help! A couple of things to note. One is that Samba has a way to handle individual user (home dir) shares. The share is called [homes] and it is for all users. The second thing is where you are creating your mount point. Although you can create the mount point anywhere in the file system, I find that it makes more sense to keep the Samba shares under its own mount point. I use /smb Hello renzokuken..thanks for the quick response! So I could just install my server and setup my user id so I can login and administer the server. I don't need to setup userid's for each employee on the server. Instead I create samba users (which could be the userid of their windows PC's). I didn't know you could do this? I thought each user who used the share needed to have an Ubuntu login id? if you are using a ubuntu server and accessing it through windows workstations, you can simply install SAMBA, and then edit the smb.conf (the configuration file) through the terminal to set up file shares and permissions etc. as for users, you can simply create samba users and passwords (which could match your xp logins for simplicity) Answer SIGIR July 2009

  29. Liangjie Hong and Brian D. Davison MotivationProblem Definition Features Experiments Conclusion Questions & Answers Textual Mismatch ? Can any one help me load ubuntu 8.10 on to my pc? I have a asus AS V3-P5V900 but when i load from cd it keeps crashing, i think i does not reconise the graphics card. When i boot from cd it asks me what language ENGLISH then whentryto load it crash again i have tried help and put in via=771 any help please? You mighttryusing the “alternate” install CD: http://www.ubuntu.com/getubuntu/downloadmirrors#alternate SIGIR July 2009

  30. Liangjie Hong and Brian D. Davison MotivationProblem Definition Features Experiments Conclusion A Summary • Two sub-tasks: • Question detection • Answer detection • Our method: • As a classification problem (not retrieval) • Explore simple/combination features vs. NLP • Better performance on two real world datasets SIGIR July 2009

  31. Liangjie Hong and Brian D. Davison MotivationProblem Definition Features Experiments Conclusion A Summary The only page you need to remember  • Two sub-tasks: • Question detection • Answer detection • Our method: • As a classification problem (not retrieval) • Explore simple/combination features vs. NLP • Better performance on two real world datasets SIGIR July 2009

  32. Liangjie Hong and Brian D. Davison MotivationProblem Definition Features Experiments Conclusion For questions • Question mark (1 feature) • 5W1H words (6) • why, what, where, which, when, how • Thread length (1) • total number of posts • Authorship (1) • #first post/#total posts • N-gram (1000-3000) • Carvalho et al. • Improving email speech acts analysis via n-gram selection. HLT/NAACL 2006. SIGIR July 2009

  33. Liangjie Hong and Brian D. Davison MotivationProblem Definition Features Experiments Conclusion For answers • Post position (2) • Authorship (1) • N-gram (1000-3000) • Stopwords (571) • Query Likelihood Model (Language Model) (1) SIGIR July 2009

  34. Liangjie Hong and Brian D. Davison MotivationProblem Definition Features ExperimentsConclusion Datasets • PhotographyOnTheNet(http://www.photography-on-the.net/) • 721, 442 threads • UbuntuForums (http://www.ubuntuforums.org) • 555, 954 threads • Sampled approximately 500 threads • for each sub-task and dataset SIGIR July 2009

  35. Liangjie Hong and Brian D. Davison MotivationProblem Definition Features ExperimentsConclusion Classification Method • Manually labeled • questions vs. non-questions • one best answer per thread • 10-fold cross validation • libSVM for classification • Measured performance by Precision, Recall, F-Measure, Accuracy SIGIR July 2009

  36. Liangjie Hong and Brian D. Davison MotivationProblem Definition Features ExperimentsConclusion Comparisons to existing methods • For questions • Part-Of-Speech tagging • Stanford Log-linear POS tagger • Sequential Pattern Mining • For answers • Graph-based model incorporated with inter-posts relevance, authorship and similarity • Cong et. al. Finding question-answer pairs from online forums. SIGIR 2008. SIGIR July 2009

  37. Liangjie Hong and Brian D. Davison MotivationProblem Definition Features ExperimentsConclusion Question Detection Single Feature (UbuntuForums) SIGIR July 2009

  38. Liangjie Hong and Brian D. Davison MotivationProblem Definition Features ExperimentsConclusion Question Detection Single Feature (Photography) SIGIR July 2009

  39. Liangjie Hong and Brian D. Davison MotivationProblem Definition Features ExperimentsConclusion Question Detection Combined Feature (UbuntuForums) SIGIR July 2009

  40. Liangjie Hong and Brian D. Davison MotivationProblem Definition Features ExperimentsConclusion Question Detection Combined Feature (Photography) SIGIR July 2009

  41. Liangjie Hong and Brian D. Davison MotivationProblem Definition Features ExperimentsConclusion Answer Detection Single Feature (Ubuntu) SIGIR July 2009

  42. Liangjie Hong and Brian D. Davison MotivationProblem Definition Features ExperimentsConclusion Answer Detection Single Feature (Photography) SIGIR July 2009

  43. Liangjie Hong and Brian D. Davison MotivationProblem Definition Features ExperimentsConclusion Answer Detection Combined Feature (UbuntuForums) SIGIR July 2009

  44. Liangjie Hong and Brian D. Davison MotivationProblem Definition Features ExperimentsConclusion Answer Detection Combined Feature (Photography) SIGIR July 2009

  45. Liangjie Hong and Brian D. Davison MotivationProblem Definition Features Experiments Conclusion Summary • Question Detection • N-gram • Authorship+Question Mark+5W1H+Length • Answer Detection • Position, Authorship • Position+Authorship • Language Model+Position+Authorship SIGIR July 2009

  46. Liangjie Hong and Brian D. Davison MotivationProblem Definition Features Experiments Conclusion Summary • Question Detection • N-gram • Authorship+Question Mark+5W1H+Length • Answer Detection • Position, Authorship • Position+Authorship • Language Model+Position+Authorship This is only a starting point! SIGIR July 2009

  47. Liangjie Hong and Brian D. Davison MotivationProblem Definition Features Experiments Conclusion Thank you! • Questions? • Contact Info: • Liangjie Hong • hongliangjie@lehigh.edu • WUME Laboratory • Computer Science and Engr. • Lehigh University • Bethlehem, PA 18015 USA SIGIR July 2009

More Related