1 / 50

Opinion Integration and Summarization

Opinion Integration and Summarization. Yue Lu University of Illinois at Urbana-Champaign. Opinions needed in all kinds of decision processes. “What do people complain about iPhone ?” “How do people like the new drug ?” “How is the new policy received?”. Business intelligence.

torin
Download Presentation

Opinion Integration and Summarization

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Opinion Integration and Summarization Yue Lu University of Illinois at Urbana-Champaign

  2. Opinions neededin all kinds of decision processes “What do people complain about iPhone?” “How do people like the new drug?” “How is the new policy received?” • Business intelligence • Health informatics • Political science http://sifaka.cs.uiuc.edu/yuelu2/

  3. Online opinions cover all kinds of topics Topics: People Events Products Services, … … Sources: Blogs Microblogs Forums Reviews ,… 53M blogs 1307M posts 65M msgs/day 45M reviews 115Musers 10M groups … http://sifaka.cs.uiuc.edu/yuelu2/

  4. After collecting opinions using Google How could I read them all? http://sifaka.cs.uiuc.edu/yuelu2/

  5. Online opinions are complicated Aspect Sentiment Quality Low quality High quality http://sifaka.cs.uiuc.edu/yuelu2/

  6. Vision: Opinion Integration & Summarization Online Opinions Major Challenge: develop generaltechniques that work for arbitrary topics Sentence1 … Sentence 2 Sentence 100 … Sentence 900 … • Opinion • Integration • Sentiment • Analysis • Quality Prediction Topic = t Integrated Summary Sentence 512 Sentence 823 Sentence 21 Sentence 153 http://sifaka.cs.uiuc.edu/yuelu2/

  7. Existing work cannot scale to different topics • Heavily rely on domain specific • Hand-labeled training data • Hand-written heuristics/rules • Review summarization • Unsupervised feature extraction + opinion polarity identification: [Hu&Liu 04], [Popescu&Etzioni 05], … • Supervised aspect extraction: [Zhuang et al] … • Hidden aspect discovery: [Hofmann99] [[Chen&Dumais00] [Blei et al03] [Zhai et al04] [Li&McCallum06] [Titov&McDonald08]… • Sentiment classification • Binary classification:[Pang&Lee02] [Kim&Hovy04] [Cui et al06] … • Rating classification: [Pang&Lee05] [Snyder&Barzilay07] … • Opinion Quality Prediction: [Zhang&Varadarajan`06] [Kim et al. `06] [Liu et al. `08] [Ghose&Ipeirotis `10]… How to? develop generaltechniques that work for arbitrary topics … http://sifaka.cs.uiuc.edu/yuelu2/

  8. New idea: exploit naturally available resources Online Opinions Structured Ontology Expert Articles Sentence1 … Sentence 2 [COLING'10] [WWW‘08] Sentence 100 Sentence 900 … Topic = t Overall Sentiment Ratings Social Networks Integrated Summary [WWW‘09] [KDD’10] [WWW’11] [WWW'10] http://sifaka.cs.uiuc.edu/yuelu2/

  9. Intuition: scalable to different topics 22 M topics 3.5 M things 3.5 M articles >3 K products/y • Opportunities? • Provide domain-specific guidance • Alleviate heavy dependence on human labors • Challenges? • Cannot directly apply supervised machine learning • Need for new methods >3 M users 500 M users 45M reviews http://sifaka.cs.uiuc.edu/yuelu2/

  10. My Work Online Opinions Sentence1 … Sentence 2 Sentence 100 … Sentence 900 … • Opinion • Integration • Sentiment • Analysis • Quality Prediction Topic = t Integrated Summary [WWW’08] [COLING'10] [WWW’10] [WWW’09] [KDD’10] [WWW’11] Sentence 512 Sentence 823 Sentence 21 Sentence 153 http://sifaka.cs.uiuc.edu/yuelu2/

  11. Roadmap • [WWW’11] “Automatic Construction of a Context-Aware Sentiment Lexicon: an Optimization Approach” • Opinion • Integration • Sentiment • Analysis • Quality Prediction [WWW’08] [COLING'10] [WWW’10] [WWW’09] [KDD’10] [WWW’11] Integrated Summary Sentence 512 Sentence 823 Sentence 21 Sentence 153 http://sifaka.cs.uiuc.edu/yuelu2/

  12. A well-known challenge: sentiments are domain dependent Existing Work • Linguistic heuristics[Hatzivassiloglou&McKeown `97], [Kanayama&Nasukawa `06], … • Morphology, synonymy [Neviarouskaya et al `09], [Mohammad et al `09], … • Seed sentiment words[Turney&Littman `03], … • Document-level sentiment rating [Choi and C. Cardie. `09], … Domain = Movie “unpredictable” Domain = Laptop http://sifaka.cs.uiuc.edu/yuelu2/

  13. Sentiments are also aspectdependent Aspect = Screen “large” Aspect = Battery Domain = Laptop http://sifaka.cs.uiuc.edu/yuelu2/

  14. New problem: • constructing aspect-dependent sentiment lexicon Input: Laptop Collection “Aspects” • SCREEN: screen, LCD, display, … • BATTERY: battery, power, charger, … • PRICE: price, cost, money, … • … + A challenging problem: due to increased sparseness Output: “Aspect-Adj”: sentiment_score SCREEN-large +1 SCREEN-great +1 BATTERY-large -1 … … http://sifaka.cs.uiuc.edu/yuelu2/

  15. Our idea: exploit multiple resources Overall Sentiment Ratings General Sentiment Lexicon Dictionary excellent, awesome, … bad, terrible, … Screen: text… Battery: text… … 1 2 large~ big, … large<->tiny, … ? Language Heuristics 1. “and” clue • Challenges: • signals in different format • contradictory signals 2. “but” clue • Synonyms • Antonyms 3. “negation” clue SCREEN-large SCREEN-great BATTERY-large 3 4

  16. A Novel Optimization Framework Objective function designed to encode signals from multiple resources S = argmin λprior S + λrating SCREEN-large S1 SCREEN-great S2 BATTERY-large S3 … … + λsim + λoppo + δ • S: Aspect-Dependent • Sentiment Lexicon Constraints subject to http://sifaka.cs.uiuc.edu/yuelu2/

  17. 1. sentiment prior SCREEN-great 1 SCREEN-bad -1 BATTERY-great 1 … … S = argmin λprior S + λrating SCREEN-large S1 SCREEN-great S2 BATTERY-large S3 … … • G: General-purpose • Sentiment Lexicon + λsim + λoppo + δ • S: Aspect-Dependent • Sentiment Lexicon http://sifaka.cs.uiuc.edu/yuelu2/

  18. 2. overall sentiment rating S = argmin λprior S + λrating SCREEN-large S1 SCREEN-great S2 BATTERY-large S3 … … + λsim + λoppo + δ • S: Aspect-Dependent • Sentiment Lexicon • X: Review Word Matrix • Predicted Ratings * • O: Review Overall Ratings R1 1 R2 1 R3 -1 R4 0 … .. R1 0.8 R2 0.5 R3 -0.7 R4 0.1 … .. R1 SCREEN-bright 0.2 R1 BATTERY-large 0.3 R1 SCREEN-great 0.5 R2 SCREEN-awesome 0.4 … .. ~ =

  19. 3. similar sentiments S = argmin λprior S + λrating SCREEN-large S1 SCREEN-great S2 BATTERY-large S3 … … + λsim + λoppo + δ • S: Aspect-Dependent • Sentiment Lexicon • A: Similar-Sentiment Matrix • (from synonyms and “and” clues) SCREEN-large SCREEN-big 1 SCREEN-bad SCREEN-terrible 1 BATTERY-small BATTERY-tiny 1 … … http://sifaka.cs.uiuc.edu/yuelu2/

  20. 4. opposite sentiment SCREEN-large SCREEN-small 1 SCREEN-excellent BATTERY-big 1 BATTERY-small BATTERY-big 1 … … S = argmin λprior S + λrating SCREEN-large S1 SCREEN-great S2 BATTERY-large S3 … … • B: Opposite-Sentiment Matrix (from antonyms and “but” clues) + λsim + λoppo + δ • S: Aspect-Dependent • Sentiment Lexicon Sign is different Abs Value is similar subject to Separate the representation of Sj: - Sign: only one of Sj+ , Sj- is active - Abs Value: value of the active one http://sifaka.cs.uiuc.edu/yuelu2/

  21. A Novel Optimization Framework General sentiment lexicon 1 S = argmin λprior S Overall rating 2 + λrating +δ 3 Synonyms “and” clues + λsim 4 + λoppo 3 • Weights set as the degree • we trust each signal + δ Antonyms “but” clues • Transform to linear programming • solved efficiently using GAMS/CPLEX 4 subject to http://sifaka.cs.uiuc.edu/yuelu2/

  22. Evaluation: Data Sets Evaluation (1): Lexicon Quality Evaluation (2): Doc-Aspect Sentiment, aggregate the sentiment of lexicon entries to doc level http://sifaka.cs.uiuc.edu/yuelu2/

  23. Evaluation (1): Lexicon QualityOPT > Global > Dictionary Hotel Data Guess 1,0,-1 uniformly 39% General dictionary only 27% Overall ratings only 15% [Lu et. al. WWW09] Our method with equal weights, i.e. (λprior:λrating:λsim:λoppo = 1:1:1:1) • Interesting sample results using OPT: • Hotel Data: ROOM-private, FOOD-excelent • Printer Data: INK-fast, SUPPORT-fast http://sifaka.cs.uiuc.edu/yuelu2/

  24. Tuning weights further improves performance OPT default: equal weights Dropping one term More weights on important terms http://sifaka.cs.uiuc.edu/yuelu2/

  25. Evaluation (2): Doc-Aspect Sentiment:OPT > Global > Dictionary Printer Data 144% 18% 33% 13% 1% 8% Hotel Data 17% 11% 8% 9% 2% 6% http://sifaka.cs.uiuc.edu/yuelu2/

  26. Roadmap • [WWW’10]: Exploiting Social Context for Review Quality Prediction • Opinion • Integration • Sentiment • Analysis • Quality Prediction [WWW’08] [COLING'10] [WWW’10] [WWW’09] [KDD’10] [WWW’11] Integrated Summary Sentence 512 Sentence 823 Sentence 21 Sentence 153 http://sifaka.cs.uiuc.edu/yuelu2/

  27. Existing Work of Quality Prediction • As a supervised learning problem Labeled × √ ? Not Helpful Very Helpful ? ? Unlabeled √ ? ? ? ? ? ? • Textual features • Meta-data features [Zhang&Varadarajan`06] [Kim et al. `06] [Liu et al. `08] [Ghose&Ipeirotis `10] http://sifaka.cs.uiuc.edu/yuelu2/

  28. Base model: Linear Regression Labels are expensive to obtain! i i Quality( ) w = argmin Textual Features w = argmin{ } w Labeled Closed-form: w= = Weights × FeatureVector( ) http://sifaka.cs.uiuc.edu/yuelu2/

  29. Our idea: social context can help! Quality( ) We also observe… Social Context + Reviewer Identity Social Network Quality( ) How to use them to help prediction? Intuitions: is related to Quality( ) is related to its Social Network http://sifaka.cs.uiuc.edu/yuelu2/

  30. Our approach: add social context as graph-based regularizers How to design the regularizers? Advantages: • Semi-supervised: make use of unlabeled data • Applicable to reviews without social context • Baseline • Loss function • Trade-off • parameter • Designed to “favor” • our intuitions w = argmin • { + β× Graph Regularizer} w Labeled Unlabeled http://sifaka.cs.uiuc.edu/yuelu2/

  31. Hypothesis 1: Reviewer Consistency Quality( ) 2 Quality( ) ~ 1 Quality( ) Quality( ) ~ 3 • 4 Reviewers are consistent! 2 1 3 4 http://sifaka.cs.uiuc.edu/yuelu2/

  32. Regularizer for Reviewer Consistency • Reviewer Regularizer 2 1 Same-Author Graph (A) Closed-form solution! =∑ [ Quality( ) - Quality( ) ]2 2 1 [Zhou et al. 03] [Zhu et al. 03] [Belkin et al 06] • w= 3 Review-Feature Matrix Graph Laplacian 4 http://sifaka.cs.uiuc.edu/yuelu2/

  33. Hypothesis 2: Trust Consistency Quality( ) - Quality( ) ≤ 0 I trust people with quality at least as good as mine! http://sifaka.cs.uiuc.edu/yuelu2/

  34. Regularizer for Trust Consistency • Trust Regularizer =∑max[0, Quality( ) - Quality( )]2 Trust Graph No closed-form solution… Still convexGradient Descent http://sifaka.cs.uiuc.edu/yuelu2/

  35. Hypothesis 3 &4 Hypothesis 4: LinkConsistency Hypothesis 3: Co-citationConsistency Trust Graph Co-citation Graph Link Graph http://sifaka.cs.uiuc.edu/yuelu2/

  36. Mathematical Formulations 1. Reviewer Consistency: 2. Trust Consistency: 3. Co-citation Consistency: 4. Link Consistency: Closed form Gradient descent Closed form Closed form http://sifaka.cs.uiuc.edu/yuelu2/

  37. Evaluation: Data Sets from Ciao UK http://sifaka.cs.uiuc.edu/yuelu2/

  38. Our methods are most effective with limited labeled data • Reg:Cocitation • Reg:Reviewer • Reg:Trust Reg:Link Baseline % of MSE Difference 10% 25% 50% 100% Percentage of labeled Data Better (Cellphone) http://sifaka.cs.uiuc.edu/yuelu2/

  39. Our methods are most effective with rich social context Cellphone Beauty Digital Camera Baseline % of MSE Difference Reviews/Reviewer ratio = 1.06 Reg:Link • Reg:Trust • Reg:Cocitation • Reg:Reviewer Better http://sifaka.cs.uiuc.edu/yuelu2/

  40. Summary of this talk … • Opinion • Integration • Sentiment • Analysis • Quality Prediction Integrated Summary Sentence 512 Sentence 823 Sentence 21 Sentence 153 http://sifaka.cs.uiuc.edu/yuelu2/

  41. Summary of this talk • Sentiment Analysis: construct aspect-dependent sentiment lexicon • Quality Prediction: exploit social context • Opinion • Integration • Sentiment • Analysis • Quality Prediction [WWW’08] [COLING'10] [WWW’10] [WWW’09] [KDD’10] [WWW’11] Integrated Summary Sentence 512 Sentence 823 Sentence 21 Sentence 153 http://sifaka.cs.uiuc.edu/yuelu2/

  42. Future Directions Task-support Applications Efficient Algo for Real-time Interaction 53M blogs 1307M posts 65M msgs/day 45M reviews 115Musers 10M groups Integrative Analysis http://sifaka.cs.uiuc.edu/yuelu2/

  43. Summary of my other work:Text Information Management “Investigation of Topic Models” [KDD 10] [IRJ 10] [COLING 10] Opinion Integration and Summarization Text Mining [WWW 08] “An open system for microarray clustering” [WWW 09] [NAR 07] [WWW 10] [WWW 11] Bioinformatics [NAR 10] “Bio literature mining” Information Retrieval [IRJ 09] • [TREC 07] “Bio literature IR” http://sifaka.cs.uiuc.edu/yuelu2/

  44. Thank you!&Questions?

  45. Backup Slides

  46. References [WWW'11]YueLu, MaluCastellanos, UmeshwarDayal, ChengXiangZhai. "Automatic Construction of a Context-Aware Sentiment Lexicon: An Optimization Approach", To Appear at WWW’11 [COLING'10] Yue Lu, Huizhong Duan, Hongning Wang and ChengXiang Zhai. "Exploiting Structured Ontology to Organize Scattered Online Opinions", In Proceedings of the 23rd International Conference on Computational Linguistics Pages: 734--742. [KDD’10]Hongning Wang, Yue Lu, and ChengXiangZhai. "Latent Aspect Rating Analysis on Review Text Data: A Rating Regression Approach", In Proceedings of the 16th ACM SIGKDD Conference on Knowledge Discovery and Data MiningPages: 783-792 [WWW'10]Yue Lu, Panayiotis Tsaparas, Alexandros Ntoulas, and Livia Polanyi. "Exploiting Social Context for Review Quality Prediction", In Proceedings of the 19th International World Wide Web Conference Pages: 691-700. [WWW'09]Yue Lu, ChengXiang Zhai and Neel Sundaresan. "Rated Aspect Summarization of Short Comments", In Proceedings of the 18th International World Wide Web Conference Pages: 131-140. [WWW'08]Yue Lu and ChengXiang Zhai. "Opinion Integration Through Semi-supervised Topic Modeling", In Proceedings of the 17th International World Wide Web Conference Pages: 121-130. http://sifaka.cs.uiuc.edu/yuelu2/

  47. Other Publications [IRJ’10] Yue Lu, Qiaozhu Mei, ChengXiangZhai. "Investigating Task Performance of Probabilistic Topic Models - An Empirical Study of PLSA and LDA", Information Retrieval. [NAR’10] X. He, Y. Li, R. Khetani, B. Sanders, YueLu, X. Ling, C.-X. Zhai, B. Schatz. “BSQA: Integrated Text Mining Using Entity Relation Semantics Extracted from Biological Literature of Insects", Nucleic Acids Research. [IRJ’09] Yue Lu, Hui Fang and ChengXiang Zhai. "An Empirical Study of Gene Synonym Query Expansion in Biomedical Information Retrieval", Information Retrieval Volume 12, Issue1 (2009), Pages: 51-68. [TREC'07] YueLu, Jing Jiang, Xu Ling, Xin He, ChengXiangZhai. "Language Models for Genomics Information Retrieval: UIUC at TREC 2007 Genomics Track", In Proceedings of the 16th Text REtrievalConference. [NAR’07] YueLu, Xin He and Sheng Zhong. “Cross-species microarray analysis with the OSCAR system suggests an INSR->Pax6->NQO1 neuro-protective pathway in ageing and Alzheimer's disease", Nucleic Acids Research 105-114 Topic models Bioinformatics Biomedical IR Biomedical IR Bioinformatics http://sifaka.cs.uiuc.edu/yuelu2/

  48. Generating Candidate Lexicon Entries Input: The LCD is great but battery is so large. Parsed: [The/DT LCD/NN is/VBZ great] but/CC [battery/NN is/VBZ so/RB large/JJ]./. Aspect Tagged: [The/DT (LCD/NN):SCREEN is/VBZ great/JJ] but/CC [(battery/NN):BATTERYis/VBZ so/RB large/JJ]./. SCREEN-large SCREEN-great BATTERY-large … ? SCREEN-great BATTERY-large Candidates: http://sifaka.cs.uiuc.edu/yuelu2/

  49. Hypotheses Testing (1):Reviewer Consistency From same reviewer From different reviewers Density Qg( ) Qg( ) 2 3 Qg( ) - 1 Qg( ) - 1 Difference in Review Quality Hypothesis 1: Reviewer Consistency is supported by data (Cellphone) http://sifaka.cs.uiuc.edu/yuelu2/

  50. Hypotheses Testing (2-4):Social Network-based Consistencies Qg( ) - Qg( ) B A Density B is not linked to A B trusts A B is co-cited with A B is linked to A Hypotheses 2-4: Social Network-based Consistencies supported by data Difference in Reviewer Quality (Cellphone) http://sifaka.cs.uiuc.edu/yuelu2/

More Related