1 / 47

Recognizing Contextual Polarity in Phrase-Level Sentiment Analysis

Recognizing Contextual Polarity in Phrase-Level Sentiment Analysis. Theresa Wilson Janyce Wiebe Paul Hoffmann University of Pittsburgh. Introduction. Sentiment analysis task of identifying positive and negative opinions, emotions, and evaluations

turner
Download Presentation

Recognizing Contextual Polarity in Phrase-Level Sentiment Analysis

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Recognizing Contextual Polarity in Phrase-Level Sentiment Analysis Theresa Wilson Janyce Wiebe Paul Hoffmann University of Pittsburgh

  2. Introduction Sentiment analysis task of identifying positive and negative opinions, emotions, and evaluations • How detailed? Depends on the application. • Flame detection, review classification  document-level analysis • Question answering, review mining  sentence or phrase-level analysis HLT-EMNLP 2005

  3. Question Answering Example Q: What is the international reaction to the reelection of Robert Mugabe as President of Zimbabwe? African observers generallyapproved of his victory while Western Governments denounced it. HLT-EMNLP 2005

  4. Prior Polarity versus Contextual Polarity • Most approaches use a lexicon of positive and negative words Prior polarity: out of context, positive or negative beautiful positive horrid  negative • A word may appear in a phrase that expresses a different polarity in context Contextual polarity “Cheers to Timothy Whitfield for the wonderfullyhorrid visuals.” HLT-EMNLP 2005

  5. Example Philip Clap, President of the National Environment Trust, sums up well the general thrust of the reaction of environmental movements: there is no reason at all to believe that the polluters are suddenly going to become reasonable. HLT-EMNLP 2005

  6. Example Philip Clap, President of the National Environment Trust, sums up well the general thrust of the reaction of environmental movements: there is no reason at all to believe that the polluters are suddenly going to become reasonable. HLT-EMNLP 2005

  7. Example Philip Clap, President of the National Environment Trust, sums up well the general thrust of the reaction of environmental movements: there is no reason at all to believe that the polluters are suddenly going to become reasonable. Contextual polarity prior polarity HLT-EMNLP 2005

  8. Goal of Our Research • Automatically distinguish prior and contextual polarity HLT-EMNLP 2005

  9. Lexicon Step 1 Step 2 All Instances Polar Instances Neutral or Polar? Contextual Polarity? Corpus Approach • Use machine learning and variety of features • Achieve significant results for a large subset of sentiment expressions HLT-EMNLP 2005

  10. Outline • Introduction • Manual Annotations • Corpus • Prior-Polarity Subjectivity Lexicon • Experiments • Previous Work • Conclusions HLT-EMNLP 2005

  11. Manual Annotations Need: sentiment expressions with contextual polarity  positive and negative expressions of emotions, evaluations, stances Had: subjective expression annotations in MPQA Opinion Corpus http://nrrc.mitre.org/NRRC/publications.htm words/phrases expressing emotions, evaluations, stances, speculations, etc. sentiment expressions  subjective expressions Decision: annotate subjective expressions in MPQA Corpus with their contextual polarity HLT-EMNLP 2005

  12. Annotation Scheme • Mark polarity of subjective expressions as positive, negative, both, or neutral positive African observers generally approved of his victory while Western governments denounced it. negative Besides, politicians refer to good and evil … both Jerome says the hospital feels no different than a hospital in the states. neutral HLT-EMNLP 2005

  13. Annotation Scheme • Judge the contextual polarity of sentiment ultimately being conveyed They have not succeeded, and will never succeed, in breaking the will of this valiant people. positive HLT-EMNLP 2005

  14. Agreement Study • 10 documents with 447 subjective expressions • Kappa: 0.72 (82%) • Remove uncertain cases  at least one annotator marked uncertain (18%) • Kappa: 0.84 (90%) (But all data included in experiments) HLT-EMNLP 2005

  15. Lexicon Step 1 Step 2 All Instances Polar Instances Neutral or Polar? Contextual Polarity? Corpus Outline • Introduction • Manual Annotations • Corpus • Prior-Polarity Subjectivity Lexicon • Experiments • Previous Work • Conclusions HLT-EMNLP 2005

  16. Corpus • 425 documents from MPQA Opinion Corpus • 15,991 subjective expressions in 8,984 sentences • Divided into two sets • Development set • 66 docs / 2,808 subjective expressions • Experiment set • 359 docs / 13,183 subjective expressions • Divided into 10 folds for cross-validation HLT-EMNLP 2005

  17. Lexicon Step 1 Step 2 All Instances Polar Instances Neutral or Polar? Contextual Polarity? Corpus Outline • Introduction • Manual Annotations • Corpus • Prior-Polarity Subjectivity Lexicon • Experiments • Previous Work • Conclusions HLT-EMNLP 2005

  18. Prior-Polarity Subjectivity Lexicon • Over 8,000 words from a variety of sources • Both manually and automatically identified • Positive/negative words from General Inquirer and Hatzivassiloglou and McKeown (1997) • All words in lexicon tagged with: • Prior polarity: positive, negative, both, neutral • Reliability: strongly subjective (strongsubj), weakly subjective (weaksubj) HLT-EMNLP 2005

  19. Outline • Introduction • Manual Annotations • Corpus • Prior-Polarity Subjectivity Lexicon • Experiments • Previous Work • Conclusions HLT-EMNLP 2005

  20. Lexicon Step 1 Step 2 All Instances Polar Instances Neutral or Polar? Contextual Polarity? Corpus Experiments • Give each instance its own label Both Steps: • BoosTexter AdaBoost.HM 5000 rounds boosting • 10-fold cross validation HLT-EMNLP 2005

  21. Definition of Gold Standard Given an instance inst from the lexicon: if inst not in a subjective expression: goldclass(inst) = neutral else if inst in at least one positive and one negative subjective expression: goldclass(inst) = both else if inst in a mixture of negative and neutral: goldclass(inst) = negative else if inst in a mixture of positive and neutral: goldclass(inst) = positive else: goldclass(inst) = contextual polarity of subjective expression HLT-EMNLP 2005

  22. Features • Many inspired by Polanya & Zaenen (2004): Contextual Valence Shifters Example: little threat little truth • Others capture dependency relationships between words Example: wonderfullyhorrid pos mod HLT-EMNLP 2005

  23. Lexicon Step 1 Step 2 All Instances Polar Instances Neutral or Polar? Contextual Polarity? Corpus • Word features • Modification features • Structure features • Sentence features • Document feature HLT-EMNLP 2005

  24. Word features Modification features Structure features Sentence features Document feature Word token terrifies Word part-of-speech VB Context that terrifies me Prior Polarity negative Reliability strongsubj Lexicon Step 1 Step 2 All Instances Polar Instances Neutral or Polar? Contextual Polarity? Corpus HLT-EMNLP 2005

  25. Word features Modification features Structure features Sentence features Document feature Binary features: Preceded by adjective adverb (other than not) intensifier Self intensifier Modifies strongsubj clue weaksubj clue Modified by strongsubj clue weaksubj clue Lexicon Step 1 Step 2 All Instances Polar Instances Neutral or Polar? Contextual Polarity? Corpus poses subj obj report challenge det det p mod adj adj The human rights a substantial … Dependency Parse Tree HLT-EMNLP 2005

  26. Word features Modification features Structure features Sentence features Document feature Binary features: In subject The humanrights report poses In copular I am confident In passive voice must be regarded Lexicon Step 1 Step 2 All Instances Polar Instances Neutral or Polar? Contextual Polarity? Corpus poses subj obj report challenge det det p mod adj adj The human rights a substantial … HLT-EMNLP 2005

  27. Word features Modification features Structure features Sentence features Document feature Count of strongsubj clues in previous, current, next sentence Count of weaksubj clues in previous, current, next sentence Counts of various parts of speech Lexicon Step 1 Step 2 All Instances Polar Instances Neutral or Polar? Contextual Polarity? Corpus HLT-EMNLP 2005

  28. Word features Modification features Structure features Sentence features Document feature Document topic (15) economics health Kyoto protocol presidential election in Zimbabwe Lexicon Step 1 Step 2 All Instances Polar Instances Neutral or Polar? Contextual Polarity? Corpus … Example: The disease can be contracted if a person is bitten by a certain tick or if a person comes into contact with the blood of a congo feversufferer. HLT-EMNLP 2005

  29. Lexicon Step 1 Step 2 All Instances Polar Instances Neutral or Polar? Contextual Polarity? Corpus Results 1a HLT-EMNLP 2005

  30. Lexicon Step 1 Step 2 All Instances Polar Instances Neutral or Polar? Contextual Polarity? Corpus Results 1b HLT-EMNLP 2005

  31. Lexicon Step 1 Step 2 All Instances Polar Instances Neutral or Polar? Contextual Polarity? Corpus Step 2: Polarity Classification Classes • positive, negative, both, neutral 19,506 5,671 HLT-EMNLP 2005

  32. Lexicon Step 1 Step 2 All Instances Polar Instances Neutral or Polar? Contextual Polarity? Corpus • Word token • Word prior polarity • Negated • Negated subject • Modifies polarity • Modified by polarity • Conjunction polarity • General polarity shifter • Negative polarity shifter • Positive polarity shifter HLT-EMNLP 2005

  33. Word token Word prior polarity Negated Negated subject Modifies polarity Modified by polarity Conjunction polarity General polarity shifter Negative polarity shifter Positive polarity shifter Word token terrifies Word prior polarity negative Lexicon Step 1 Step 2 All Instances Polar Instances Neutral or Polar? Contextual Polarity? Corpus HLT-EMNLP 2005

  34. Word token Word prior polarity Negated Negated subject Modifies polarity Modified by polarity Conjunction polarity General polarity shifter Negative polarity shifter Positive polarity shifter Binary features: Negated For example: not good does not look very good not onlygood but amazing Negated subject Nopolitically prudent Israelicould support either of them. Lexicon Step 1 Step 2 All Instances Polar Instances Neutral or Polar? Contextual Polarity? Corpus HLT-EMNLP 2005

  35. Word token Word prior polarity Negated Negated subject Modifies polarity Modified by polarity Conjunction polarity General polarity shifter Negative polarity shifter Positive polarity shifter Modifies polarity 5 values: positive, negative, neutral, both, not mod substantial: negative Modified by polarity 5 values: positive, negative, neutral, both, not mod challenge: positive Lexicon Step 1 Step 2 All Instances Polar Instances Neutral or Polar? Contextual Polarity? Corpus substantial(pos)challenge(neg) HLT-EMNLP 2005

  36. Word token Word prior polarity Negated Negated subject Modifies polarity Modified by polarity Conjunction polarity General polarity shifter Negative polarity shifter Positive polarity shifter Conjunction polarity 5 values: positive, negative, neutral, both, not mod good: negative Lexicon Step 1 Step 2 All Instances Polar Instances Neutral or Polar? Contextual Polarity? Corpus good(pos) and evil(neg) HLT-EMNLP 2005

  37. Word token Word prior polarity Negated Negated subject Modifies polarity Modified by polarity Conjunction polarity General polarity shifter Negative polarity shifter Positive polarity shifter General polarity shifter pose littlethreat contains little truth Negative polarity shifter lack of understanding Positive polarity shifter abate the damage Lexicon Step 1 Step 2 All Instances Polar Instances Neutral or Polar? Contextual Polarity? Corpus HLT-EMNLP 2005

  38. Lexicon Step 1 Step 2 All Instances Polar Instances Neutral or Polar? Contextual Polarity? Corpus Results 2a HLT-EMNLP 2005

  39. Lexicon Step 1 Step 2 All Instances Polar Instances Neutral or Polar? Contextual Polarity? Corpus Results 2b HLT-EMNLP 2005

  40. Lexicon Step 1 Step 2 All Instances Polar Instances Neutral or Polar? Contextual Polarity? Corpus Ablation experiments removing features: • Negated, negated subject • Modifies polarity, modified by polarity • Conjunction polarity • General, negative, positive polarity shifters HLT-EMNLP 2005

  41. Outline • Introduction • Manual Annotations • Corpus • Prior-Polarity Subjectivity Lexicon • Experiments • Previous Work • Conclusions HLT-EMNLP 2005

  42. Previous Work • Learn prior polarity of words and phrases e.g., Hatzivassiloglou & McKeown (1997), Turney (2002) • Sentence-level sentiment analysis e.g., Yu & Hatzivassiloglou (2003), Kim & Hovy (2004) • Phrase-level contextual polarity classification e.g., Yi et al. (2003) HLT-EMNLP 2005

  43. At HLT/EMNLP 2005 • Popescu & Etizioni: Extracting Product Features and Opinions from Reviews • Choi, Cardie, Riloff & Patwardhan: Identifying Sources of Opinions with Conditional Random Fields and Extraction Patterns • Alm, Roth & Sproat: Emotions from Text: Machine Learning for Text-based Emotion Prediction HLT-EMNLP 2005

  44. Outline • Introduction • Manual Annotations • Corpus • Prior-Polarity Subjectivity Lexicon • Experiments • Previous Work • Conclusions HLT-EMNLP 2005

  45. Conclusions • Presented a two-step approach to phrase-level sentiment analysis • Determine if an expression is neutral or polar • Determines contextual polarity of the ones that are polar • Automatically identify the contextual polarity of a large subset of sentiment expression HLT-EMNLP 2005

  46. Thank you HLT-EMNLP 2005

  47. Acknowledgments This work was supported by • Advanced Research and Development Activity (ARDA) • National Science Foundation HLT-EMNLP 2005

More Related