Multiple Aspect Ranking using the Good Grief Algorithm

1. Multiple Aspect Ranking using the Good Grief Algorithm • Benjamin Snyder and Regina Barzilay at MIT • 4.1.2010 Elizabeth Kierstead

2. Introduction • Want to create a system that accounts for many aspects of a user’s satisfaction, and use agreement across aspects to better model their reviews • Sentiment analysis started out as a binary classification task (“Good Restaurant”/ “Bad Restaurant”) (Pang et al. 2002) • (Pang and Lee, 2005) expanded this to account for polarity and a multipoint scale for modeling sentiments • Other work (Crammer and Singer 2001) allowed for ranking multiple aspects of a review, but only addressed the aspects independently, failing to capture important relations across aspects

3. Example: Restaurant review may rate food, decor and value, and if a user says that “The food was good but the value was better” independently ranking aspects fails to exploit dependencies across aspects, and key information is lost • The authors’ algorithm uses the Agreement Relation to model dependencies across aspects • Good Grief algorithm predicts a set of ranks (one for each aspect) to minimize the difference between the individual rankers and the agreement model • Their method uses the Good Grief decoder to predict a set of ranks based on both agreement and individual ranking models, and they find that their joint model significantly outperforms individual ranking models

4. The Algorithm • m-aspect ranking model with m + 1 components: (<w[1], b[1]>, ... , <w[m], b[m]>, a) - First m components are the individual ranking models, one per aspect, and the final vector is the agreement model • w[i]: a vector of weights on the input features for the ith aspect b[i]: a vector of boundaries dividing the real line into k intervals, corresponding to k ranks of the aspect • default ranking using PRank (Crammer and Singer 2001), which performs rank predictions for individual aspects of a review • agreement model- vector of weights a- If all m aspects are equal, a x > 0, otherwise a x < 0

5. | a x| indicates the confidence of the agreement prediction • The authors use a joint prediction criterion that simultaneously takes into account all model components, assessing the level of grief associated with ith aspect ranking model g_i(x, r[i]) and the grief of the agreement model g_a(x, r) • Decoder picks the ranks that minimize overall grief

6. Feature Representation • Ranking Model- following previous work on sentiment classification (Pang et al., 2002), they extract unigrams and bigrams, discarding those that occur less than three times (30,000 features extracted) • Agreement Model- also use lexicalized features like unigrams and bigrams, but introduce a new feature to quantify the contrastive distance between a pair of words • Ex: “delicious” and “dirty” would have high contrast, while “expensive” and “slow” would have low contrast

7. Results • The Good Grief algorithm can rank a training set perfectly if the independent ranking models can do so • The Good Grief algorithm can also perfectly rank some training sets that the independent ranking models could not rank, because of the benefits of using the agreement model • Ex: The food was good, but not the ambience. The food was good, and so was the ambience. The food was bad, but not the ambience. The food was bad, and so was the ambience.

8. Results

9. Results