Discovery of Manner Relations and their Applicability to Question Answering

1. Discovery of Manner Relations and their Applicability to QuestionAnswering Roxana Girju1,2, Manju Putcha1, and Dan Moldovan1 University of Texas at Dallas1 Baylor University2 ACL Workshop on Multilingual Summarization and Question Answering, 2003

2. Manner Relation • An example: • We want to worktogether to build our new economy, creating jobsby investing in technology so America can continue to lead the worldin growth and opportunity. Q: How do Democrats want to build the economy? A: by creating jobs Q: How do Democrats want to create jobs? A: by investing in technology

3. Manner Relation (Cont.) • In WordNet, the manner relation is defined as a way of acting or behaving. • There are different ways of expressing manner. • A possible way to check: • answer to “In what manner/how <to verb>?” • “He runs quickly.” vs. “How hoes he run?” • but not to “Where <verb >?”, or “When <verb>?” • “He runs on the field.”, “He runs quite often.”

4. Lexico-Syntactic Patterns Expressing Manner • The most frequently occurring form of manner is as a semantic role (Quirk et al., 1985). • Manner is encoded as a relationship between a verb and one of its arguments. • One of the most frequently used patterns expressing manner is verb-adverb. • Adverbs can be further classified as follows:

5. Manner Expressing (Cont.) • Verb-Adverb Patterns • Adverbs of manner that end in “-ly” • Adverbs of manner that do not end in “-ly” • Quality description adverbs: fast, good, well • Adverbial expressions • as much as 60% faster, louder than ever • Compound adverbs of manner • radio-style, tax-free • Foreign adverbial expressions • a la Gorbachev, en masse

6. Manner Expressing (Cont.) • Other forms of manner relations (1) complex nominals (fast car), (2) verbs of implicit manner (for example whisperis a manner of speaking), (3) verb-PP (I took your coat by mistake) (4) verb-NP (He breathed a deep breath) (5) verb clauses (I cook vegetables as Chinese do)

7. Manner Expressing (Cont.) • All these lexico-syntactic patterns are ambiguous. • In this paper we focus only on the discovery of manner semantic roles expressed as verb-adverb pairs. • The method, however, is extendable to many other manner forms and even to other semantic relations.

8. Approach • Supervised learning • Naive Bayes Classifier approach • Adverb Dictionary (as reference) • Adverbs in WordNet with a gloss like “in a ~~ manner” • Adverbs that are annotated in TreeBank as MNR adverb-verb pairs

9. Features

10. Features (Cont.) (1) Specific adverb statistics • Whether the adverb is in Adverb Dictionary or not. Statistics is the occurrences in training data. (2) Parent phrase type • The syntactic tag of the parent of the adverb (3) Whether or not Adverb is present in the Dictionary • The same as (1), but the statistics is the probability of being a manner adverb in the Adverb Dictionary.

11. Features (Cont.) (4) Distance between verb and adverb • Counted as words between them (5) Component before the adverb • The syntactic roles before the adverb. (6) Component after the adverb • The syntactic roles after the adverb. (7) Adverb ends in “ly” • True or false

12. Estimating Probabilities • Class prior probabilities: • Portion of positive (Vj=1) or negative (Vj=0) examples in the training data • Class conditional probability (of each feature): • pos_freq: occurrences of the feature in positive class • VOCAB, TEXT: the distinct and total numbers of instances for the feature

13. Naive Bayes Classifier • Vnb is the output of the Naive Bayes Classifier, • vj is the class in the target set V={+,-} • fi are the individual features from the set F of the seven features. • P(fi /+)=Prob(+) for fi • P(fi /-)=Prob(-) for fi • The classifier will generate a Look-Up table for classification use.

14. Training and Test Corpus • Positive examples: ADVP-MNR annotations from Treebank2 (UPenn’s) • Training:Testing=3:1

15. First Experiment • Training = (1176 Positive + 2546 Negative) = 3722 examples • Testing = (507 Positive + 1183 Negative) =1690 examples. • Output of the program: • Prior Positive Probability = 0.315959162 • Prior Negative Probability = 0.684040838 • Precision = 191/242 = 78.92% • Recall = 191/507 = 37.62%

16. Second Experiment • Some adverbs which always indicate in one class were removed. • Negative: moreover, then, thus • Positive: much, very, so

17. Application to Question Answering • For questions tagged as MNR • First locate the paragraph where the potential answer is in, • Then identify the MNR tag in that paragraph. Q: How did Bob Marley die? A: Bob Marley died [of Melanoma][MNR]. Q: How was little Johnny dressed last night? A: Dressed [in a cowboy style][MNR] , Johnny walked proudly on the street.

18. Conclusion • The method presented in this paper for the detection and validation of manner relations is automatic and novel. • Main flaw: • Naive Bayes Classifier assumes feature independence • But most features here are dependent on each other (except 1 and 4).