1 / 23

The Role of Implicit Argumentation in Nominal SRL

The Role of Implicit Argumentation in Nominal SRL. Matt Gerber*. Language & Interaction Research. Joyce Y. Chai*. Adam Meyers +. *Department of Computer Science and Engineering Michigan State University. + Computer Science Department New York University. Semantic role labeling (SRL).

brit
Download Presentation

The Role of Implicit Argumentation in Nominal SRL

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. The Role of Implicit Argumentation in Nominal SRL Matt Gerber* Language & Interaction Research Joyce Y. Chai* Adam Meyers+ *Department of Computer Science and Engineering Michigan State University +Computer Science Department New York University

  2. Semantic role labeling (SRL) • Verbal • Suntory distributes Brown-Forman bourbons in Japan. • Nominal • Searle will give pharmacists brochures on the use of prescription drugs for distribution in their stores. • General approach (Gildea and Jurafsky, 2002) • Supervised machine learning • Features extracted from a full syntactic analysis [Agent Suntory] [Predicate distributes] [Theme Brown-Forman bourbons] [Location in Japan]. Searle will give [Agent pharmacists] [Theme brochures on the use of prescription drugs] for [Predicate distribution] [Location in their stores].

  3. Nominal SRL via NomBank (Meyers, 2007) • Lexicon • ~4700 distinct predicates • Example • distribution: send a package • Corpus annotation • Penn TreeBank • ~115k manual SRL analyses • Searle will give pharmacists brochures on the use of prescription drugs for distribution in their stores. • Source: distribute • Arg0: distributor • Arg1: thing distributed • Arg2: recipient Searle will give [Arg0 pharmacists] [Arg1 brochures on the use of prescription drugs] for [Predicate distribution] [Location in their stores].

  4. Related work • Liu and Ng (2007) • Best reported argument F1: 0.7283 • Based on predicates annotated in NomBank • CoNLL 2008 Shared Task (Surdeanu, 2008) • Joint parsing of syntactic and semantic dependencies • Automatic corpus construction • Syntax: constituent to dependency • Semantics: PropBank/NomBank • Identify predicates and arguments

  5. Implicit argumentation • Not all nominal predicates have overt arguments • The distribution represents available cash flow from the partnership between Aug. 1 and Oct. 31. • Implicit argument • Understood by reader • Not annotated in NomBank and not evaluated • Open questions 1. How does implicit argumentation affect nominal SRL performance? 2. How much can we improve SRL performance by taking implicit argumentation into account? The [Predicate distribution] represents available cash flow from the partnership between Aug. 1 and Oct. 31.

  6. Outline • Implicit argumentation in nominal SRL • A two-stage approach • Conclusions and future work

  7. Outline • Implicit argumentation in nominal SRL • Baseline nominal SRL system • Prevalence and impact of implicit argumentation • A two-stage approach • Conclusions and future work

  8. A baseline nominal SRL system • Extended feature set • Variations on the parse tree path • Single-stage logistic regression argument prediction • Independent model for incorporated arguments • Petrolane is the second-largest distributor in the U.S. • Results using ground-truth predicates • Previous best argument F1: 0.7283 (Liu and Ng, 2007) • Our best argument F1: 0.7630 • Strong baseline for investigation Petrolane is the second-largest [Arg0/Predicate distributor] [Location in the U.S.]

  9. Prevalence of implicit argumentation • In a practical setting, we don’t have ground-truth predicates • Not all predicates have overt arguments • Overall nominal markability: 57%

  10. Impact of implicit argumentation • Extended evaluation • Process each token in test data • Attempt SRL for known nouns • Argument identification results

  11. Impact of implicit argumentation • Correct output • Canadian investment in the US has declined. • Compare to • Canadian investment rules require that big foreign takeovers meet that standard. [Arg0 Canadian] [Predicate investment] [Arg2 in the US] has declined. [Arg0 Canadian] [Predicate investment] rules require that big foreign takeovers meet that standard. Not in an argument-bearing environment

  12. Outline • Implicit argumentation in nominal SRL • A two-stage approach • Filter out nominals whose arguments are implicit • Identify arguments for remaining nominals • Conclusions and future work

  13. Two-stage implicit argumentation model • Stage 1: predicate classification • Binary decision • Top-ranked feature: ancestor grammar rules • Captures syntactic environment of candidate predicate S : AGR3 = S → N, VP N (John) : AGR2 = VP → V, NP VP V (made) : AGR1 = NP → Det, N NP Det (a) N (sale)

  14. Stage 1: predicate classification • Predicate classification features (sorted by gain) • AGR • Stem • Category of right sibling • “investment [N rules]” versus “investment [PP in the US]” • Parse tree paths to support verbs • Last word of left sibling • … • Logistic regression • LibLinear (Fan, 2008)

  15. Predicate classification results • Baseline systems • Naïve: all known nouns bear arguments • MLE: similar to naïve model, but uses MLE scoring • Results • Naïve recall < 1.0 • POS errors • Unknown predicates • LR • Balanced P/R near 0.9

  16. Predicate classification results by markability • Significant gains for rarely markable nominals • Gains diminish as nominals become reliably markable

  17. NomLex-PLUS • Semi-automatic extension of NomLex • Nominal complement structure • Most frequent classes • Nom (46%): distribution (from distribute) • Partitive (13%): percent • Nom-like (12%): breather (from pause) • … • Not all nominals denote events

  18. Nom: +10 F1 points Partitive: +30 F1 points Nom-like: +20 F1 points Predicate classification results by NomLex class • Gains vary widely across classes • Large gains for most frequent classes

  19. Predicate-argument classification results • Accounting for implicit argumentation improves results • Within 0.005 of the previous best score using ground-truth predicates

  20. Predicate-argument classification results by markability • Significant gains for rarely markable nominals • Gains diminish as nominals become reliably markable

  21. Predicate-argument classification results by NomLex class • Nominals with unambiguous NomLex class • Event nominals appear to be significantly more difficult • Accounting for implicit argumentation helps all classes

  22. Outline • Implicit argumentation in nominal SRL • A two-stage approach • Conclusions and future work

  23. Conclusions and future work • Implicit argumentation poses a serious problem for nominal SRL • Performance loss of nearly 10% • We can effectively account for implicit argumentation in nominal SRL • Predicate F1: 0.90 • Argument F1: 0.72 • Implicit argumentation affects some classes of nominals more than others • Future work: identify implicit arguments in surrounding discourse

More Related