The Role of Implicit Argumentation in Nominal SRL

1. The Role of Implicit Argumentation in Nominal SRL Matt Gerber* Language & Interaction Research Joyce Y. Chai* Adam Meyers+ *Department of Computer Science and Engineering Michigan State University +Computer Science Department New York University

2. Semantic role labeling (SRL) • Verbal • Suntory distributes Brown-Forman bourbons in Japan. • Nominal • Searle will give pharmacists brochures on the use of prescription drugs for distribution in their stores. • General approach (Gildea and Jurafsky, 2002) • Supervised machine learning • Features extracted from a full syntactic analysis [Agent Suntory] [Predicate distributes] [Theme Brown-Forman bourbons] [Location in Japan]. Searle will give [Agent pharmacists] [Theme brochures on the use of prescription drugs] for [Predicate distribution] [Location in their stores].

3. Nominal SRL via NomBank (Meyers, 2007) • Lexicon • ~4700 distinct predicates • Example • distribution: send a package • Corpus annotation • Penn TreeBank • ~115k manual SRL analyses • Searle will give pharmacists brochures on the use of prescription drugs for distribution in their stores. • Source: distribute • Arg0: distributor • Arg1: thing distributed • Arg2: recipient Searle will give [Arg0 pharmacists] [Arg1 brochures on the use of prescription drugs] for [Predicate distribution] [Location in their stores].

4. Related work • Liu and Ng (2007) • Best reported argument F1: 0.7283 • Based on predicates annotated in NomBank • CoNLL 2008 Shared Task (Surdeanu, 2008) • Joint parsing of syntactic and semantic dependencies • Automatic corpus construction • Syntax: constituent to dependency • Semantics: PropBank/NomBank • Identify predicates and arguments

5. Implicit argumentation • Not all nominal predicates have overt arguments • The distribution represents available cash flow from the partnership between Aug. 1 and Oct. 31. • Implicit argument • Understood by reader • Not annotated in NomBank and not evaluated • Open questions 1. How does implicit argumentation affect nominal SRL performance? 2. How much can we improve SRL performance by taking implicit argumentation into account? The [Predicate distribution] represents available cash flow from the partnership between Aug. 1 and Oct. 31.

6. Outline • Implicit argumentation in nominal SRL • A two-stage approach • Conclusions and future work

7. Outline • Implicit argumentation in nominal SRL • Baseline nominal SRL system • Prevalence and impact of implicit argumentation • A two-stage approach • Conclusions and future work

8. A baseline nominal SRL system • Extended feature set • Variations on the parse tree path • Single-stage logistic regression argument prediction • Independent model for incorporated arguments • Petrolane is the second-largest distributor in the U.S. • Results using ground-truth predicates • Previous best argument F1: 0.7283 (Liu and Ng, 2007) • Our best argument F1: 0.7630 • Strong baseline for investigation Petrolane is the second-largest [Arg0/Predicate distributor] [Location in the U.S.]

9. Prevalence of implicit argumentation • In a practical setting, we don’t have ground-truth predicates • Not all predicates have overt arguments • Overall nominal markability: 57%

10. Impact of implicit argumentation • Extended evaluation • Process each token in test data • Attempt SRL for known nouns • Argument identification results

11. Impact of implicit argumentation • Correct output • Canadian investment in the US has declined. • Compare to • Canadian investment rules require that big foreign takeovers meet that standard. [Arg0 Canadian] [Predicate investment] [Arg2 in the US] has declined. [Arg0 Canadian] [Predicate investment] rules require that big foreign takeovers meet that standard. Not in an argument-bearing environment

12. Outline • Implicit argumentation in nominal SRL • A two-stage approach • Filter out nominals whose arguments are implicit • Identify arguments for remaining nominals • Conclusions and future work

13. Two-stage implicit argumentation model • Stage 1: predicate classification • Binary decision • Top-ranked feature: ancestor grammar rules • Captures syntactic environment of candidate predicate S : AGR3 = S → N, VP N (John) : AGR2 = VP → V, NP VP V (made) : AGR1 = NP → Det, N NP Det (a) N (sale)

14. Stage 1: predicate classification • Predicate classification features (sorted by gain) • AGR • Stem • Category of right sibling • “investment [N rules]” versus “investment [PP in the US]” • Parse tree paths to support verbs • Last word of left sibling • … • Logistic regression • LibLinear (Fan, 2008)

15. Predicate classification results • Baseline systems • Naïve: all known nouns bear arguments • MLE: similar to naïve model, but uses MLE scoring • Results • Naïve recall < 1.0 • POS errors • Unknown predicates • LR • Balanced P/R near 0.9

16. Predicate classification results by markability • Significant gains for rarely markable nominals • Gains diminish as nominals become reliably markable

17. NomLex-PLUS • Semi-automatic extension of NomLex • Nominal complement structure • Most frequent classes • Nom (46%): distribution (from distribute) • Partitive (13%): percent • Nom-like (12%): breather (from pause) • … • Not all nominals denote events

18. Nom: +10 F1 points Partitive: +30 F1 points Nom-like: +20 F1 points Predicate classification results by NomLex class • Gains vary widely across classes • Large gains for most frequent classes

19. Predicate-argument classification results • Accounting for implicit argumentation improves results • Within 0.005 of the previous best score using ground-truth predicates

20. Predicate-argument classification results by markability • Significant gains for rarely markable nominals • Gains diminish as nominals become reliably markable

21. Predicate-argument classification results by NomLex class • Nominals with unambiguous NomLex class • Event nominals appear to be significantly more difficult • Accounting for implicit argumentation helps all classes

22. Outline • Implicit argumentation in nominal SRL • A two-stage approach • Conclusions and future work

23. Conclusions and future work • Implicit argumentation poses a serious problem for nominal SRL • Performance loss of nearly 10% • We can effectively account for implicit argumentation in nominal SRL • Predicate F1: 0.90 • Argument F1: 0.72 • Implicit argumentation affects some classes of nominals more than others • Future work: identify implicit arguments in surrounding discourse