1 / 21

A Structured Model for Joint Learning of Argument Roles and Predicate Senses

A Structured Model for Joint Learning of Argument Roles and Predicate Senses. Yotaro Watanabe Masayuki Asahara Yuji Matsumoto. Tohoku University Nara Institute of Science and Technology. ACL 2010 Uppsala, Sweden July 12, 2010.

abner
Download Presentation

A Structured Model for Joint Learning of Argument Roles and Predicate Senses

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. A Structured Model for Joint Learning of Argument Roles and Predicate Senses Yotaro Watanabe Masayuki Asahara Yuji Matsumoto Tohoku University Nara Institute of Science and Technology ACL 2010 Uppsala, Sweden July 12, 2010

  2. Predicate-Argument Structure Analysis(Semantic Role Labeling) • Task of analyzing predicates and its arguments • A predicate represents a state or an event, and its arguments have relations to the predicate • Each of arguments has a particular semantic role (Agent, Theme, etc) • In recent years, predicate sense disambiguation has been included in predicate-argument structure analysis [Surdeanu+ 08, Hajič+ 09] • ‘sell.01’ means that ‘sold’ is an instance of the first sense of ‘sell’ • Important for many NLP applications • MT, QA, RTE, etc. The luxury auto maker last year sold 1,214 cars in the U.S. Temporal Product Agent Theme Agent Location maker.01 sell.01

  3. Two Types of Dependencies of Elements in Predicate-Argument Structures • Inter-dependencies between a predicate and its arguments • A1: car => we can infer that the correct sense is drive.01 • Non-local dependencies among arguments • Two or more arguments do not have the same role • Basically, obligatory roles of the predicate should appear in sentences OBJ drive.01: drive a vehicle A0: driver A1: vehicle drive.02: cause to move A0: driver A1: things in motion NMOD SBJ car Paul drove his A0 A1 drive.01 In order to realize robust predicate-argument structure analysis, it is necessary to deal with these types of dependencies

  4. Previous Work • Non-local dependencies among arguments: Re-ranking [Johansson and Nugues 2008, etc.] • Generate N-best assignments of argument roles, then obtain global features for each assignment, finally select the argmax using the re-ranker • Can not explicitly capture inter-dependencies between a predicate and its arguments • Inter-dependencies between a predicate and its arguments: Markov Logic Networks [Meza-Ruiz and Riedel 2009, etc.] • Jointly learn and classify pred. senses and arg. roles simultaneously • MLN can not deal with particular types of global features Currently, no existing (discriminative) approach sufficiently handles both types of dependencies

  5. Previous Work • Non-local dependencies among arguments: Re-ranking [Johansson and Nugues 2008, etc.] • Generate N-best assignments of argument roles, then obtain global features for each assignment, finally select the argmax using the re-ranker • Can not explicitly capture inter-dependencies between a predicate and its arguments • Inter-dependencies between a predicate and its arguments: Markov Logic Networks [Meza-Ruiz and Riedel 2009, etc.] • Jointly learn and classify pred. senses and arg. roles simultaneously • MLN can not deal with particular types of global features Currently, no existing (discriminative) approach sufficiently handles both types of dependencies We propose a structured model that can capture both types of dependencies simultaneously

  6. The proposed model OBJ drive.01: drive a vehicle A0: driver A1: vehicle drive.02: cause to move A0: driver A1: things in motion NMOD SBJ car Paul drove his

  7. The proposed model OBJ drive.01: drive a vehicle A0: driver A1: vehicle drive.02: cause to move A0: driver A1: things in motion NMOD SBJ car Paul drove his Expand the possible labels of predicate senses and argument roles drove drive.01 drive.02 A0 A1 A0 A1 … NONE NONE … Paul car

  8. The proposed model OBJ drive.01: drive a vehicle A0: driver A1: vehicle drive.02: cause to move A0: driver A1: things in motion NMOD SBJ car Paul drove his Expand the possible labels of predicate senses and argument roles drove We use four types of factors which score labels of elements in predicate-argument structures drive.01 drive.02 A0 A1 A0 A1 … NONE NONE … Paul car

  9. The proposed model OBJ drive.01: drive a vehicle A0: driver A1: vehicle drive.02: cause to move A0: driver A1: things in motion NMOD SBJ car Paul drove his Expand the possible labels of predicate senses and argument roles drove We use four types of factors which score labels of elements in predicate-argument structures drive.01 drive.02 These factors are defined by (linear model) A0 A1 A0 A1 … NONE NONE … Paul car

  10. The proposed model OBJ drive.01: drive a vehicle A0: driver A1: vehicle drive.02: cause to move A0: driver A1: things in motion NMOD SBJ car Paul drove his drove use a factor which scores sense labels of the predicate 1.4754 0.7268 FP drive.01 drive.02 A0 A1 A0 A1 … NONE NONE … Paul car

  11. The proposed model OBJ drive.01: drive a vehicle A0: driver A1: vehicle drive.02: cause to move A0: driver A1: things in motion NMOD SBJ car Paul drove his drove use a factor which scores role labels of each argument FP drive.01 drive.02 A0 A1 A0 A1 … NONE NONE FA … 0.238 0.876 Paul car 1.784 -1.665 -1.235 -1.482

  12. The proposed model OBJ drive.01: drive a vehicle A0: driver A1: vehicle drive.02: cause to move A0: driver A1: things in motion NMOD SBJ car Paul drove his add a factor which scores label pairs of a predicate sense and a semantic role of an argument drove FP drive.01 drive.02 FPA 0.261 0.764 A0 A1 A0 A1 … NONE NONE FA … Paul car

  13. The proposed model OBJ drive.01: drive a vehicle A0: driver A1: vehicle drive.02: cause to move A0: driver A1: things in motion NMOD SBJ car Paul drove his add a factor which captures plausibility of the whole predicate-argument structure (use global features) FG drove A0,drive01,A1 … 1.865 FP drive.01 drive.02 FPA A0 A1 A0 A1 … NONE NONE FA … Paul car

  14. The proposed model OBJ drive.01: drive a vehicle A0: driver A1: vehicle drive.02: cause to move A0: driver A1: things in motion NMOD SBJ car Paul drove his add a factor which captures plausibility of the whole predicate-argument structure (use global features) FG drove A0,drive01,A1 … 1.865 FP drive.01 drive.02 FPA The predicate ‘drive’ has all obligatory roles A0 and A1 => FG assigns the higher score to the weight corresponds to this feature A0 A1 A0 A1 … NONE NONE FA … Paul car

  15. The proposed model OBJ drive.01: drive a vehicle A0: driver A1: vehicle drive.02: cause to move A0: driver A1: things in motion NMOD SBJ car Paul drove his The proposed model combines these types of factors FG drove A0,drive01,A1 … 1.4754 0.7268 1.865 FP drive.01 drive.02 FPA 0.425 0.764 A0 A1 A0 A1 … NONE NONE FA … 0.238 0.876 Paul car 1.784 -1.665 -1.235 -1.482

  16. The proposed model OBJ drive.01: drive a vehicle A0: driver A1: vehicle drive.02: cause to move A0: driver A1: things in motion NMOD SBJ car Paul drove his The proposed model combines these types of factors A0 A1 drive.01 FG The highest scoring assignment is returned by the proposed model drove A0,drive01,A1 … 1.4754 0.7268 1.865 FP drive.01 drive.02 FPA 0.425 0.764 A0 A1 A0 A1 … NONE NONE FA … 0.238 0.876 Paul car 1.784 -1.665 -1.235 -1.482

  17. Dealing with global (non-local) features • Introduce the fundamental idea of [Kazama and Torisawa 2007] • Features are divided into local features and global features • Inference: N-best based approach (1) Generate N-best assignments using only local features (2) Obtain global features in the N-best assignments (3) Select the argmax • Learning: train parameters with two margin constraints • All: train parameters so as to ensure a sufficient margin using all features (both local features and global features) • Local only: when the constraint All is satisfied, train parameters so as to ensure a sufficient margin using only local features • K&T proposed a Margin-Perceptron Learning Algorithm

  18. Inference and Learning Algorithm of the Proposed Model • Inference: generate N-best assignments for each predicate sense • Learning: the online Passive-Aggressive Algorithm [Crammer 2006] • The parameters are trained by solving the optimization problem used in PA with the two margin constraints: All (local + global) and Local only (2) Local only (1) All (local + global) positive positive other other margin margin

  19. Results on the CoNLL-2009 ST Dataset (average) • The best performance is obtained by using the all factors • Our model achieved the competitive results with the top system in the CoNLL-2009 Shared Task without any feature selection procedure FG FP sense FPA … roleN role1 role2 FA

  20. Results on the CoNLL-2009 ST Dataset (average) • By adding two types of factors FPA and FG, we obtained performance improvements in both tasks (predicate sense disambiguation and argument role labeling) => Succeeded in joint learning FG FP sense FPA … roleN role1 role2 FA

  21. Summary • We proposed a structured model that can capture two types of dependencies • Non-local dependencies among arguments • Inter-dependencies between a predicate and its arguments • The proposed model achieved the competitive results with the state-of-the-art SRL systems without any feature selection procedure • By adding two types of factors, we obtained performance improvements on both predicate sense disambiguation and argument role labeling => succeeded in joint learning • Future Work • exploiting unlabeled data (unsupervised or semi-supervised predicate-argument structure analysis)

More Related