Semantic Role Chunking Combining Complementary Syntactic Views

1. Semantic Role Chunking Combining Complementary Syntactic Views Sameer Pradhan, Kadri Hacioglu, Wayne Ward, James H. Martin, Daniel Jurafsky Center for Spoken Language Research Department of Computer Science University of Colorado at Boulder  Department of Linguistics Stanford University

2. Different Syntactic Views • Hypothesis: Different views make different errors • Two views: • Phrase structure based (Charniak, Collins) • Chunk based

3. Constituents from Charniak parse tree Constituents from Collins parse tree Charniak Parse Tree Collins Parse Tree Constituent Views Johnkickedthe ball .

4. Chunk View [Hacioglu & Ward 2003] • Chunk using an IOB representation [Ramshaw & Marcus, 1995] • Yamcha [Kudo & Matsumoto, 2001] Salomon will buy sufficient shares to cover its entire position • Bottom up as opposed to top down • Flat representation • Uses flat syntactic chunks O O O B-A2 I-A2 O B-V B-A1 I-A1 I-A1

5. Algorithm • Generate Charniak and Collins parse based features • Add few features from one to the other • Generate semantic IOB tags using these views • Use them as features • Generate the final semantic role label set using a phrase-based chunking paradigm

6. Charniak Collins Words Phrases IOB IOB IOB Features Chunker IOB Semantic Role Labels Architecture

7. 2 H 1 B B B I I I I I I I I I I O I O O O O B O O I O Classifier B B I I O O O O I O O O O O O Model B B B I I I I I I Illustration 2 R 1 O O B I O O O O I I O O I O O O O O Model Train B B B I I I I I I O I O B B O B B B I I I I I I

8. Features • Semantic IOB tags for Charniak and Collins based semantic role labels [Pradhan et al., 2005] • Phrase level chunk features [Hacioglu et al., 2004]

9. Active Learning • Randomly selelected 10k examples and trained a NULL vs ARGUMENT classifier • Classified remaining examples using this classifier • Added misclassified examples to the seed set • Iterated • Final data amounted to about a third of the total

10. Combination Results ID + Class P R F1 System 80 75 77 ASSERTCharniak 79 74 76 ASSERTCollins 81 76 78 ASSERTCombined Train : Sections 02-21 of PropBank Test : Section 24 of PropBank

11. 81.9 75.1 78.3 82.9 74.7 78.6 74.5 63.3 68.4 Bug fixed System Results ID + Class Section 24 Section 23 Brown P R F1 P R F1 P R F1 System 80.9 75.4 78.0 81.9 73.3 77.4 73.7 61.5 67.1 Submitted System

12. Thank You Arda AQUAINT program contract OCG4423B NSF grant IS-9978025

13. Software • ASSERT (Automatic Statistical SEmantic Role Tagger) • Publicly downloadable at http://oak.colorado.edu/assert • Downloaded by more than 50 research groups

14. Null Filtering • Removed constituents with P(NULL) > 0.9 • Removed phrases with P(NULL) > 0.8 after incorporating context

15. Analysis • Active learning using confidence threshold • Constituent level instead of Sentence level • N-Best Charniak parses

16. Features (Constituent)

17. Features (Constituent)

18. Features (Phrase)

19. Features (Phrase)

20. Representation

21. Features

22. Features

23. But analysts reckon underlying support for sterling has been eroded by the chancellor 's failure to announce any new policy measures in his Mansion House speech last Thursday Minipar-based Semantic Labeling • Rule-based dependency parser