Applications of Natural Language Processing

1. Applications of Natural Language Processing Course 5 – 22 March 2012 Diana Trandabățdtrandabat@info.uaic.ro

2. Content • Semantics in Language • What are Semantic Roles? • Semantic Role Resources • Semantic Role Labeling • Applications

3. Semantics in Language • The three basic layers of grammatical organization are summarized as: • Syntactic relations: perspectives through which situations are presented in linguistic expressions; • Semantic relations: the roles that participants play; • Pragmatic relations: the informational status of linguistic expressions.

4. Semantics in Language • A key concern in NLP is how meaning attaches to larger chunks of text. • This is Natural Language Semantics: • WSD • Anaphora • Argument structure/semantic role • Discourse analysis • etc.

5. Semantic intuitions • Nonsenses • Colorless green ideas sleep furiously • Kim frightened sincerity • Contradictions • It is raining and it is not raining • Kim killed Mary but she walked away. • Implications • John walked > John walked slowly • John sold the book to Mary > Mary bought the book.

6. What are Semantic Roles? • Linguistic background • Predicationality • Valence • Case Grammar • Description of Semantic Roles • Types of semantic roles • Characteristics of semantic roles • Semantic Role Resources • FrameNet, PropBank, VerbNet – for English • Resources for other languages

7. Linguistic background • Predicates(predicationalwords) designate events, properties of, or relations between, entities. • Linguistic expressions can be dependent or independent. • hat - can be understood outside any circumstance, time, or person, it is an individual. • red - cannot be understood outside its association with an individual: red hat. • In linguistic terms, dependent phenomena are predicates, while individuals are arguments.

8. Semantic Roles • Predications are treated as structures, named predicate-frames (Dik, 1987) or semantic frames (Chomsky, 1965). • Within predicate frames, each entity plays a role, called: • theta-role Chomsky (1965). • thematic relation (Gruber, 1965; Jackendoff, 1990) • semantic case (Fillmore, 1968) • semantic role (Dillon, 1977) • thematic role (Frawley, 1992) • Semantic roles are semantic relations that connect entities to events, more particularly, individuals to predicates.

9. Semantic Roles

10. Semantic Roles • Who, what, where, when, why? • Predicates • Verbs: sell, buy, cost, etc. • Nouns: acquisition, etc. Actors buyer seller goods money time etc..

11. Semantic Roles • Who, what, where, when, why? • Predicates • Verbs: sell, buy, cost, etc. • Nouns: acquisition, etc. • Actors • buyer • seller • goods • money • time • etc..

12. Semantic Roles • Who, what, where, when, why? • Predicates • Verbs: sell, buy, cost, etc. • Nouns: acquisition, etc. • Actors • buyer • seller • goods • money • time • etc.. Semantic Frame

13. Semantic Roles • Who, what, where, when, why? • Predicates • Verbs: sell, buy, cost, etc. • Nouns: acquisition, etc. • Actors • buyer • seller • goods • money • time • etc.. Semantic Frame Semantic Roles

14. Ioanaa cumpăratroşiide la Ion cu 2 lei. cumpărător de la vânzător a cumpăra bunuri cu bani

15. Ioanaa plătit2 leipentru roşiivânzătorului. cumpărător vânzătorului a plăti pentru bunuri bani

16. Vânzătoruli-a cerut Ioanei2 leipentru roşii. cumpărător vânzător a cere pentru bunuri bani

17. Valence • A predicational word needs, in order to complete its sense, arguments (mandatory) and adjuncts (optional). • Arguments: • Ion pleacă. • Ion citeşte o carte. • Ion îi dă Marieio carte. • Adjuncts (circumstantial complements) • Ion pleacă grăbit. • Ion citeşte o carte în tren. • Ion i-a dat Mariei o cartepentru trei zile.

18. Ioanaa cumpăratroşiide la Ion cu 2 lei. cumpărător de la vânzător a cumpăra bunuri cu bani

19. Ioanaa plătit2 leipentru roşiivânzătorului. cumpărător vânzătorului a plăti pentru bunuri bani

20. Vânzătoruli-a cerut Ioanei2 leipentru roşii. cumpărător vânzător a cere pentru bunuri bani

21. Vânzătoruli-a cerut Ioanei2 leipentru roşii. cumpărător vânzător a cere pentru bunuri bani

22. Cumpărarearoşiilorde către Ioana de la Ion cu 2 leia fost o afacere bună. cumpărător de la vânzător cumpărare bunuri cu bani

23. Cumpărarearoşiilorde către Ioana de la Ion cu 2 leia fost o afacere bună. cumpărător de la vânzător cumpărare bunuri cu bani

24. Case Grammar • Linguistic knowledge (Chomsky, 1968): • Surface Structure (syntax) • Deep structure (semantics) • Columb a descoperit America. • America a fost descoperită de Columb. • The linguistic process begins at the Deep Structure level with a non-verbal representation (an idea or a though) and ends in the Surface Structure, as we express ourselves. • Case Roles: representations of the lexical arguments of a predicate at semantic level.

25. Semantic Roles Agent– Columb a descoperit America. Patient/Theme – Columb a descoperit America. Experiencer – Rechinii au mirosit sânge. Beneficiary – I-a trimis mamei de săptămâna trecutăscrisoarea. Instrument– Au plecat cu vaporul. Location – În Hawaii e mult soare. Temporal – Ieri a nins. Commitatif– Copiii se joacăcu Maria. …

26. Classifications of Semantic Roles • Specificity: • Abstract roles: Agent, Patient, etc. • Specific roles: specific to a certain verb or class of verbs, as for instance Seller or Buyer for the verb sell. • Importance: • Core roles or Arguments • Peripheral roles or Adjuncts

27. (Assumed) Characteristics of Semantic Roles There is a relatively small fixed set of semantic roles. Semantic roles are atomic (one role does not subsume another). Every argument of every verb is assigned a semantic role or another. Each argument of a verb is assigned exactly one semantic role. Semantic roles are uniquely assigned within a verb (e.g., only one argument can be dubbed agent). Semantic roles are non-relational: the presence of a patient role does not imply the presence of an agent role.

28. Problems • A small fixed set of thematic roles has never been agreed on. Proposals range from just a few to hundreds. • Distinctness is hard to establish: • John met with Mary. • A is similar to B. • How and where to establish the boundary between role types (Instrument or Commitatif?) • John burgled the house [with an accomplice]. • John won the appeal [with a highly-paid lawyer].

29. Semantic Role Resources • FrameNet - University Berkeley California • Create frame; • Identify semantic roles (frame elements) and predicates (lexical units); • Search for examples containing the predicates in the British National Corpus; • Annotate them on different levels: Semantic roles (Frame Elements - FE), Grammatical Function (GF) and Phrase Type (PT). • 11,600 lexical units in more than 960 semantic frames, exemplified in more than 150,000 annotated sentences.

30. Semantic Role Resources • PropBank – University Penn • focuses on the argument structure of verbs (NomBank for nouns) • provides a complete corpus annotated with semantic roles, including arguments and adjuncts. • PropBank defines semantic roles on a verb by verb basis (defines a semantic frame for each verb, groups examples in frame sets for each verb). • Semantic arguments of verbs are numbered from 0 to 4. (Arg0 - Agent, Arg1 - Patient or Theme) • Verbs can have also general roles (adjuncts) marked as ARG-Ms (modifiers): TMP, LOC, MNR, REC, MOD, NEG, etc.

31. Semantic Role Resources • FrameNet: • [Chunk]Buyer bought [a car]Goods [from Jerry]Seller [for $100]Money. • [Jerry]Seller sold [a car]Goods [to Chuck]Buyer [for $100]Money. • PropBank • [Chunk]Arg0 bought [a car]Arg1 [from Jerry]Arg2 [for $100]Arg3. • [Jerry]Arg0 sold [a car]Arg1 [to Chuck]Arg2 [for $100]Arg3.

32. Semantic Role Resources for languages other than English Merge approach Expand approach • Two methods for creating new resources from English: • Merge: independent resources for different languages are first built from scratch, than linked. • Expand: produce structurally similar resources • German FrameNet • Japanese FrameNet • Spanish FrameNet • Romanian FrameNet

33. Creating a Semantic Role Resource for Romanian • Creating from scratch a semantic role resource implies several steps before the annotation process itself: • finding a corpus, • establishing an annotation schema and defining annotation guidelines, • choosing/creating an annotation software, • training annotators.

34. Intuition • The intuition behind the import program, presented in (Trandabat et al., 2005), is that most of the frames defined in the English FN are likely to be valid cross-linguistically, because semantic frames express conceptual structures, language independent, at the deep structure level. • load sentences from an annotated English frame; • translate them; • align the English and Romanian versions; • import the roles; • visualize, correct and save the annotations.

35. Semantic Role Transfer Architecture

36. Alignment For the development phase of the import program, the English sentences have been aligned at word level using RACAI's aligner COWAL. Although COWAL has an alignment precision of more than 87%, since the RACAI Aligner does not have yet a web service interface, and due to the fact that the import program is supposed to work also unassisted, the alignment have been implemented to work only with GIZA++.

37. Semantic Role Import • reading the English XML files and the alignment files; • linking each English word with its corresponding semantic role (FE); • mapping the English words with the aligned Romanian translation, thus transferring the annotation of a specific role from English to Romanian. The mapping was performed by considering the import as a sequential labeling problem, with a B-I-O encoding. • The_B_Event incident_I_Event occurred_B_TARGET after_B_Time/Cause a_I_Time/Cause dispute_I_Time/Cause between_I_Time/Cause the_I_Time/ Cause man_I_Time/Cause and_I-Time/Cause staff_I_Time/Cause at_B_Place a_I_Place branch_I_Place of_I_Place the_I_Place Bank_I_Place of_I_Place Ireland_I_Place in_I_Place Cahir_I_Place ._O_NO-Frame

38. Semantic Role Import One-to-zero import: the English word has no Romanian correspondent -> no action is needed. One-to-one import: the English word has only one Romanian word as translation -> transfer. One-to-many import: one English word being translated with two or more Romanian words. Based on empiric observations and the semantic roles nature, we decided to apply the following rule:

39. Semantic Role Import Many-to-one import: two or more English words align with the same Romanian word (e.g. the definite article). Zero-to-one import: This case requires introducing a frame annotation to the Romanian word, without having a frame annotation in English. A Romanian has an I_Fi annotation when the word interrupts a frame (it breaks a consecutive sequence of B_Fi and I_Fi). Otherwise, the Romanian word will have the frame O NO-Frame.

40. Import evaluation • overall accuracy of approx. 85% • Most frequent error types: • Double Annotation • The most frequent case of double annotation is for the Time/Cause roles, • Imbrications • Unexpressed Semantic Frames

41. Applications of Semantic Role Labeling Systems • Semantic Roles in Question Answering • Annotate both questions and snippets for answer extraction with semantic roles; • Create patterns of type: • ARG0 forced the adoption of the decision. • Really useful for purpose and temporal/local questions

42. Applications of Semantic Role Labeling Systems • Semantic Roles for Prosody Generation • Apply Topic-Focus Articulation algorithm over the text; • Create topic intonational structure; • Use semantic roles for determining which constituents of a sentence are more accentuated: • Predicate < Arguments < Adjuncts

43. Requirements (Team:max 1 person, Deadline: 29 March) • 1) Write an automatic role import program, that transfers the roles in language A into language B • 2) Can it be used also for other import types? Which one? Explain why.

44. Bibliography • Henriëtte de Swart, 1998, Introduction to Natural Language Semantics, CSLI Publications, Stanford. • GildeaDaniel and Jurafsky Daniel, Automatic labeling of semantic roles, Computational Linguistics, 28(3):245-288, 2002 • LluísMàrquez, Xavier Carreras, Kenneth C. Litkowski, Suzanne Stevenson, Semantic Role Labeling: An Introduction to the Special Issue, Computational Linguistics, Vol. 34, No. 2, Pages 145-159, 2008. • Trandabăț D., Natural Language Processing Using Semantic Frames, PhD Thesis, University Al. I. Cuza Iasi, Romania.

45. Links • FrameNethttps://framenet.icsi.berkeley.edu/fndrupal/ • Propbankhttp://verbs.colorado.edu/~mpalmer/projects/ace.html • Semantic Role Labeling demo: http://cogcomp.cs.illinois.edu/demo/srl/

46. Thanks!

48. PropBank -example • predicate lemma="sell"> • Roleset sell.01 • Arg0: Seller • Arg1: Thing Sold • Arg2: Buyer • Arg3: Price Paid • Arg4: Benificiary • [It]Arg0 will sell [the ad time] Arg1 [to its clients] Arg2 [at a discount] Arg3 back