Nlp 1 an introduction to pragmatics in nlp
Download
1 / 45

NLP 1 - PowerPoint PPT Presentation


  • 493 Views
  • Uploaded on

NLP 1 An Introduction to Pragmatics in NLP. GSLT, Göteborg, March 2006. Barbara Gawronska, Högskolan i Skövde. Reading list:. Jurafsky & Martin, part IV

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'NLP 1' - ivanbritt


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Nlp 1 an introduction to pragmatics in nlp l.jpg

NLP 1An Introduction to Pragmatics in NLP

GSLT,

Göteborg, March 2006

Barbara Gawronska, Högskolan i Skövde


Reading list l.jpg
Reading list:

  • Jurafsky & Martin, part IV

  • Mitkov, R. 2000. "Towards a more consistent and comprehensive evaluation of anaphora resolution algorithms and systems." Proceedings of the Discourse Anaphora and Anaphora Resolution Colloquium (DAARC-2000), 96-107, Lancaster, UK (pdf)

    http://clg.wlv.ac.uk/papers/Lancaster2000.PDF

  • Mitkov, R. and Barbu, C. 2002. "Using corpora to improve pronoun resolution." Languages in context, 4(1). (pdf )

    http://clg.wlv.ac.uk/papers/mitkov02.pdf

  • Hutchins, J. 2003. "Has Machine Translation improved?" An expanded version [PDF, 288KB] of a paper presented at MT Summit IX: Proceedings of the Ninth Machine Translation Summit, New Orleans, USA, September 23-27, 2003, 181-188. [East Stroudsburg, PA: AMTA.] [PDF, 191KB]

    http://ourworld.compuserve.com/homepages/WJHutchins/HasMTimproved-exp.pdf


Outline l.jpg
Outline:

  • The notion ”Pragmatics”

  • Pragmatics vs. Semantics

  • Pragmatics and NLP: Discourse Processing

    • Anaphora resolution

    • NL Generation

    • Information Extraction and Text Summarization

    • Machine Translation

    • CALL

  • Future directions


Pragmatics vs semantics 1 l.jpg
Pragmatics vs. Semantics (1)

  • Austin 1962 – Pragmatics = the study of "how to do things with words“

  • Leech & Weisser 2003 – Pragmatics = “the branch of linguistics which seeks to explain the meaning of linguistics messages in terms of their context of use” ,

    while

    Semantics investigates meaning ”as part of the language system irrespective of wider context”


Pragmatics vs semantics 2 l.jpg
Pragmatics vs. Semantics (2)

  • Classical work on pragmatics (Austin 1962, Searle 1969, Grice 1975); problems as:

    • Discourse referents; what entities does a given message refer to?

    • What background knowledge is needed to understand a given message?

    • How do the beliefs of speaker and hearer interact in the interpretation of a message?

    • What is a relevant answer to a given question?


Pragmatics vs semantics 3 l.jpg
Pragmatics vs. Semantics (3)

This implies that the study object of semantics and pragmatics comprisesinteractions between entities on different levels of the linguistic structure. Pragmatics takes even interactions between the linguistic and the non-linguistic reality into account.

E.g. identification of discourse referents (entities referred to) in spoken language requires an interplay between phonetic/phonological, morphological, syntactic, and semantic factors as well as the use of extralinguistic knowledge.


Problems with reference in spoken language processing an example from august kth l.jpg
Problems with reference in spoken language processing – an example (from “August”, KTH)

User and system have different background knowledge:

  • User: Finns det en bra restaurang i närheten? (Is there a good restaurant nearby?)

  • System: Du måste ange gatan (You have to name the street)

    The system gives an answer that is true, but not relevant:

  • User: Var är vi? (Where are we?)

  • System: Vi är ju här. (We are here.)


Pragmatics in nlp l.jpg
Pragmatics in NLP example (from “August”, KTH)

Discourse processing for

  • Dialogue systems

  • Natural Language Generation

  • Reading Comprehension (e.g. in Q/A systems, in summarization systems)

  • Machine Translation

  • Multifunctional NLP systems

  • Computer Assisted Language Learning (CALL)


Discourse processing 1 l.jpg
Discourse processing (1) example (from “August”, KTH)

  • Discourse level – beyond the sentence level

  • Traditional distinctions:

    • Spoken/written discourse

    • Monologue/dialogue

  • New discourse types related to new ways of communicating: SMS, chatting, e-mail...


Discourse processing 2 l.jpg
Discourse processing (2) example (from “August”, KTH)

  • The main aspects:

    • Anaphora resolution

    • Cohesion and coherence

    • Discourse structure


Anaphora resolution 1 l.jpg
Anaphora resolution (1) example (from “August”, KTH)

  • Theoretical work: Karttunen 1976, Kamp 1979, 1981, Grosz and Sidner 1986, Hobbs 1978, 1982, Dagan & Itai 1990, Lappin & Leass 1944, Mitkov and Barbu 2000, 2002...)

  • Basic notions:

    • Anaphora

    • Antecedent

    • Discourse referent

    • Coreference chain


Anaphora resolution 2 l.jpg
Anaphora resolution (2) example (from “August”, KTH)

  • Sources of knowledge:

    • Syntactic and morphosyntactic constraints (boundedness, gender, number, grammatical roles):

      Mary met John. He/She/They decided...

      She helped her/herself

    • Semantic features, selectional restrictions

      I bought a bottle of wine, sat down on a stone, and drankit


Anaphora resolution 3 l.jpg
Anaphora resolution (3) example (from “August”, KTH)

  • Ontological knowledge, domain knowledge

    in interaction with semantic and grammatical constraints

    My friends have a greyhound. They are really huge beasts

    They prohibited them from demonstrating because they feared violence

    They prohibited them from demonstrating because they advocated violence

    (Winograd 1972:33)


Algorithms for anaphora resolution 1 l.jpg
Algorithms for anaphora resolution (1) example (from “August”, KTH)

  • Based on parse trees (naïve):

    Left-to right, breadth-first search, starting with the sentence containing the pronoun

  • Based on syntactic roles: The centering algorithm (Grosz et al 1995, Lappin and Leass 1994)

  • Based on lexical and collocational indicators: Mitkov’s knowledge poor approach (Mitkov 1998)

  • Based on so-kalled ”pragmatic functions”: the Mental Space model (Fauconnier 1985,1998)


Algorithms for anaphora resolution 2 the centering algorithm l.jpg
Algorithms for anaphora resolution (2): example (from “August”, KTH)The centering algorithm

Backward lookning center (CB) - the entity currently in focus

Forward looking centers (CF) - an ordered list of entities:

Subject > Predicative NP > direct object >oblique > PP

”Preferred center” (CP) - the highest ranked forward looking element

A ranked set of transitions:

  • Continue: CB = CP = CB of the previous utterance

  • Retain: CB\= CP; CB = CB of the previous utterance

  • Smooth –shift: CB = CP; CB \= CB of the previous utterance

  • Rough-shift: CB \= CP; CB \= CB of the previous utterance


Algorithms for anaphora resolution 3 the knowledge poor approach mitkov 1998 2000 l.jpg
Algorithms for anaphora resolution (3): The knowledge-poor approach (Mitkov 1998, 2000)

  • Input: a text processed by a POS-tagger and an NP extractor

  • Locate all NPs which precede the anaphor within a distance of 3 sentences

  • Check number and gender agreement, filter out NP:s that do not fulfil agreement conditions

  • Apply boosting and impeding indicators to the remaining NP:s


Algorithms for anaphora resolution 4 the knowledge poor approach mitkov 1998 l.jpg
Algorithms for anaphora resolution (4): The knowledge-poor approach (Mitkov 1998)

  • Boosting indicators (some examples):

    • First NP in a sentence

    • Lexical Iteration (NP:s repeated twice or more in the papagraph before the pronoun)

    • Section Heading Preference

    • Collocation Pattern Preference (Press the key down and turn the volume up. Press it again…)*

    • Term preference (terms characteristic for the genre

      * For an interesting approach to collocations (collostructional analysis), see Gries, Hampe, & Schönefeld 2005


Algorithms for anaphora resolution 4 the knowledge poor approach mitkov 199818 l.jpg
Algorithms for anaphora resolution (4): The knowledge-poor approach (Mitkov 1998)

  • Impeding indicators (some examples):

    • Indefiniteness

    • Complement of a preposition

    • Referential distance

      Evaluation:

      Success rate: Number of sucessfully resolved anaphors/Number of all anaphors

      (Different variants paying atention to trivial and non-trivial anaphors)


Slide19 l.jpg

The Theory of Mental Spaces (Fauconnier approach (Mitkov 1998)1985, Fauconnier and Sweetser 1996 – focus on beliefs and attitudes)


Slide20 l.jpg

The Theory of Mental Spaces (2) approach (Mitkov 1998)

(Fauconnier 1985, Sweetser & Fauconnier 1996, Sanders & Redeker 1996)


Natural language generation 1 l.jpg
Natural Language Generation (1) approach (Mitkov 1998)

  • Discourse planning

    • Templates – partially pre-defined text frames

    • Algorithms based on discourse theories (e.g. Rhetorical Strucure Theory (RST) – Mann & Thompson

  • Sentence planning (sentence aggregation, generation of referring expressions, lexical selection)

  • Surface realization (word order and agreement control, graphic realization)


Natural language generation 2 l.jpg
Natural Language Generation (2) approach (Mitkov 1998)

  • Main issues: cohesion and coherence

    • Cohesion: establishing anaphoric connections (the ”reverse” of anaphora resolution)

    • Coherence: ”nucleus-satellite” relations (RST)

      e.g. result, cause, elaboration, contrast, parallel...


Natural language generation 3 l.jpg
Natural Language Generation (3) approach (Mitkov 1998)

  • Some examples of coherence relations:

    John bought a dog. His wife went furious (result)

    John hid Bill’s car keys. Bill had drunk too much (explanation)

    John bought a Mercedes. Bill leased a BMW (parallel)

  • An insufficiently cohesive/coherent text:

    I saw a little dog. Dogs like bones. Bones are white. White is my favourite colour...


Sentence aggregation an example visit http www iccs informatics ed ac uk jbos anna for more demos l.jpg
Sentence aggregation – an example (visit approach (Mitkov 1998)http://www.iccs.informatics.ed.ac.uk/~jbos/anna/ for more demos)

  • TEXT WITHOUT AGGREGATION:IcelandIceland is situated in the north Atlantic. Iceland has a coastline of 5 000 km. Iceland has an area of 103 000 sq km. The highest point is 2 119 m. The highest point is Hvannadalshnukur. Iceland has a temperate climate. Iceland has mild, windy winters and cool, damp summers. Iceland has 280 000 inhabitants. The population density is 3 people/sq km. The life expectancy is 79 years. The fertility rate is 2 children. The official language is Icelandic. Icelandic is a germanic language. Iceland is a constitutional republic. The capital is Reykjavik. Reykjavik has 107 000 inhabitants. The national holiday is June 17.


Slide25 l.jpg

TEXT WITH AGGREGATION: approach (Mitkov 1998)IcelandIceland is situated in the north Atlantic.

Iceland has a coastline of 5 000 km and an area of 103 000 sq km. The highest point, Hvannadalshnukur, is 2 119 m. Iceland has a temperate climate with mild, windy winters and cool, damp summers. Iceland has 280 000 inhabitants and the population density is 3 people/sq km. The life expectancy is 79 years and the fertility rate is 2 children.

The official language is Icelandic, a germanic language. Iceland is a constitutional republic. The capital, Reykjavik, has 107 000 inhabitants.The national holiday is June 17.



Text summarization types of summaries l.jpg
Text Summarization: Types of summaries approach (Mitkov 1998)

  • With respect to content:

    • Indicative: provide an idea what the text is about

    • Informative: shortened versions of the text

  • With respect to the way of creating:

    • Extracts: reused portions of the text (text retrieval)

    • Abstracts: re-generated text reflecting the important content (information extraction and text generation)

    • Compressed texts: (Knight & Marcu 2000): compressing parse trees in order to get a shorter text

    • Dialogue summarization: selecting successful ’dialog transactions’

    • Non-verbal ”summaries” (e.g. graphical representations)


Abstract creation l.jpg
Abstract creation: approach (Mitkov 1998)

  • Template-based summarization (templates, “sketchy frames”, “extraction patterns”: frames containing slots with constraints and variables; relay on prior domain knowledge); some examples:

    • DeJong 1982: FRUMP (Fast Reading Understanding and Memory Program)

    • Rilloff 1996: CIRCUS (terrorism domain)

    • McKeown and Radev 1999: SUMMONS (SUMMarizing Online NewS articles)

  • “Plot units” (selecting causal relations; Lehnert 1981)




Slide31 l.jpg

Some principles for selection of claims to be rendered: (Gawronska et al. 2004)

  • 1) Informatives:

  • Neutral, the sender is not marked for high status: officials said, the news agency reported, reportedly…A claim p introduced by a neutral informative is rendered in the summary; the source is omitted if there are no denials or confirmations of p in the text and if the source is not marked for high status, like ‘President’

  • Neutral, the sender marked for high status, and ‘declarations’: the President said…the government condemned…The source is rendered if it is marked for high status

  • Affirmative; confirmations of explicit claims: Israeli sources confirmed that…Confirmations of previous explicit claims are omitted in the summary

  • Affirmative; confirmations of claims that are not explicitly mentioned:Both the information source and the claim, including the type of the speech act phrase, are rendered in the summary, if the speech act is a confirmation of a claim not present in the news report


Slide32 l.jpg

Some principles for selection of claims (Gawronska et al. 2004)

to be rendered:

  • 1) Informatives:

  • Negative, or neutral followed by denied claims:The president denied, The Israeli source said that it is not true…Both the initial claim and its denial are rendered in the summary together with the information about the senders


Slide33 l.jpg

Some principles for selection of claims (Gawronska et al. 2004)

to be rendered (2):

2) Utterance refusal, negated speech act phrases, hypotheses, commissives, interpretations: The Israeli sources neither denied or confirmed, the minister did not say, if…, the defense secretary declined to say…, the government had no immediate comments…

Utterance refusals or negated speech act phrases related to an explicit claim are omitted

If a source refuses to confirm/deny a claim that has not been explicitly mentioned in the previous part of the text, the whole speech act is rendered, inclusive the type of the speech act

Hypotheses and commissives are rendered together with their sources and marked for unsure epistemic status


Slide34 l.jpg

Some principles for selection of claims to be rendered (3): (Gawronska et al. 2004)

3) Epistemic spaces: e. g. no one knows if the device was planted deliberately or if it was leftover from New Year’s Eve

If two claims would exclude each other in the same mental space, and if no source in the text takes responsibility for any of these claims, both claims are to be rendered as hypotheses


Slide35 l.jpg

Some challenges… (Gawronska et al. 2004)

RAMALLAH, West Bank -- Palestinian leader Yasser Arafat said Thursday that elections as part of a reform of the Palestinian Authority will be held this winter, whether or not Israeli forces withdraw from the Palestinian territories.

That represented a change of course from Arafat, who said last week that no elections would be held until the Israelis pulled back. Shortly after Arafat's announcement, a committee he had appointed to set up elections resigned, according to Israel Radio, because Arafat would not agree to a specific date for the elections. Other Palestinian leaders said the resignations were a procedural matter. Arafat also condemned Wednesday's suicide bombing in the Israeli town of Rishon Letzion . Two Israelis were killed and at least 37 others wounded when the bomber detonated explosives in the center of a crowded pedestrian district.

The terror attack marked the second time in two weeks a suicide bombing directed at civilians has rocked Rishon Letzion, a town about 15 miles southeast of Tel Aviv. On May 8, a suicide attack at a pool hall killed 15 people and wounded dozens of others.

"Suddenly there was an explosion," 16-year-old Shmuel Voller told The Associated Press on Wednesday.

The bombing occurred on Rothschild Street in the heart of the town around 9:15 p.m. (2:15 p.m. ET).


Slide36 l.jpg

An example of non-verbal “summaries”: extracting pathway maps similar to those in Kyoto Encyclopaedia of Genes and Genomes (KEGG) from biomedical literature


A sample comprimed tree expressing a biological process l.jpg
A sample comprimed tree expressing a biological process maps similar to those in Kyoto Encyclopaedia of Genes and Genomes (KEGG)


Machine translation combining nl understanding and nl generation 1 l.jpg
Machine Translation – combining NL Understanding and NL Generation (1)

  • 1940... – the first attempts: direkt word-to-word translation + some morphosyntactic processing (e.g. case recognition in Russian)

  • 1970...-syntax-based approaches: interlingua and transfer

  • 1990 – Brown et al. – foundation of stochastic MT (computing translation probabilities on the basis of parallel corpora)


Machine translation 2 l.jpg
Machine Translation (2) Generation (1)

  • Knowledge Based Machine Translation – KBMT – Nirenburg et al., Hobbs, Wilks mm

    - knowledge stored in lexicons, onomasticons, and ontologies

  • rule-based parsing and semantico-pragmatic analysis aimed at conceptuel representations

  • Example Based MT – EBMT - translation in analogy with best match in the corpus of previously translated texts

  • Hybrid systems (e.g. Verbmobil – Wahlster et al 2000)


Slide41 l.jpg
The multi engine architecture of the MT system Verbmobil Generation (1)(a simplified version of Figure 11, p. 17 in [33])


Mt evaluation some useful links l.jpg
MT evaluation – some useful links Generation (1)

  • Hutchins, John (2000) The IAMT Certification initiative and defining translation system categories. (Presented at EAMT Workshop, Ljubljana, May 2000)

    http://ourworld.compuserve.com/homepages/WJHutchins/IAMTcert.htmhttp://ourworld.compuserve.com/homepages/WJHutchins/Compendium-4.pdf

  • http://www.issco.unige.ch/projects/isle/femti/


Mt current trends l.jpg
MT, current trends: Generation (1)

  • Towards hybrid systems: integration of rule-based approaches and stochastic approaches

  • Spoken language translation

  • Sign language translation

  • Combined MT and Intormation Extraction

  • Computer aided translation



Conclusions future l.jpg
Conclusions; Future? communicative competence

Pragmatics – still a challenge for NLP

Research needed on:

  • General vs. domain-specific resources and algorithms

  • User models (beliefs, attitudes, etc.)

  • The interplay between prosody, syntax, and semantics

  • New means of communication, new types of discourse

  • Synergy between rule-based and stochastic approaches


ad