1 / 28

The Practical Value of Statistics for Sentence Generation: - PowerPoint PPT Presentation

The Practical Value of Statistics for Sentence Generation: The Perspective of the Nitrogen System. Irene Langkilde-Geary. How well do statistical n-grams make linguistic decisions?. Subject-Verb Agreement Article-Noun Agreement

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation

PowerPoint Slideshow about 'The Practical Value of Statistics for Sentence Generation:' - victoria

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript

The Practical Value of Statistics for Sentence Generation:The Perspective of the Nitrogen System

Irene Langkilde-Geary

Subject-Verb AgreementArticle-Noun Agreement

I am 2797 a trust 394 an trust 0 the trust 1355

I are 47 a trusts 2 an trusts 0 the trusts 115

I is 14

Singular vs PluralWord Choice

their trust 28 reliance 567 trust 6100

their trusts 8 reliances 0 trusts 1083

Relative pronounPreposition

visitor who 9 visitors who 20 in Japan 5413 to Japan 1196

visitor which 0 visitors which 0

visitor that 9 visitors that 14 came to 2443 arrived in 544

came in 1498 arrived to 35

Singular vs Plural came into 244 arrived into 0

visitor 575 visitors 1083

came to Japan 7 arrived to Japan 0

Verb Tense came into Jap 1 arrived into Japan 0

admire 212 admired 211 came in Japan 0 arrived in Japan 4

admires 107

• Enumerate all possible expressions

• Rank them in order of probabilistic likelihood

Why two steps? They are independent.

• Ngram model

Formula for bigrams:

P(S) = P(w1|START) * P(w2|w1) * … * P(w n|w n-2)

• Probabilistic syntax (current work)

• A variant of probabilistic parsing models

Random path: (out of a set of 11,664,000 semantically-related sentences)

Visitant which came into the place where it will be Japanese has admired that there was Mount Fuji.

Top three:

Visitors who came in Japan admire Mount Fuji .

Visitors who came in Japan admires Mount Fuji .

Visitors who arrived in Japan admire Mount Fuji .

Strengths

• Reflects reality that 55% (Stolke et al. 1997) of dependencies are binary, and between adjacent words

• Embeds linear ordering constraints

ExampleReason

Visitors come inJapan. A three-way dependency

He planned increase in sales. Part-of-speech ambiguity

A tourist who admire Mt. Fuji... Long-distance dependency

A dog eat/eats bone. Previously unseen ngrams

I cannot sell their trust. Nonsensical head-arg relationship

The methods must be modified to Improper subcat structure

the circumstances.

Representation of enumerated possibilities(Easily on the order of 1015 to 1032 or more)

• List

• Lattice

• Forest

• Issues

• space/time constraints

• redundancy

• localization of dependencies

• non-uniform weights of dependencies

Number of phrases versus time (in seconds) inputsfor 15 sample inputs

INPUT  ( <label> <feature> VALUE )

VALUE  INPUT -OR- <label>

Labels are defined in:

• input

• user-defined lexicon

• WordNet-based lexicon

(~ 100,000 concepts)

Example Input:

(a1 :template (a2 / “eat”

:agent YOU

:patient a3)

:filler (a3 / |poulet| ))

Mapping Rules inputs

• Recast one input to another

• (implicitly providing varying levels of abstraction)

• Assign linear order to constituents

• Add missing info to under-specified inputs

Matching Algorithm

• Rule order determines priority. Generally:

• Recasting < linear ordering < under-specification

• High (more semantic) level of abstraction < low (more syntactic)

• Distant position (adjuncts) from head < near (complements)

• Basic properties < specialized

(a1 :venue < inputsvenue>

:cusine <cuisine> )

(a2 / |serve|

:agent <venue>

:patient <cuisine> )

(a2 / |have the quality of being|

:domain (a3 / “food type”

:possessed-by <venue>)

:range (b1 / |cuisine|))

Recasting

(a1 :venue < inputsvenue>

:region <region> )

(a2 / |serve|

:agent <venue>

:patient <cuisine>

(a3 / |serve|

:voice active

:subject <venue>

:object <cuisine> )

(a3 / |serve|

:voice passive

:subject <cuisine>

:adjunct (b1 / <venue>

:anchor |BY| ))

Recasting

Linear ordering inputs

(a3 / |serve|

:voice active

:subject <venue>

:object <cuisine> )

<venue>

(a4 / |serve|

:voice active

:object <cuisine> )

Under-specification inputs

(a4 / |serve|)

(a6 / |serve|

:cat noun)

(a5 / |serve|

:cat verb)

Under-specification inputs

(a4 / |serve|)

(a5 / |serve|

:cat verb)

(a5 / |serve|

:cat verb

:tense past)

(a5 / |serve|

:cat verb

:tense present)

Syntactic relations

:subject :object :dative :compl :pred :adjunct :anchor :pronoun :op :modal :taxis :aspect :voice :article

Functional relations

:logical-sbj :logical-obj :logical-dat:obliq1 :obliq2 :obliq3 :obliq2-of :obliq3-of :obliq1-of :attr :generalized-possesion :generalized-possesion-inverse

Semantic/Systemic Relations

:agent :patient :domain :domain-of :condition :consequence :reason :compared-to :quant :purpose :exemplifier :spatial-locating :temporal-locating :temporal-locating-of :during :destination :means :manner :role :role-of-agent :source :role-of-patient :inclusive :accompanier :sans :time :name :ord

Dependency relations

:arg1 :arg2 :arg3 :arg1-of :arg2-of :arg3-of

:cat [nn, vv, jj, rb, etc.]

:polarity [+, -]

:number [sing, plural]

:tense [past, present]

:person [1s 2s 3s 1p 2p 3p s p all]

:mood [indicative, pres-part, past-part, infinitive, to-inf, imper]

Sentence  Constituent+

Constituent  Constituent+ OR Leaf

Leaf  Punc* FunctionWord* ContentWord FunctionWord* Punc*

FunctionWord  ``and'' OR ``or'' OR ``to'' OR ``on'' OR ``is'' OR ``been'' OR ``the'' OR ….

ContentWord  Inflection(RootWord,Morph)

RootWord  ``dog'' OR ``eat'' OR ``red'' OR ....

Morph  none OR plural OR third-person-singular ...

Computational Complexity inputs

(x2/A2) + (y2/B2) = 1

???

Y

X

Advantages of a statistical approach inputsfor symbolic generation module

• Shifts focus from “grammatical” to “possible”

• Significantly simplifies knowledge bases

• Broadens coverage

• Potentially improves quality of output

• Dramatically reduces information demands on client

• Greatly increases robustness