Modeling grammaticality l.jpg
This presentation is the property of its rightful owner.
Sponsored Links
1 / 17

Modeling Grammaticality PowerPoint PPT Presentation


  • 61 Views
  • Uploaded on
  • Presentation posted in: General

Modeling Grammaticality. [mostly a blackboard lecture]. Word trigrams: A good model of English?. Which sentences are grammatical?. names. . all. has. ?. s. . ?. forms. ?. was. his house. . no main verb. same. 600.465 - Intro to NLP - J. Eisner. 2. has. s. has.

Download Presentation

Modeling Grammaticality

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Modeling grammaticality l.jpg

Modeling Grammaticality

[mostly a blackboard lecture]

600.465 - Intro to NLP - J. Eisner


Word trigrams a good model of english l.jpg

Word trigrams: A good model of English?

Which sentences are grammatical?

names

all

has

?

s

?

forms

?

was

his house

no main verb

same

600.465 - Intro to NLP - J. Eisner

2

has

s

has


Why it does okay l.jpg

Why it does okay …

  • We never see “the go of” in our training text.

  • So our dice will never generate “the go of.”

    • That trigram has probability 0.


Why it does okay but isn t perfect l.jpg

You shouldn’t eat these chickens because these chickens eat arsenic and bone meal …

3-gram model

Training sentences

… eat these chickens eat …

Why it does okay … but isn’t perfect.

  • We never see “the go of” in our training text.

  • So our dice will never generate “the go of.”

    • That trigram has probability 0.

  • But we still got some ungrammatical sentences …

    • All their 3-grams are “attested” in the training text, but still the sentence isn’t good.


Why it does okay but isn t perfect5 l.jpg

Why it does okay … but isn’t perfect.

  • We never see “the go of” in our training text.

  • So our dice will never generate “the go of.”

    • That trigram has probability 0.

  • But we still got some ungrammatical sentences …

    • All their 3-grams are “attested” in the training text, but still the sentence isn’t good.

  • Could we rule these bad sentences out?

    • 4-grams, 5-grams, … 50-grams?

    • Would we now generate only grammatical English?


Grammatical english sentences l.jpg

Training sentences

Possible under trained 3-gram model(can be built from observed 3-grams by rolling dice)

Possible under trained 4-gram model

Grammatical English sentences

Possible undertrained 50-grammodel ?


What happens as you increase the amount of training text l.jpg

Training sentences

Possible under trained 3-gram model(can be built from observed 3-grams by rolling dice)

Possible under trained 4-gram model

What happens as you increase the amount of training text?

Possible undertrained 50-grammodel ?


What happens as you increase the amount of training text8 l.jpg

What happens as you increase the amount of training text?

Training sentences (all of English!)

Now where are the 3-gram, 4-gram, 50-gram boxes?

Is the 50-gram box now perfect?

(Can any model of language be perfect?)

Can you name some non-blue sentences in the 50-gram box?


Are n gram models enough l.jpg

Are n-gram models enough?

  • Can we make a list of (say) 3-grams that combine into all the grammatical sentences of English?

  • Ok, how about only the grammatical sentences?

  • How about all and only?


Can we avoid the systematic problems with n gram models l.jpg

Can we avoid the systematic problems with n-gram models?

  • Remembering things from arbitrarily far back in the sentence

    • Was the subject singular or plural?

    • Have we had a verb yet?

  • Formal language equivalent:

    • A language that allows strings having the forms a x* b and c x* d(x* means “0 or more x’s”)

    • Can we check grammaticality using a 50-gram model?

    • No? Then what can we use instead?


Finite state models l.jpg

Finite-state models

  • Regular expression:a x* b | c x* d

  • Finite-state acceptor:

x

Must remember whether first letter was a or c. Where does the FSA do that?

a

b

x

c

d


Context free grammars l.jpg

Context-free grammars

  • Sentence  Noun Verb Noun

  • S  N V N

  • N  Mary

  • V  likes

  • How many sentences?

  • Let’s add: N  John

  • Let’s add: V  sleeps, S  N V


Write a grammar of english l.jpg

What’s a grammar?

Write a grammar of English

Syntactic rules.

  • You have a week. 

  • 1S NP VP .

  • 1VP VerbT NP

  • 20NP Det N’

  • 1NP Proper

  • 20N’ Noun

  • 1N’ N’ PP

  • 1PP Prep NP


Now write a grammar of english l.jpg

Now write a grammar of English

Syntactic rules.

Lexical rules.

  • 1Noun castle

  • 1Noun king

  • 1Proper  Arthur

  • 1Proper  Guinevere

  • 1Det a

  • 1Det every

  • 1VerbT covers

  • 1VerbT rides

  • 1Misc that

  • 1Misc bloodier

  • 1Misc does

  • 1S NP VP .

  • 1VP VerbT NP

  • 20NP Det N’

  • 1NP Proper

  • 20N’ Noun

  • 1N’ N’ PP

  • 1PP Prep NP


Now write a grammar of english15 l.jpg

NP

VP

.

Now write a grammar of English

Here’s one to start with.

  • 1S NP VP .

  • 1VP VerbT NP

  • 20NP Det N’

  • 1NP Proper

  • 20N’ Noun

  • 1N’ N’ PP

  • 1PP Prep NP

S

1


Now write a grammar of english16 l.jpg

20/21

Det

N’

1/21

Now write a grammar of English

Here’s one to start with.

S

  • 1S NP VP .

  • 1VP VerbT NP

  • 20NP Det N’

  • 1NP Proper

  • 20N’ Noun

  • 1N’ N’ PP

  • 1PP Prep NP

NP

VP

.


Now write a grammar of english17 l.jpg

Det

N’

drinks [[Arthur [across

the [coconut in the castle]]]

[above another chalice]]

Noun

every

castle

Now write a grammar of English

Here’s one to start with.

S

  • 1S NP VP .

  • 1VP VerbT NP

  • 20NP Det N’

  • 1NP Proper

  • 20N’ Noun

  • 1N’ N’ PP

  • 1PP Prep NP

NP

VP

.


  • Login