Natural language processing
Download
1 / 50

Natural Language Processing - PowerPoint PPT Presentation


  • 41 Views
  • Uploaded on

Natural Language Processing. what it does what is involved why is it difficult brief history. sentence  structured rep n s of meaning. "how old is my help3.doc file?" Lisp: (query (file-detail 'date "C:/help3.doc")) "the large cat chased the rat"

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about ' Natural Language Processing' - lily


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Natural language processing
Natural Language Processing

  • what it does

  • what is involved

  • why is it difficult

  • brief history


Sentence structured rep n s of meaning
sentence  structured repns of meaning

"how old is my help3.doc file?"

Lisp: (query (file-detail 'date "C:/help3.doc"))

"the large cat chased the rat"

Logic: (1s1large(s1)  feline(s1))

 (1s2rodent(s2))  chased(s1, s2 )

"the young boy ate a bad apple"

CD Graph ...see next page...


Cd graph the young boy ate a bad apple
CD graph "the young boy ate a bad apple"


What is involved
what is involved

symbolic computationie: symbols manipulated by symbol processors

search & inference

knowledge representation techniques


why is it difficult

prejudice, politics, etc

ambiguity...

  • syntactic

  • semantic

  • pragmatic


example sentences

  • the old man the boats

  • my car drinks petrol

  • I saw the Eiffel Tower flying to Paris

  • he opened the door with the keyhe opened the door with the squeaking hinge

  • the boy kicked the ball under the treethe boy kicked the wall under the tree

  • put the bottles in the box on the shelf by the door


(brief) history of language processing

1950s Russian  English translation

1956 Chomsky

1960s Pattern matching

1970s Parsing & some KnRep

1980s Kn & inference

1990s big dreams small results

2000+ quietly promising



Matching student
matching:Student


Matching elisa
matching:Elisa


A modern approach

input

sentence

a modernapproach

morphological

processing

lexicon

syntax

analysis

(parsing)

grammar

semantic

rules

semantic

analysis

pragmatic

analysis

contextual

information

target

representation


Step 1 morphological processing
step 1- morphological processing

objective: strip words into roots & modifiers

issues

  • inflection (cat pl cat-s)

  • derivation (happy adj happiness noun)

  • compounding (toothpaste)


Morphological processing notes
morphological processing - notes

  • all(?) spoken lngs exhibit morphology

  • easier to handle in written lngs if not iconic

  • some morphology describes infm beyond syntax

    eg: proximity (Tamil, Setswana, etc)

    case

    speaker / listener peer relationship



Step 2 syntax analysis
step 2- syntax analysis

objectives: 1 check for correctness

2 produce phrase structure

uses

  • parser a rule-based search engine

  • grammar context-free production rules

  • lexicon dictionary of words & their categories


Syntax rules
syntax rules

parts of speech

rules of combination

consider

  • the cat chases the mouse

  • all large black dogs chase cats


Example 1 using lkit
example 1 - using Lkit

(build-lexicon

'((a determiner) (cat noun) (dog noun)

(the determiner) (chased verb)))

(build-grammar

'((s1 (sentence -> noun-phrase verb-phrase))

(np (noun-phrase -> determiner noun))

(vp (verb-phrase -> verb noun-phrase))

))


Example 1 output
example 1 - output

(parse 'sentence '(the dog chased a cat))

complete-edge 0 5 s1 sentence (the dog ...) nil

s1 sentence -> (noun-phrase verb-phrase)

Syntax

(sentence

(noun-phrase (determiner the) (noun dog))

(verb-phrase

(verb chased)

(noun-phrase (determiner a) (noun cat))))

Semantics

(sentence)


So what we want meaning

so what ?we want meaning


Remember the young boy ate a bad apple
Remember: "the young boy ate a bad apple"

how can semantics be encoded as symbols?

the boy / an apple?

young/old, happy/sad, good/bad?

how can semantics be generated?

what can be inferred from semantics?


Reminder the young boy ate a bad apple
Reminder: "the young boy ate a bad apple"


Symbolic representation of semantics
symbolic representation of semantics

(actor (root boy) (id boy#732)

(tags animate human male)

(qual (age (val 5) (approx 3)))

(quant specific))

(action (primitve INGEST))

(object (root apple) (id nil)

(tags physob veg fruit food)

(qual (phy-state -4))

(quant non-specific))


Semantics in lexicon
semantics in lexicon

a simple example

(build-lexicon

'((a det any )

(cat noun feline )

(chased verb hunts )

(dog noun canine )

(the det specific)

))


Semantics in grammar rules
semantics in grammar rules

(s1 (sentence -> noun-phrase verb-phrase)

(actor . noun-phrase)

(action . verb-phrase.action)

(object . verb-phrase.object)

)

(np (noun-phrase -> det noun)

(det . noun)

)

(vp (verb-phrase -> verb noun-phrase)

(action . verb)

(object . noun-phrase)

)


Semantics results
semantics - results

(parse 'sentence '(the dog chased a cat))

complete-edge 0 5 s1 sentence (the dog...) nil

s1 sentence -> (noun-phrase verb-phrase)

Syntax

(sentence (noun-phrase (det the) (noun dog))

(verb-phrase (verb chased)

(noun-phrase (det a) (noun cat))))

Semantics

(sentence (actor (specific canine))

(action hunts)

(object (any feline)))


Semantics in lexicon checks 1
semantics in lexicon - checks 1

(a det (sems . any))

(all det (sems . every))

(cat noun (sems . feline) (num . sing))

(cats noun (sems . feline) (num . plur))

(chase verb (sems . hunts) (num . plur))

(chases verb (sems . hunts) (num . sing))

(dog noun (sems . canine) (num . sing))

(dogs noun (sems . canine) (num . plur))

(the det (sems . specific))


Semantics in grammar checks 1
semantics in grammar - checks 1

(s1 (sentence -> noun-phrase verb-phrase)

(actor . noun-phrase.sems)

(action . verb-phrase.action)

(object . verb-phrase.object)

; check number of noun-phrase & verb-phrase

(if (noun-phrase.number

= verb-phrase.number)

numeric-agreement-ok

numeric-agreement-bad

)

)


Semantics results1
semantics - results

(parse 'sentence '(the dog chases a cat))

complete-edge 0 5 s1 sentence (the dog...) nil

s1 sentence -> (noun-phrase verb-phrase)

Syntax

(sentence (noun-phrase (det the) (noun dog))

(verb-phrase (verb chases)

(noun-phrase (det a) (noun cat))))

Semantics

(sentence (actor specific canine)

(action . hunts)

(object any feline)

numeric-agreement-ok)


Semantics results2
semantics - results

(parse 'sentence '(the dogs chases a cat))

complete-edge 0 5 s1 sentence (the dog...) nil

s1 sentence -> (noun-phrase verb-phrase)

Syntax

(sentence (noun-phrase (det the) (noun dog))

(verb-phrase (verb chases)

(noun-phrase (det a) (noun cat))))

Semantics

(sentence (actor specific canine)

(action . hunts)

(object any feline)

numeric-agreement-bad)


Semantics in grammar checks 2
semantics in grammar - checks 2

(s1 (sentence -> noun-phrase verb-phrase)

(fail if noun-phrase.number

/= verb-phrase.number)

(actor . noun-phrase.sems)

(action . verb-phrase.action)

(object . verb-phrase.object)

)


Semantics results3
semantics - results

(parse 'sentence '(the dog chases a cat))

Semantics

(sentence (actor specific canine)

(action . hunts)

(object any feline))

(parse 'sentence '(the dogs chases a cat))

.... failed ....


Semantics in grammar checks 3
semantics in grammar - checks 3

(s1 (sentence -> noun-phrase verb-phrase)

(glitch numeric-agreement

if not noun-phrase.number

= verb-phrase.number)

(actor . noun-phrase.sems)

(action . verb-phrase.action)

(object . verb-phrase.object)

)


Semantics results4
semantics - results

(parse 'sentence '(the dogs chases a cat))

complete-edge 0 5 s1 sentence (the dogs...) nil

Glitches: (numeric-agreement)

s1 sentence -> (noun-phrase verb-phrase)

Syntax

(sentence (noun-phrase (det the) (noun dogs))

(verb-phrase (verb chases)

(noun-phrase (det a) (noun cat))))

Semantics

(sentence (actor specific canine)

(action . hunts)

(object any feline))


Example 2 lexicon
example 2 - lexicon

(a det any )

(cat noun feline )

(chase verb hunts )

(dog noun canine )

(the det specific)

(black adj (color black))

(large adj (size 7/10))

(small adj (size 3/10))


Example 2 grammar
example 2 - grammar

(build-grammar

'((np (noun-phrase -> ?det *adj noun)

(if det

(quantification . det)

(quantification undefined))

(qualifiers . *.adj)

(object . noun)

))

))


Example 2 results
example 2 - results

(parse 'noun-phrase '(small black dog))

complete-edge 0 3 np noun-phrase (small...) nil

np noun-phrase -> (?det *adj noun)

Syntax

(noun-phrase (adj small) (adj black) (noun dog))

Semantics

(noun-phrase

(quantification undefined)

(qualifiers ((size . 3/10)) ((color . black)))

(object canine))


Example 2 results1
example 2 - results

small dogs chase the small cats

and large dogs chase the large cats

(sentence conjunction

((actor (quant undefined) (qual (size . 3/10))

(object . canine))

(action . hunts)

(object (quant . specific) (qual (size . 3/10))

(object . feline)))

((actor (quant undefined) (qual (size . 7/10))

(object . canine))

(action . hunts)

(object (quant . specific) (qual (size . 7/10))

(object . feline))))


Semantic processing one approach
semantic processing (one approach)

  • semantic rules in grammar  1st stage case frame

  • verb form  primitive action case frame

  • disambiguate & fill additional case frame slots

  • check references with world and/or dialog

  • do statement level inference

  • integrate with dialog

  • do event sequence dialog


Step 1 produce raw case frame
step-1: produce raw case frame

  • verb cases

    the cat chased the rat in the kitchen

    the cat chased the rat into the kitchen

  • common cases

    source start-time instrument

    destination end-time beneficiary

    location duration


The ambiguity problem
the ambiguity problem

eg: the boy kicked the ball under the tree

grammar rules

S  S PP

S  NP VP

NP  ?det *adj noun

NP  NP PP


Example frame 1
example frame #1

actor (quant specific)

(tags animate male human)

(qual (age (range 3 13)))

(root boy)

action (root kick)

object (root ball)

(tags manip)

(posn-relative

(locator beneath)

(object (root tree)

...etc... )


Example frame 2
example frame #2

actor (quant specific)

(tags animate male human)

(qual (age (range 3 13)))

(root boy)

action (root kick)

object (root ball)

(tags manip)

dest (posn-relative

(locator beneath)

(object (root tree)

...etc... )


Example verb form 1
example verb form #1

primitive strike

prohibited object (tags manip)

slots instrument (part-of $actor foot)

legal start-time, end-time, duration

instrument, beneficiary, location

illegal source, dest


Example verb form 2
example verb form #2

primitive push

required object (tags manip)

slots instrument (part-of $actor foot)

legal source, dest, start-time, end-time,

instr, beneficiary, locatn, duration


Semantic processing one approach1
semantic processing (one approach)

  • semantic rules in grammar  1st stage case frame

  • verb form  primitive action case frame

  • disambiguate & fill additional case frame slots

  • check references with world and/or dialog

  • do statement level inference

  • integrate with dialog

  • do event sequence dialog


Integration with dialog
integration with dialog

dialogs have...

  • players (actors)

  • props (objects)

  • locations (from case frames)

  • themes (derived)

  • event sequences (from themes)

  • plans (from themes and/or derived)


Event sequence
event sequence

set of...

  • players (actors)

  • props (objects)

    series of...

  • semantically encoded activities (matched)

  • escapes, exceptions & alternatives


Reading grammars etc
reading – grammars, etc

 A good source of links & references...

“Computational Analysis of Prepositions” http://knol.google.com/k/abdul-baqi-sharaf/computational-analysis-of-prepositions/3hc3uny2z7r41/4#

if you only plan to read one article...

Baldwin, T. Kordoni, V and Villavicencio, A. 2009. Prepositions in Applications: A Survey and Introduction to the Special Issue ". Computational Linguistics 35 (2): 119–149.

also...

Litkowski, Kenneth C. and Orin Hargraves. 2007. SemEval-2007 task 06: Word-sense disambiguation of prepositions. In Proceedings of the 4th International Workshop on Semantic Evaluations, pages 24–29, Prague.

Disambiguation of Preposition Sense Using Linguistically Motivated

Features, Stephen Tratz and Dirk Hovy. Proceedings of the NAACL HLT Student Research Workshop and Doctoral Consortium, pages 96–100,

Boulder, Colorado, June 2009. c 2009 Association for Computational Linguistics


Reading grammars etc1
reading – grammars, etc

 the NLP dictionary: www.cse.unsw.edu.au/~billw/nlpdict.html

for practical help with building grammars check the following (it is about 10 years old but then so is the English language :o)

A Grammar Writer’s Cookbook. Miriam Butt, Tracy Holloway King, Marma-Eugenia Niño and Fridirique Segond

also (for writing larger grammars) it is useful to find a book on grammar for tutors and/or students of English as a second language.

for a broad (if a little formal) take on semantics try dipping into...

Semantics-Oriented Natural Language Processing Mathematical Models and Algorithms. Vladimir Fomichov A. 2010


Reading kn rep for nlp
reading – kn rep for NLP

logic and knowledge representation – a guide

http://dspace.dsto.defence.gov.au/dspace/bitstream/1947/9996/1/DSTO-TR-2324%20PR.pdf

representing events for NLP

http://www.google.co.uk/url?sa=t&rct=j&q=knowledge%20representation%20%22representing%20events%22&source=web&cd=6&sqi=2&ved=0CEgQFjAF&url=http%3A%2F%2Fwww.aaai.org%2Focs%2Findex.php%2FFSS%2FFSS10%2Fpaper%2Fdownload%2F2183%2F2819&ei=f6oWT_e7DeKC4gTMpaijBA&usg=AFQjCNFYmurwJR9oqfCRBimVprWRK45kew&cad=rja

semantic networks & frames (2005)

http://www.cs.bham.ac.uk/~jxb/IAI/w6.pdf

VERL: An Ontology Framework for Video Events (2005)

http://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=1524892


ad