Language games and other meaningful pursuits
Download
1 / 43

Language Games and - PowerPoint PPT Presentation


  • 359 Views
  • Updated On :

Language Games and Other Meaningful Pursuits. Michael L. Littman [email protected] AT&T Labs  Research (sort of). Motivation. Software that can understanding language. Question answering Constructing databases from text Natural-language interaction

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Language Games and' - Michelle


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Language games and other meaningful pursuits l.jpg

Language Games andOther Meaningful Pursuits

Michael L. Littman

[email protected]

AT&T LabsResearch

(sort of)

Language Games


Motivation l.jpg
Motivation

Software that can understanding language.

  • Question answering

  • Constructing databases from text

  • Natural-language interaction

  • Automatic summarization/briefing

    Semantics tricky; word meanings informal.

Language Games


What is meaning l.jpg
What is Meaning?

I don’t really like bananas, but have long respected their humorous potential.

How capture this sentence?

  • Predicate-argument structure?

  • Bag of words plus associated terms?

    Evaluate by use.

    Might need multiple reps. or none at all.

    Can we make incremental progress?

Language Games


Language games l.jpg
Language Games

Like other games:

  • Evaluation process clean.

  • Challenging (and fun!) for people.

    Unlike logical games:

  • Meaning matters!

  • No closed world assumption; messy.

  • Learning necessary... moving target.

    Machine performance far from humans’.

Language Games


Word games l.jpg
Word Games

Super-human performance common:

  • Scrabble™: Maven, near-perfect (Sheppard 02)

  • Boggle™: millisecond solutions (Boyan 98)

  • Hangman (Littman 00)

    • 99.97% 9-letter words under 5 guesses

    • 1.35 misses on average

Language Games


Language game examples l.jpg
Language Game Examples

  • Crossword puzzles Acrostics

  • Cryptic crosswords Wheel of Fortune™

  • Trivial Pursuit™ Tribond™

  • College Bowl Taboo™

  • Who Wants to Be a Scattergories™ Millionaire™

  • Weakest Link™ Dictionary

  • Synonyms/Analogies

Language Games


Slide7 l.jpg

Word meaning matters:

  • Pages semantically relevant to query

    Multiple methods of determining relevance:

  • query words near each other on page

  • query words appear anywhere on page

    Successful theme: combine

  • precision high, coverage low

  • precision low, coverage high

Language Games


Combination rule l.jpg
Combination Rule

Module set: coverage ci and precision pi

Let Rjbe the modules selecting response j.

  • Pick argmaxjPiin RjpiPiin URj’(1-pi)

    If modules have independent precision:

  • Expected accuracy beats maxicipi.

Language Games


Crossword puzzles shortz 98 l.jpg
Crossword Puzzles (Shortz 98)

ThesaurusCut off _ _ _ _ _ _ _ _

Puns & WordplayMonk’s head? _ _ _ _ _

Arts & Literature“Foundation

Trilogy” author _ _ _ _ _ _

Popular CulturePal of Pooh _ _ _ _ _ _

EncyclopedicMountain known

locally as Chomolungma _ _ _ _ _ _ _

CrosswordeseKind of coal or coat _ _ _

ISOLATED

ABBOT

ASIMOV

TIGGER

EVE REST

PEA

Language Games


P roverb system design l.jpg
PROVERB: System Design

Candidate generation(Keim et al. 99)

  • Like information retrieval: clue implies target

  • Variety of approaches used simultaneously

    Merging

  • Like meta search engine: create master list

    Grid filling(Shazeer et al. 99)

  • Like constraint satisfaction: fit answer to grid

  • Solved via probabilistic optimization

Language Games


Modules cluedb l.jpg
Modules: ClueDB

Collection of published clues.

Nymph pursuer: SATYR Bugs pursuer: ELMER

Nymph chaser: SATYR Place for an ace: SLEEVE

Highball ingredient: RYEHighball ingredient: ICE

exact: Highball ingredient: RYE

partial: Ace place?: SLEEVE

TransModule: Bugs chaser: ELMER

X chaser Xpursuer

Language Games


Modules special purpose l.jpg
Modules: Special Purpose

Database modules: Transform clue to DB query.

imdb: Warner of Hollywood: OLAND

wordnet: Fruitless: ARID

Syntactic: Variations of fill-in-the-blanks.

blanks_movies: “Heavens ____!”: ABOVE

Web search: Not used in experimental system.

google: “The Way To Natural Beauty” author, 1980: TIEGS

Language Games


Modules backstops l.jpg
Modules: Backstops

Word lists: Ignore clue, return all words.

wordList: 10,000 words, perhaps: NOVELETTE

Implicit modules: Probability distributions over all strings of words (e.g., bigram).

segmenter: 1934 Hall and Nordhoff adventure novel: PITCAIRNISLAND

Language Games


Grid filling l.jpg
Grid Filling

Language Games


Grid filling15 l.jpg
Grid Filling

Language Games


Grid filling16 l.jpg
Grid Filling

Language Games


Grid filling17 l.jpg
Grid Filling

Language Games


Grid filling18 l.jpg
Grid Filling

Language Games


Grid filling19 l.jpg
Grid Filling

Language Games


Grid filling20 l.jpg
Grid Filling

Language Games


Grid filling21 l.jpg
Grid Filling

Language Games


Grid filling22 l.jpg
Grid Filling

Language Games


Grid filling23 l.jpg
Grid Filling

Language Games


Grid filling24 l.jpg
Grid Filling

Language Games



P roverb results l.jpg
PROVERB Results

Test collection

  • 95% words, 98% letters, 46% puzzles

  • NYT: 89.5% (95.5% MTW, 85.0% TFSS)

    American Crossword Puzzle Tournament

  • 1998: 190/251, 80% words (vs. 100%)

    • tricks: letter pairs, words in single square

  • 1999: 147/261, 75% words

    • tricks: cloonerized spues!

spoonerized clues!

Language Games


Cryptic crosswords l.jpg
Cryptic Crosswords

Popular in UK

A clue like this about Japanese religion (6)

SHINTO

HINT

SO

=

about( HINT,SO)

Language Games


Crossword maestro tunstall pedoe l.jpg
Crossword Maestro (Tunstall-Pedoe)

Solves this clue:

`japanese religion' is the definition. I am not sure about the `japanese' bit but `Shinto' can be an answer for `religion'.

`a clue like this about' is the subsidiary indication. `a clue‘ becomes `hint', `like this' becomes `so', `about' means one lot of letters goes inside another. `hint' placed inside `so' is `shinto'.

http://www.genius2000.com/cm.html

Language Games


Cm analysis l.jpg
CM Analysis

CM “knows”:

  • Shinto is a religion (although not Japanese)

  • Back-off: “Japanese religion” kind of religion.

    • other classes: manmade objects, bodily activities, natural acts, verbs in base form, adjectives…

  • “a clue” is hint, “like this” is so

  • “about” indicates containment (game knowl.)

    Wordnet? http://www.cogsci.princeton.edu/~wn/

Language Games


Cm results l.jpg
CM Results

Ads claim 75% on individual clues (perhaps full puzzles).

New York Times puzzles, under 5%.

Very sophisticated:

  • NL explanations

  • broad coverage

  • real time

Language Games


Question answering l.jpg
Question Answering

Blend of IR and NLP.

  • query, tag type

  • find relevant passages

  • extract entities, sort by score (combination)

Language Games


Who invented the first computer l.jpg
Who invented the first computer?

  • The stunning shortsightedness and obtusity of XEROX's top-level suits has been well anatomized in “Fumbling The Future: How XEROX Invented, Then Ignored, the First Personal Computer” by Douglas K. Smith and Robert C. Alexander (William Morrow & Co., 1988, ISBN 0-688-09511-9).

  • 1876 Alexander Graham Bell invented the telephone. 1958 First electronic computers were built.

  • Steve Jobs and Steve Wozniak were two young men who invented the first-ready made computer, the Apple.

  • A Brief History of Computers and Data Communications. … 1820 Charles Babbage invented the difference engine … 1879 First private telephone switch board was installed.

Language Games


Trivial pursuit l.jpg
Trivial Pursuit™

Race around board, answer questions.

Categories: Geography, Entertainment, History, Literature, Science, Sports

Language Games


Wigwam l.jpg
Wigwam

QA via AQUA (Abney et al. 00)

  • back off: word match in order helps score.

  • “When was Amelia Earhart's last flight?”

    • 1937, 1897 (birth), 1997 (reenactment)

      Move selection via MDP (Littman 00)

  • Estimate category accuracy.

  • Minimize expected turns to finish.

Language Games


Simulation results l.jpg
Simulation Results

Language Games


Toefl synonyms l.jpg
TOEFL Synonyms

Used in college applications.

fish

  • scale

  • angle

  • swim

  • dredge

Language Games


Synonym approaches l.jpg
Synonym Approaches

Latent Semantic Indexing (Landauer & Dumais 97)

  • Analyze 30k paragraphs, 300d embedding

  • 64% ~Boulder http://lsa.colorado.edu/

    Pointwise Mutual Information-IR (Turney 01)

  • Counts in Altavista (350M); 74% (us: 77%)

    Thesaurus

  • http://Wordsmyth.net: 98% prec.; 74% cov.

  • Combine with PMI-IR: 92%

Language Games


Verbal analogies l.jpg
Verbal Analogies

Used in college boards (SATs, GREs), and as an intelligence test.

cat : meow ::

  • mouse : scamper

  • bird : peck

  • dog : bark

  • horse : groom

  • lion : scratch

Language Games


Preliminary results l.jpg
Preliminary Results

Undergraduate project at Princeton

  • 17 groups

  • 229 examples (ETS, web, class)

  • 76 test problems from SATs

Language Games


Analogy results sats l.jpg
Analogy Results: SATs

Three groups sig. better than chance.

Language Games


Some successful techniques l.jpg
Some Successful Techniques

Individual groups used combinations:

  • Wordnet for synonyms, antonyms, is-a

  • web search with “linking words”

    • x (characterized by OR possesses OR possessed OR possess OR characterizes) y

  • automatically identify linking words

  • word association in definitions

  • Language Games


    Conclusion l.jpg
    Conclusion

    Language games interesting, challenging.

    Progress on solving “meaning” problems.

    • Word order

    • Words, as a backoff

    • Meaning-preserving transformations

    • Probabilistic models

    • Combinations!

      Exciting learning and reasoning to be done

    Language Games


    Challenges l.jpg
    Challenges

    What year was Dartmouth College founded?

    Dr. Koop graduated from DartmouthCollege in 1937 and received his M.D. degree from Cornell Medical College in 1941.

    Although DartmouthCollege was established in 1769 to educate American Indians and…

    Language Games


    ad