language games and other meaningful pursuits l.
Skip this Video
Loading SlideShow in 5 Seconds..
Language Games and Other Meaningful Pursuits PowerPoint Presentation
Download Presentation
Language Games and Other Meaningful Pursuits

Loading in 2 Seconds...

play fullscreen
1 / 43

Language Games and Other Meaningful Pursuits - PowerPoint PPT Presentation

  • Uploaded on

Language Games and Other Meaningful Pursuits. Michael L. Littman AT&T Labs  Research (sort of). Motivation. Software that can understanding language. Question answering Constructing databases from text Natural-language interaction

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation

PowerPoint Slideshow about 'Language Games and Other Meaningful Pursuits' - Michelle

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
language games and other meaningful pursuits

Language Games andOther Meaningful Pursuits

Michael L. Littman

AT&T LabsResearch

(sort of)

Language Games


Software that can understanding language.

  • Question answering
  • Constructing databases from text
  • Natural-language interaction
  • Automatic summarization/briefing

Semantics tricky; word meanings informal.

Language Games

what is meaning
What is Meaning?

I don’t really like bananas, but have long respected their humorous potential.

How capture this sentence?

  • Predicate-argument structure?
  • Bag of words plus associated terms?

Evaluate by use.

Might need multiple reps. or none at all.

Can we make incremental progress?

Language Games

language games
Language Games

Like other games:

  • Evaluation process clean.
  • Challenging (and fun!) for people.

Unlike logical games:

  • Meaning matters!
  • No closed world assumption; messy.
  • Learning necessary... moving target.

Machine performance far from humans’.

Language Games

word games
Word Games

Super-human performance common:

  • Scrabble™: Maven, near-perfect (Sheppard 02)
  • Boggle™: millisecond solutions (Boyan 98)
  • Hangman (Littman 00)
    • 99.97% 9-letter words under 5 guesses
    • 1.35 misses on average

Language Games

language game examples
Language Game Examples
  • Crossword puzzles Acrostics
  • Cryptic crosswords Wheel of Fortune™
  • Trivial Pursuit™ Tribond™
  • College Bowl Taboo™
  • Who Wants to Be a Scattergories™ Millionaire™
  • Weakest Link™ Dictionary
  • Synonyms/Analogies

Language Games

Word meaning matters:
  • Pages semantically relevant to query

Multiple methods of determining relevance:

  • query words near each other on page
  • query words appear anywhere on page

Successful theme: combine

  • precision high, coverage low
  • precision low, coverage high

Language Games

combination rule
Combination Rule

Module set: coverage ci and precision pi

Let Rjbe the modules selecting response j.

  • Pick argmaxjPiin RjpiPiin URj’(1-pi)

If modules have independent precision:

  • Expected accuracy beats maxicipi.

Language Games

crossword puzzles shortz 98
Crossword Puzzles (Shortz 98)

ThesaurusCut off _ _ _ _ _ _ _ _

Puns & WordplayMonk’s head? _ _ _ _ _

Arts & Literature“Foundation

Trilogy” author _ _ _ _ _ _

Popular CulturePal of Pooh _ _ _ _ _ _

EncyclopedicMountain known

locally as Chomolungma _ _ _ _ _ _ _

CrosswordeseKind of coal or coat _ _ _







Language Games

p roverb system design
PROVERB: System Design

Candidate generation(Keim et al. 99)

  • Like information retrieval: clue implies target
  • Variety of approaches used simultaneously


  • Like meta search engine: create master list

Grid filling(Shazeer et al. 99)

  • Like constraint satisfaction: fit answer to grid
  • Solved via probabilistic optimization

Language Games

modules cluedb
Modules: ClueDB

Collection of published clues.

Nymph pursuer: SATYR Bugs pursuer: ELMER

Nymph chaser: SATYR Place for an ace: SLEEVE

Highball ingredient: RYEHighball ingredient: ICE

exact: Highball ingredient: RYE

partial: Ace place?: SLEEVE

TransModule: Bugs chaser: ELMER

X chaser Xpursuer

Language Games

modules special purpose
Modules: Special Purpose

Database modules: Transform clue to DB query.

imdb: Warner of Hollywood: OLAND

wordnet: Fruitless: ARID

Syntactic: Variations of fill-in-the-blanks.

blanks_movies: “Heavens ____!”: ABOVE

Web search: Not used in experimental system.

google: “The Way To Natural Beauty” author, 1980: TIEGS

Language Games

modules backstops
Modules: Backstops

Word lists: Ignore clue, return all words.

wordList: 10,000 words, perhaps: NOVELETTE

Implicit modules: Probability distributions over all strings of words (e.g., bigram).

segmenter: 1934 Hall and Nordhoff adventure novel: PITCAIRNISLAND

Language Games

grid filling
Grid Filling

Language Games

grid filling15
Grid Filling

Language Games

grid filling16
Grid Filling

Language Games

grid filling17
Grid Filling

Language Games

grid filling18
Grid Filling

Language Games

grid filling19
Grid Filling

Language Games

grid filling20
Grid Filling

Language Games

grid filling21
Grid Filling

Language Games

grid filling22
Grid Filling

Language Games

grid filling23
Grid Filling

Language Games

grid filling24
Grid Filling

Language Games

p roverb results

Test collection

  • 95% words, 98% letters, 46% puzzles
  • NYT: 89.5% (95.5% MTW, 85.0% TFSS)

American Crossword Puzzle Tournament

  • 1998: 190/251, 80% words (vs. 100%)
    • tricks: letter pairs, words in single square
  • 1999: 147/261, 75% words
    • tricks: cloonerized spues!

spoonerized clues!

Language Games

cryptic crosswords
Cryptic Crosswords

Popular in UK

A clue like this about Japanese religion (6)





about( HINT,SO)

Language Games

crossword maestro tunstall pedoe
Crossword Maestro (Tunstall-Pedoe)

Solves this clue:

`japanese religion' is the definition. I am not sure about the `japanese' bit but `Shinto' can be an answer for `religion'.

`a clue like this about' is the subsidiary indication. `a clue‘ becomes `hint', `like this' becomes `so', `about' means one lot of letters goes inside another. `hint' placed inside `so' is `shinto'.

Language Games

cm analysis
CM Analysis

CM “knows”:

  • Shinto is a religion (although not Japanese)
  • Back-off: “Japanese religion” kind of religion.
    • other classes: manmade objects, bodily activities, natural acts, verbs in base form, adjectives…
  • “a clue” is hint, “like this” is so
  • “about” indicates containment (game knowl.)


Language Games

cm results
CM Results

Ads claim 75% on individual clues (perhaps full puzzles).

New York Times puzzles, under 5%.

Very sophisticated:

  • NL explanations
  • broad coverage
  • real time

Language Games

question answering
Question Answering

Blend of IR and NLP.

  • query, tag type
  • find relevant passages
  • extract entities, sort by score (combination)

Language Games

who invented the first computer
Who invented the first computer?
  • The stunning shortsightedness and obtusity of XEROX's top-level suits has been well anatomized in “Fumbling The Future: How XEROX Invented, Then Ignored, the First Personal Computer” by Douglas K. Smith and Robert C. Alexander (William Morrow & Co., 1988, ISBN 0-688-09511-9).
  • 1876 Alexander Graham Bell invented the telephone. 1958 First electronic computers were built.
  • Steve Jobs and Steve Wozniak were two young men who invented the first-ready made computer, the Apple.
  • A Brief History of Computers and Data Communications. … 1820 Charles Babbage invented the difference engine … 1879 First private telephone switch board was installed.

Language Games

trivial pursuit
Trivial Pursuit™

Race around board, answer questions.

Categories: Geography, Entertainment, History, Literature, Science, Sports

Language Games


QA via AQUA (Abney et al. 00)

  • back off: word match in order helps score.
  • “When was Amelia Earhart's last flight?”
    • 1937, 1897 (birth), 1997 (reenactment)

Move selection via MDP (Littman 00)

  • Estimate category accuracy.
  • Minimize expected turns to finish.

Language Games

simulation results
Simulation Results

Language Games

toefl synonyms
TOEFL Synonyms

Used in college applications.


  • scale
  • angle
  • swim
  • dredge

Language Games

synonym approaches
Synonym Approaches

Latent Semantic Indexing (Landauer & Dumais 97)

  • Analyze 30k paragraphs, 300d embedding
  • 64% ~Boulder

Pointwise Mutual Information-IR (Turney 01)

  • Counts in Altavista (350M); 74% (us: 77%)


  • 98% prec.; 74% cov.
  • Combine with PMI-IR: 92%

Language Games

verbal analogies
Verbal Analogies

Used in college boards (SATs, GREs), and as an intelligence test.

cat : meow ::

  • mouse : scamper
  • bird : peck
  • dog : bark
  • horse : groom
  • lion : scratch

Language Games

preliminary results
Preliminary Results

Undergraduate project at Princeton

  • 17 groups
  • 229 examples (ETS, web, class)
  • 76 test problems from SATs

Language Games

analogy results sats
Analogy Results: SATs

Three groups sig. better than chance.

Language Games

some successful techniques
Some Successful Techniques

Individual groups used combinations:

  • Wordnet for synonyms, antonyms, is-a
  • web search with “linking words”
      • x (characterized by OR possesses OR possessed OR possess OR characterizes) y
  • automatically identify linking words
  • word association in definitions

Language Games


Language games interesting, challenging.

Progress on solving “meaning” problems.

  • Word order
  • Words, as a backoff
  • Meaning-preserving transformations
  • Probabilistic models
  • Combinations!

Exciting learning and reasoning to be done

Language Games


What year was Dartmouth College founded?

Dr. Koop graduated from DartmouthCollege in 1937 and received his M.D. degree from Cornell Medical College in 1941.

Although DartmouthCollege was established in 1769 to educate American Indians and…

Language Games