random indexing on space and meaning l.
Skip this Video
Loading SlideShow in 5 Seconds..
Random indexing: On space and meaning PowerPoint Presentation
Download Presentation
Random indexing: On space and meaning

Loading in 2 Seconds...

play fullscreen
1 / 28

Random indexing: On space and meaning - PowerPoint PPT Presentation

  • Uploaded on

Random indexing: On space and meaning. Simon Belak. Order of the day. Meaning Philosophy Neuroscience Computer science Space Words as points in space On dimensionality Random indexing. What’s the meaning of meaning ?. Philosophers say:. “Meaning just is use.” – Wittgenstein.

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation

Random indexing: On space and meaning

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
order of the day
Order of the day
  • Meaning
    • Philosophy
    • Neuroscience
    • Computer science
  • Space
    • Words as points in space
    • On dimensionality
  • Random indexing
philosophers say
Philosophers say:

“Meaning just is use.”

– Wittgenstein

neuroscientists say
Neuroscientists say:
  • Episodic memory  semantic memory

(concrete event  abstract concept)

  • Hebbian process
computer scientists say
Computer scientists say:


semantic networks






  • Co-occurrences
  • Hebbian process
    • Self-organisation
    • Clustering
  • Evolution of language
    • Coach(Kocs carriage  train  car)
problem homonym s
Problem: homonyms


1.a. An article of furniture supported by one or more vertical legs and having a flat horizontal surface.

b. The objects laid out for a meal on this article of furniture.

2. The food and drink served at meals; fare: kept an excellent table.

3. The company of people assembled around a table, as for a meal.

4A plateau or tableland.

5. a. A flat facet cut across the top of a precious stone.

b. A stone or gem cut in this fashion.

6. Music

a.The front part of the body of a stringed instrument.

b.The sounding board of a harp.

7. Architecture

a.A raised or sunken rectangular panel on a wall.

b.A raised horizontal surface or continuous band on an exterior wall; a stringcourse.

8. A part of the human palm framed by four lines, analyzed in palmistry.

9. An orderly arrangement of data, especially one in which the data are arranged in columns and rows in an essentially rectangular form.

10. An abbreviated list, as of contents; a synopsis.

11. An engraved slab or tablet bearing an inscription or a device.

12. Anatomy The inner or outer flat layer of bones of the skull separated by the dipole.

solution high dimensionality
Solution: high dimensionality
  • One dimension per word
  • Tableextends into food, furniture, music,... dimensions
problem synonyms
Problem: synonyms

amazing, stupefying, staggering, awesome, awful,awe-inspiring,awing,astonishing, astounding

solution latent meaning
Solution: latent meaning
  • Reduced dimensionality
  • Closely related words fold into one
  • “Higher-order” meaning
the idea
The idea
  • Word is the sum of it’s contexts
  • Context is the sum of it’s words
  • Grounding?
the algorithm
The algorithm
  • Take a context of words
  • Generate a context index vector
  • Add index to all the word vectors
  • Go to 1)

Episodic memory (2) + Hebbian process (3)

dimensionality reduction
Dimensionality reduction
  • Sparse high-dimensional ternary index

(a small number of randomly distributed +1s and -1s)

  • Nearly orthogonal
    • Distances approximately preserved
the good
The good
  • Fast, scalable
  • Trivially parallelised
    • Per word
    • Addition is associative, commutative
  • Stable
    • Words are independent
    • Integer arithmetics
  • Incremental
the bad
The bad
  • Memory hungry
    • Caching (Zipf’s law)
  • Comparing words to words
    • Query expnasion
  • Comparing documents to documents
    • Clustering
    • Search
    • Recomendations
  • Comparing documents to words
    • Keyword extraction
4 random indexing
4. Random indexing
  • Cognitive rationale
  • Simple
  • Fast, scalable
key points26
Key points
  • Meaning is use
  • Words in space
  • Many meanings, many dimensions
  • Random indexing
    • Cognitive rationale
    • Simple
    • Fast, scalable
  • http://www.sics.se/~mange/papers/KarlgrenSahlgren2001.pdf
  • http://www.kfs.org/~jonathan/witt/tlph.html
  • http://www.mtsu.edu/~sschmidt/Cognitive/semantic/semantic.html
  • http://memory.syr.edu/marc/papers/HowaAddiJingKaha-LSAChap-doc.pdf
  • http://memory.psych.upenn.edu/research/research_episodic_memory.php
  • http://www.rni.org/kanerva/cogsci2k-poster.txt
  • http://www.sics.se/~mange/papers/RI_intro.pdf
  • http://code.google.com/p/cl-random-indexing