Indexing by latent semantic analysis
1 / 22

Indexing by Latent Semantic Analysis - PowerPoint PPT Presentation

  • Uploaded on

Indexing by Latent Semantic Analysis. Written by Deerwester, Dumais, Furnas, Landauer, and Harshman (1990) Reviewed by Cinthia Levy. Latent Semantic Indexing. Term-matching Most retrieval systems match words of a query (keywords) with words of a document. Problem

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation

PowerPoint Slideshow about ' Indexing by Latent Semantic Analysis' - rayya

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Indexing by latent semantic analysis

Indexing by Latent Semantic Analysis

Written by Deerwester, Dumais, Furnas, Landauer, and Harshman


Reviewed by Cinthia Levy

Latent semantic indexing
Latent Semantic Indexing


Most retrieval systems match words of a query (keywords) with words of a document.


What if users want to retrieve information based upon conceptual content?

Latent semantic indexing1
Latent Semantic Indexing

Expressing a concept in keywords is

complicated and unreliable

  • Synonymy: many ways to define a concept. Results in ‘poor recall’.

  • Polysemy: most words have multiple meanings. Results in ‘poor precision’.

Latent semantic indexing2
Latent Semantic Indexing

Three factors contribute to the failure that IR systems have in overcoming problems associated w/synonymy & polysemy:

  • Identification of index terms is incomplete

  • No automatic method adequately addresses polysemy

  • Technical: the way current IR systems work

Latent semantic indexing3
Latent Semantic Indexing

Goal build an IR system that predicts what terms “really” are implied by a query or what terms “really” apply to a document (i.e. the latent semantics).

Latent semantic indexing4
Latent Semantic Indexing

Choosing a model

Proximity model: similar items are put near each other

in some space or structure.

Latent semantic indexing5
Latent Semantic Indexing

Existing proximity models include:

  • Hierarchical, partition & overlapping clusterings

  • Ultrametric & additive trees

  • Factor-analytic & multidimensional distance models

Latent semantic indexing6
Latent Semantic Indexing

Alternate model was considered, based on the following criteria:

  • Adjustable representational richness

  • Explicit representation of both terms and documents

  • Computational tractability for large datasets

Latent semantic indexing7
Latent Semantic Indexing

Singular value decomposition (SVD)

or two-mode factor analysis,

satisfied all three criteria!

SVD: a fully automatic statistical method used to determine associations among terms in a large document collection, and to create a semantic or concept space.

Latent semantic indexing8
Latent Semantic Indexing

Basis of LSI:

  • Documents are condensed to contain only “content words” w/semantic meaning

  • Patterns of word distribution (co-occurrence) are analyzed across a collection of documents.

Latent semantic indexing9
Latent Semantic Indexing

Basis of LSI:

  • Document collection is examined as a whole

    • Documents with many words in common are semantically close.

    • Documents with few words in common are semantically distant.

Latent semantic indexing10
Latent Semantic Indexing

Steps of LSI:

  • Format document: stop words removed, punctuation removed, no capitalization.

  • Select content words: words with no semantic value are removed using stop list.

  • Apply Stemming*: reduces words to root form.

    *(not applied in Deerwester, et al.)

Latent semantic indexing11
Latent Semantic Indexing

Result: List of content words

The list of content words is used to generate a

term-document matrix.

Latent semantic indexing12
Latent Semantic Indexing

Term-document matrix

Latent semantic indexing13
Latent Semantic Indexing

Term-document matrix:

  • Term weighting* is applied to each value

  • SVD algorithm is applied to the matrix

  • Matrix represents vectors in a multi-dimensional space

    *(not applied in Deerwester, et al.)

Latent semantic indexing14
Latent Semantic Indexing

Visual representation of a three-dimensional space:

Content words form three orthogonal axes (mutually perpendicular)




Latent semantic indexing15
Latent Semantic Indexing

“If you draw a line from the origin of the graph to each of these points, you obtain a set of vectors in 'bacon-eggs-and-coffee' space. The size and direction of each vector tells you how many of the three key items were in any particular order, and the set of all the vectors taken together tells you something about the kind of breakfast people favor on a Saturday morning.”

Retrieved from:

Latent semantic indexing16
Latent Semantic Indexing

Retrieved from

Latent semantic indexing17
Latent Semantic Indexing

Romans 1:22 Professing themselves to be wise, they became fools…

Romans 16:6 Greet Mary, who bestowed much labour on us.

Matthew 24:22 And except those days should be shortened, there should no flesh be saved: but for the elect's sake those days shall be shortened.

John 3:17 For God sent not his Son into the world to condemn the world; but

that the world through him might be saved.

Latent semantic indexing18
Latent Semantic Indexing


System compared to:

  • Straight term matching

  • Voorhees



    1. collection of medical abstracts (MED)

    2. information science abstracts (CISI)

Latent semantic indexing19
Latent Semantic Indexing

Summary of analyses

  • LSI performed better than or equal to simple term matching

  • LSI was shown to be superior to system described by Voorhees

  • LSI performed better than or equal to SMART

Latent semantic indexing20
Latent Semantic Indexing


  • LSI represents both terms and documents in the same space which provides for the retrieval of relevant information.

  • LSI does not rely on literal matching thus retrieves more relevant information than other methods.

  • LSI offers an adequate solution to the problem of synonymy but only a partial solution to the problem of polysemy.