information retrieval models
Download
Skip this Video
Download Presentation
Information Retrieval Models

Loading in 2 Seconds...

play fullscreen
1 / 19

Information Retrieval Models - PowerPoint PPT Presentation


  • 157 Views
  • Uploaded on

Information Retrieval Models. Retrieval Models. A retrieval model specifies the details of: Document representation Query representation Retrieval function Determines a notion of relevance. Notion of relevance can be binary or continuous (i.e. ranked retrieval ).

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Information Retrieval Models' - adanne


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
retrieval models
Retrieval Models
  • A retrieval model specifies the details of:
    • Document representation
    • Query representation
    • Retrieval function
  • Determines a notion of relevance.
  • Notion of relevance can be binary or continuous (i.e. ranked retrieval).
classes of retrieval models
Classes of Retrieval Models
  • Boolean models (set theoretic)
    • Extended Boolean
  • Vector space models (statistical/algebraic)
    • Generalized VS
    • Latent Semantic Indexing
  • Probabilistic models
other model dimensions
Other Model Dimensions
  • User Task
    • Retrieval
    • Browsing
  • Logical View of Documents
    • Index terms
    • Full text
    • Full text + Structure (e.g. hypertext)
slide5
Retrieval

Database

Browsing

Retrieval and browsing

  • The User Task
  • Retrieval
      • information or data;
      • purposeful.
  • Browsing
      • glancing around;
      • F1; cars, Le Mans, France, tourism.
slide6
structure

Full text

Index terms

Logical View of documents

  • Logical view of the documents
  • Document representation viewed as a continuum: logical view of docs might shift.

Accents

spacing

Noun

groups

Manual

indexing

Docs

stopwords

stemming

structure

slide7
Typical IR task

Docs

Index Terms

doc

match

Information Need

Ranking

query

slide8
IR keyword match
  • Matching at index term level is quite imprecise;
  • No surprise that users get frequently unsatisfied;
  • Since most users have no training in query formation, problem is even worst;
  • Frequent dissatisfaction of Web users;
  • Issue of deciding relevance is critical for IR systems: ranking.
slide9
Ranking
  • A ranking is an ordering of the documents retrieved that (hopefully) reflects the relevance of the documents to the user query;
  • A ranking is based on fundamental premises regarding the notion of relevance, such as:
    • common sets of index terms;
    • sharing of weighted terms;
    • likelihood of relevance.
  • Each set of premises leads to a distinct IR model.
slide10
Algebraic

Set Theoretic

Generalized Vector

Lat. Semantic Index

Neural Networks

Structured Models

Fuzzy

Extended Boolean

Non-Overlapping Lists

Proximal Nodes

Classic Models

Probabilistic

boolean

vector

probabilistic

Inference Network

Belief Network

Browsing

Flat

Structure Guided

Hypertext

IR Models

U

s

e

r

T

a

s

k

Retrieval:

Adhoc

Filtering

Browsing

slide11
IR Models
  • The IR model, the logical view of the docs, and the retrieval task are distinct aspects of the system.
slide12
Classic IR Models
  • Traditional IR systems employ a set of index terms to represent the documents;
  • The key idea is that the document semantics can be represented by the index terms;
  • Usual formal models:
    • Boolean;
    • Vector-space;
    • Probabilistic.
slide13
Classic IR Models
  • Each document is represented by a set of representative index terms;
  • An index term is a document word useful for remembering the document main themes;
  • Usually, index terms are nouns because nouns have meaning by themselves;
  • However, search engines assume that all words are index terms (full text representation).
slide14
Classic IR Models
  • Not all terms are equally useful for representing the document contents: less frequent terms allow identifying a narrower set of documents;
  • The importance of the index terms is represented by weights associated to them;
    • ki be an index term
    • dj be a document
    • wij is a weight associated with (ki,dj)
  • The weight wij quantifies the importance of the index term for describing the document contents.
slide15
Classic IR Models
  • ki is an index term;
  • dj is a document;
  • t is the total number of terms;
  • N is the total number of docs;
  • K = (k1, k2, …, kt) is the set of all index terms;
  • wij >= 0 is a weight associated with (ki,dj);
  • wij = 0 indicates that term does not belong to doc;
  • vec(dj) = (w1j, w2j, …, wtj) is a weighted vector associated with the document dj ;
  • gi(vec(dj)) = wij is a function which returns the weight associated with pair (ki,dj) .
retrieval tasks
Retrieval Tasks
  • Ad hoc retrieval: Fixed document corpus, varied queries.
  • Filtering: Fixed query, continuous document stream.
    • User Profile: a model of relative static preferences.
    • Binary decision of relevant/not-relevant.
  • Routing: Same as filtering but continuously supply ranked lists rather than binary filtering.
slide17
Retrieval: Ad Hoc x Filtering
  • Ad hoc retrieval:

Q1

Q2

Collection

“Fixed Size”

Q3

Q4

Q5

slide18
Retrieval: Ad Hoc x Filtering
  • Filtering:

Docs Filtered

for User 2

User 2

Profile

User 1

Profile

Docs for

User 1

Documents Stream

common preprocessing steps
Common Preprocessing Steps
  • Strip unwanted characters/markup (e.g. HTML tags, punctuation, numbers, etc.).
  • Break into tokens (keywords) on whitespace.
  • Stem tokens to “root” words
    • computational  comput
  • Remove common stopwords (e.g. a, the, it, etc.).
  • Detect common phrases (possibly using a domain specific dictionary).
  • Build inverted index (keyword  list of docs containing it).
ad