retrieval models and ranking systems n.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
Retrieval Models and Ranking Systems PowerPoint Presentation
Download Presentation
Retrieval Models and Ranking Systems

Loading in 2 Seconds...

play fullscreen
1 / 39

Retrieval Models and Ranking Systems - PowerPoint PPT Presentation


  • 135 Views
  • Uploaded on

Retrieval Models and Ranking Systems. CSC 575 Intelligent Information Retrieval. Retrieval Models. Model is an idealization or abstraction of an actual process in this case, process is matching of documents with queries, i.e., retrieval

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Retrieval Models and Ranking Systems' - keola


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
retrieval models and ranking systems

Retrieval Models and Ranking Systems

CSC 575

Intelligent Information Retrieval

retrieval models
Retrieval Models
  • Model is an idealization or abstraction of an actual process
    • in this case, process is matching of documents with queries, i.e., retrieval
  • Mathematical models are used to study the properties of the process, draw conclusions, make predictions
    • Conclusions derived from a model depend on whether the model is a good approximation to the actual situation
  • Retrieval models can describe the computational process
    • e.g. how documents are ranked
    • note that inverted file is an implementation not a model
  • Retrieval variables: queries, documents, terms, relevance judgements, users, information needs
  • Retrieval models have an explicit or implicit definition of relevance

Intelligent Information Retrieval

slide3

Lexical analysis and stop words

Information

need

Collections

Pre-process

text input

Parse

Index

Query

How is

the index

constructed?

Rank

How is the

matching and scoring done?

Result

Sets

retrieval models1
Retrieval Models
  • Customary to distinguish between exact-match and best-match retrieval
  • Exact-match
    • query specifies precise retrieval criteria every document either matches or fails to match query
    • result is a set of documents
  • Best-match
    • query describes good or “best” matching document
    • result is ranked list of documents
    • result may include estimate of quality
  • Best-match models: better retrieval effectiveness
    • good documents appear at top of ranking
    • but efficiency is better in exact match (e.g., Boolean)

Intelligent Information Retrieval

ranking algorithms
Ranking Algorithms
  • Assign weights to the terms in the query
  • Assign weights to the terms in the documents
  • Compare the weighted query terms to the weighted document terms
    • Boolean matching (exact match)
    • simple (coordinate level) matching
    • cosine similarity
    • other similarity measures (Dice, Jaccard, overlap, etc.)
    • extended Boolean models
    • probabilistic models
  • Rank order the results
    • pure Boolean has no ordering

Intelligent Information Retrieval

boolean retrieval
Boolean Retrieval
  • Boolean retrieval most common exact-match model
    • queries are logic expressions with document features as operands
    • retrieved documents are generally not ranked
    • query formulation difficult for novice users
  • “Pure” Boolean operators: AND, OR, NOT
  • Most systems have proximity operators
  • Most systems support simple regular expressions as search terms to match spelling variants

Intelligent Information Retrieval

boolean logic

B

A

Boolean Logic
  • AND and OR in a Boolean query represent intersection and union of the corresponding documents sets, respectively
  • NOT represents the complement of the corresponding set

Intelligent Information Retrieval

boolean queries
Boolean Queries
  • Boolean queries are Boolean combination of terms
    • Cat
    • Cat OR Dog
    • Cat AND Dog
    • (Cat AND Dog) OR Collar
    • (Cat AND Dog) OR (Collar AND Leash)
    • (Cat OR Dog) AND (Collar OR Leash)
  • (Cat OR Dog) AND (Collar OR Leash)
    • Each of the following combinations works:

Intelligent Information Retrieval

boolean matching
Boolean Matching

t1

t2

m1= t1t2t3

D9

D2

D1

m2= t1 t2t3

m3 = t1 t2t3

m5

m3

m6

m4 = t1t2t3

D11

D4

D5

m5 = t1t2t3

D3

m1

m6 = t1t2t3

D6

m2

m4

m7 = t1t2t3

D10

m8= t1t2t3

m7

m8

D8

D7

t3

Hit list for the query t1 AND t2 

{D1, D3, D5, D9, D10, D11} ∩{D1, D2, D4, D5, D6} = {D1, D5}

Intelligent Information Retrieval

psuedo boolean queries
Psuedo-Boolean Queries
  • A new notation, from web search
    • +cat dog +collar leash
  • Does not mean the same thing!
  • Need a way to group combinations
  • Phrases:
    • “stray cat” AND “frayed collar”
    • +“stray cat” + “frayed collar”

Intelligent Information Retrieval

faceted boolean query
Faceted Boolean Query
  • Strategy: break query into facets
    • conjunction of disjunctions (conjunctive normal form)

a1 OR a2 OR a3

b1 OR b2

c1 OR c2 OR c3 OR c4

    • each facet expresses a topic or concept

“rain forest” OR jungle OR amazon

medicine OR remedy OR cure

research OR development

AND

AND

Intelligent Information Retrieval

faceted boolean query1
Faceted Boolean Query
  • Query still fails if one facet missing
  • Alternative: a form of Coordination level ranking
    • Order results in terms of how many facets (disjuncts) are satisfied
    • Also called Quorum ranking
  • Problem: Facets still undifferentiated
    • Alternative: assign weights to facets

Intelligent Information Retrieval

boolean model
Boolean Model
  • Advantages
    • simple queries are easy to understand
    • relatively easy to implement
    • structured queries
    • queries can be automatically translated into CNF or DNF
  • Disadvantages
    • difficult to specify what is wanted
    • too much returned, or too little (acceptable precision generally means unacceptable recall)
    • ordering not well determined
    • query formulation difficult for novice users
  • Dominant language in commercial systems until the WWW

Intelligent Information Retrieval

vector space model revisited
Vector Space Model(revisited)
  • Documents are represented as “bags of words”
  • Represented as vectors when used computationally
    • A vector is an array of floating point (or binary in case of bit maps)
    • Has direction and magnitude
    • Each vector has a place for every term in collection (most are sparse)

Document Ids

a document

vector

nova galaxy heat actor film role

A 1.0 0.5 0.3

B 0.5 1.0

C 1.0 0.8 0.7

D 0.9 1.0 0.5

E 1.0 1.0

F 0.7

G 0.5 0.7 0.9

H 0.6 1.0 0.3 0.2

I 0.7 0.5 0.3

Intelligent Information Retrieval

documents query in n dimensional space
Documents & Query in n-dimensional Space
  • Documents are represented as vectors in term space
    • Terms are usually stems
    • Documents represented by binary vectors of terms
  • Queries represented the same as documents
  • Query and Document weights are based on length and direction of their vector
  • A vector distance measure between the query and documents is used to rank retrieved documents

Intelligent Information Retrieval

the notion of similarity in ir
The Notion of “Similarity” in IR
  • The notion of similarity is central to many aspects of information retrieval and filtering:
    • measuring similarity of the query to documents is the primary factor in determining what is returned (and how they are ranked)
    • similarity measures can also be used in clustering documents (I.e., grouping together documents with similar content)
    • the same similarity measures can also be used to group together related terms (based on their occurrence patterns across documents in the collection)

Intelligent Information Retrieval

vector based similarity measures
Vector-Based Similarity Measures
  • Simple Matching and Cosine Similarity
    • Simple matching = dot product of two vectors
    • Cosine Similarity = normalized dot product
    • the norm of a vector X is:
    • the cosine similarity of vectors X and Y is:

In other words, divide the dot product by the norms of the two vectors

Intelligent Information Retrieval

vector based similarity measures1
Vector-Based Similarity Measures
  • Why divide by the norm?
    • Example:
      • X = <2, 0, 3, 2, 1, 4>
      • ||X|| = SQRT(4+0+9+4+1+16) = 5.83
      • X* = X / ||X|| = <0.343, 0, 0.514, 0.343, 0.171, 0.686>
    • Now, note that ||X*|| = 1
    • So, dividing a vector by its norm, turns it into a unit-length vector
    • Cosine similarity measures the angle between two unit length vectors (i.e., the magnitude of the vectors are ignored).

Intelligent Information Retrieval

computing a similarity score 2d example
Computing a similarity score2D Example

Intelligent Information Retrieval

computing similarity scores
Computing Similarity Scores

1.0

0.8

0.6

0.4

0.2

0.2

0.4

0.6

0.8

1.0

Intelligent Information Retrieval

other vector space similarity measures
Other Vector Space Similarity Measures

Simple Matching:

Cosine Coefficient:

Dice’s Coefficient:

Jaccard’s Coefficient:

Intelligent Information Retrieval

vector space similarity measures
Vector Space Similarity Measures
  • Again consider the following two document and the query vectors:

D1 = (0.8, 0.3)

D2 = (0.2, 0.7)

Q = (0.4, 0.8)

  • Computing similarity using Jaccard’s Coefficient:
  • Computing similarity using Dice’s Coefficient:

sim(Q, D2) = 0.96

sim(Q, D1) = 0.73

Intelligent Information Retrieval

vector space similarity measures example
Vector Space Similarity MeasuresExample

Intelligent Information Retrieval

vector space similarity measures example1
Vector Space Similarity MeasuresExample

Intelligent Information Retrieval

probabilistic models
Probabilistic Models
  • Attempts to be more theoretically sound than the vector space model
    • try to predict the probability of a document’s being relevant, given the query
    • there are many variations
    • usually more complicated to compute than v.s.
    • usually many approximations are required
  • Relevance information is required from a random sample of documents and queries (training examples)
  • Works about the same (sometimes better) than vector space approaches

Intelligent Information Retrieval

basic probabilistic retrieval
Basic Probabilistic Retrieval
  • Retrieval is modeled as a classification process
  • Two classes for each query: the relevant and non-relevant documents (with respect to a given query)
    • could easily be extended to three classes (i.e. add a don’t care)
  • Given a particular document D, calculate the probability of belonging to the relevant class
    • retrieve if greater than probability of belonging to non-relevant class
    • i.e. retrieve if P(R|D) > P(NR|D)
  • Equivalently, rank by a discriminant value (also called likelihood ratio) P(R|D) / P(NR|D)
  • Different ways of estimating these probabilities lead to different models

Intelligent Information Retrieval

basic probabilistic retrieval1
Basic Probabilistic Retrieval
  • A given query divides the document collection into two sets: relevant and non-relevant
  • If a document set D has been selected in response to a query, retrieve the document if

dis(D) > 1

where

dis(D) = P(R|D) / P(NR|D)

  • is the discriminant of D
  • This criteria can be modified by weighting the two probabilities

Relevant

Documents

P(R|D)

P(NR|D)

Non-Relevant

Documents

Document

Intelligent Information Retrieval

estimating probabilities
Estimating Probabilities
  • Bayes’ Rule can be used to “invert” conditional probabilities:
  • Applying that to discriminant function:
  • Note that P(R) is the probability that a random document is relevant to the query, and P(NR) = 1 - P(R)

P(R) = n / N and P(NR) = 1 - P(R) = (N - n) / N

where n = number of relevant documents, and

N = total number of documents in the collection

Intelligent Information Retrieval

estimating probabilities1
Estimating Probabilities
  • Now we need to estimate P(D|R) and P(D|NR)
    • If we assume that a document is represented by terms t1, . . ., tn, and that these terms are statistically independent, then
    • and similarly we can compute P(D|NR)
    • Note that P(ti|R) is the probability that a term ti occurs in a relevant document, and it can be estimated based on previously available sample (e.g., through relevance feedback)
    • So, based on the probability of the distribution of terms in relevant and non-relevant documents we can estimate whether the document should be retrieved (i.e, if dis(D) > 1)
    • Note that documents that are retrieved can be ranked based on the value of the discriminant

Intelligent Information Retrieval

probabilistic retrieval example
Probabilistic Retrieval - Example

Since the discriminant is less than one, document D should not be retrieved

Intelligent Information Retrieval

probabilistic retrieval cont
Probabilistic Retrieval (cont.)
  • In practice, can’t build a model for each query
    • Instead a general model is built based on query-document pairs in the historical (training) data
    • Then for a given query Q, the discriminant is computed only based on the conditional probabilities of the query terms
      • If query term t occurs in D, take P(t|R) and P(t|NR)
      • If query term t does not appear in D, take 1-P(t|R) and 1- P(t|NR)

Q = t1, t3, t4 D = t1, t4, t5

Intelligent Information Retrieval

probabilistic models1
Strong theoretical basis

In principle should supply the best predictions of relevance given available information

Can be implemented similarly to Vector

Relevance information is required -- or is “guestimated”

Important indicators of relevance may not be term -- though terms only are usually used

Optimally requires on-going collection of relevance information

Probabilistic Models

Advantages

Disadvantages

Intelligent Information Retrieval

vector and probabilistic models
Vector and Probabilistic Models
  • Support “natural language” queries
  • Treat documents and queries the same
  • Support relevance feedback searching
  • Support ranked retrieval
  • Differ primarily in theoretical basis and in how the ranking is calculated
    • Vector assumes relevance
    • Probabilistic relies on relevance judgments or estimates

Intelligent Information Retrieval

extended boolean models
Extended Boolean Models
  • Weighted Boolean Queries
    • Weights are assigned to the operands in Boolean query

A0.6ANDB0.75A1.0ORB0.3

    • The weighting operation depends on the distance between document sets for A and B
      • a weight of 1.0 says that all of the corresponding document set is considered in the operation
      • a weight of 0 < w < 1 says that only a portion of the document set is considered
      • the documents added or deleted are those that are “closest” to the current set of documents

Intelligent Information Retrieval

weighted boolean queries
Weighted Boolean Queries

A1.0 AND B1.0 = A Ç B A1.0 OR B1.0 = A È B

A1.0 AND B0.0 = AA1.0 OR B0.0 = A

A1.0 OR B.75 =

A È (75% ofB - A)

A1.0 AND B.75 =

(A Ç B) È (25% of A - B)

A

B

A

B

Intelligent Information Retrieval

weighted boolean queries1
Weighted Boolean Queries
  • Matching Algorithm

1. Find initial matching set (non-weighted Boolean document set)

2. Find the invariant document set (set of documents that are present both when operand weight is 1.0 and when the weight is 0.0); the optional set is the remaining items

3. Compute the centroid of the invariant set

4. Find the number of documents, say k, from the optional set that will potentially be added to the invariant set (determined by the weight of the query term)

5. Compute similarity between documents in the optional set and the centroid (of the invariant set)

6. Items to be added or deleted are the top k documents in the optional set with the highest similarity scores

Intelligent Information Retrieval

demo of extended boolean query
Demo of Extended Boolean Query*

http://ir.exp.sis.pitt.edu/res2/data/66/

*Thanks to Michael Bombyk for discovering this applet!

Intelligent Information Retrieval

weighted boolean queries example
Weighted Boolean Queries - Example

Q1(initial) = (D1, D2, D3, D4, D5, D6, D8)

Q1(invariant) = (D3, D6, D8)

Q1(optional) = (D1, D2, D4, D5) => 4 items

No. selected docs. =

Centroid(Q1) = (1/3)

= (4.7, 0.7, 2.0, 2.0)

Computing Similarity (using simple matching):

SIM(Centroid,D1) = (4.7,0.7,2.0,2.0).(0,4,0,8) = 18.8

SIM(Centroid,D2) = (4.7,0.7,2.0,2.0).(0,2,0,0) = 1.4

SIM(Centroid,D4) = (4.7,0.7,2.0,2.0).(0,6,4,6) = 24.2

SIM(Centroid,D5) = (4.7,0.7,2.0,2.0).(0,4,6,4) = 22.8

So the final Hit list is : (D3, D6, D8) È (D4, D5)

Query

Q1 = A1.0 OR B.333

Intelligent Information Retrieval

weighted boolean queries example1
Weighted Boolean Queries - Example

Q2(initial) = (D3, D4, D5)

Q2(invariant) = (D3, D4, D5)

Q2(optional) = (D1, D8) => 2 items

No. selected docs. =

Centroid(Q2) = (1/3)

=

Computing Similarity (using simple matching):

SIM(Centroid,D1) =

SIM(Centroid,D8) =

Final Hit list is: (D3, D4, D5) È (D1)

Query

Q2 = C.75 AND D1.0

Intelligent Information Retrieval