an overview of bayesian network based retrieval models l.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
An Overview of Bayesian Network-based Retrieval Models PowerPoint Presentation
Download Presentation
An Overview of Bayesian Network-based Retrieval Models

Loading in 2 Seconds...

play fullscreen
1 / 49

An Overview of Bayesian Network-based Retrieval Models - PowerPoint PPT Presentation


  • 314 Views
  • Uploaded on

An Overview of Bayesian Network-based Retrieval Models. Juan Manuel Fernández Luna Departamento de Informática Universidad de Jaén jmfluna@ujaen.es. Department of Computing Science, University of Glasgow October, 21 th - 2002. Layout. Introduction Introduction to Belief Networks

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'An Overview of Bayesian Network-based Retrieval Models' - farrah


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
an overview of bayesian network based retrieval models

An Overview of Bayesian Network-basedRetrieval Models

Juan Manuel FernándezLuna

Departamento de Informática

Universidad de Jaén

jmfluna@ujaen.es

Department of Computing Science, University of Glasgow

October, 21th - 2002

layout
Layout
  • Introduction
  • Introduction to Belief Networks
  • Bayesian Network-based IR Models
    • Inference Network Model
    • Belief Network Model
    • Bayesian Network Retrieval Model
  • Relevance Feedback
  • Other applications
  • Bibliography
introduction
Introduction
  • Query and document characterizations are incomplete.
  • The query is a vague description of the users´ information need.
  • Computing relevance degree: 1 and 2 +

A) different representations that a concept may have, B) these concepts are not independent among them.

Information Retrieval  Uncertain process

introduction4
Introduction

Probabilistic models tried to overcome these problems…

Researchers focused their attention on Belief networks in order to apply them to IR because:

They show a high performance in actual problems characterised by uncertainty.

introduction to belief networks
Introduction to Belief Networks
  • Graphical models able to represent and efficiently manipulate n-dimensional probability distributions.
  • The knowledge obtained from a problem is encoded in a Belief network by means of the quantitative and qualitative componets:
introduction to belief networks6
Introduction to Belief Networks
  • Qualitative part: Directed Acyclic Graph.

G=(V,E):

      • V (Nodes)  Random variables, and
      • E (Arcs)  (In)dependence relationships.
introduction to belief networks7
Introduction to Belief Networks
  • Quantitative part A set of conditional distributions:
    • Drawn from the graph structure,
    • representing the strength of the relationships,
    • stored in each node.

Belief Network  Bayesian Network

(Conditional probability distributions)

introduction to belief networks9
Introduction to Belief Networks

Taking into account these (in)dependences, the joint probability distribution could be restored from the network:

Pa(Xi) being the set of parents of the variable Xi.

This previous expression implies an important saving in the storage space.

introduction to belief networks10
Introduction to Belief Networks
  • Construction:
    • Manual, using an expert´s knowledge.
    • Automatic, by means of a learning algorithm.
  • Inference:
    • Given a set of evidences, E, to obtain the probability with which a variable can take a certain value.
    • p(S=T | W=T)=0.430, p(R=T| W=T)= 0.708
bayesian network based ir models
Bayesian Network-based IR Models
  • Inference Network Model
  • Belief Network Model
  • Peter Bruza´s Index Belief Expressions
  • Maria Indrawan et al.´s Model
  • Bayesian Network Retrieval Model
slide12

d1

d2

dj-1

dj

r1

r2

r3

rm

q1

q2

Inference Network Model

inn

Link Matrices

Inference:

Instantiating each document, dj, and computing p(inn | dj).

belief network model

d1

d2

dj-1

dj

t1

t2

t3

tm

Belief Network Model

Q

2M assigments  unfeasible

Probabilities are defined in such a way that only one configuration is evaluated

bayesian network retrieval model
Bayesian Network Retrieval Model
  • There are strong relationships among a document and the terms that index it.
  • Document relationships are only present by means of the terms that index them.
  • Documents are conditional independent given the terms by which they were indexed.

Guidelines to build the BNR Model:

bayesian network retrieval model15

Term Subnetwork

Document

Subnetwork

Bayesian Network Retrieval Model

Ti{¬ti, ti}

Dj{¬dj, dj}

bayesian network retrieval model16

Term

Subnetwork

T1

T2

T3

T4

T5

T6

Document

Subnetwork

D1

D2

D3

D4

Bayesian Network Retrieval Model

All the terms are independent among them:

Simple Bayesian Network Retrieval Model

bayesian network retrieval model17
Bayesian Network Retrieval Model
  • Probability Distributions:
    • Term nodes: p(tj)=1/M, p(¬tj)=1-p(tj)
    • Document nodes: p(Dj | Pa(Dj)), Dj

But... If a document has been indexed by 30 terms, we need to estimate and store 230 probabilities.

Problem!!!!

bayesian network retrieval model18
Bayesian Network Retrieval Model

Solution:

Probability functions

pa(Dj) being a configuration of the parents of Dj.

bayesian network retrieval model19
Bayesian Network Retrieval Model

Retrieval:

  • Instantiate TQQ to Relevant.
  • Run a propagation algorithm in the network.
  • Rank the documents according p(dj | Q), Dj

Problem:

Great amount of nodes and existing cycles in the graph

General purpose propagation algorithms can´t be applied due to efficiency considerations.

bayesian network retrieval model20
Bayesian Network Retrieval Model

Solution:

Taking advantage of:

  • The kind of probability function used, and
  • The topology.

Propagation is substituted by

Evaluation of the probability function

in each document node

bayesian network retrieval model21
Bayesian Network Retrieval Model

Result: An efficient and exact propagation.

Including Query term frequencies:

bayesian network retrieval model22
Bayesian Network Retrieval Model

Removing the term independency restricction:

  • We are interested in representing the main relationships among terms in the collection.

Term subnetwork  Polytree

Why?

There is a set of efficient learning and propagation algorithms available for this topology.

bayesian network retrieval model23

T5

Term

Subnetwork

T2

T4

T6

T1

T3

Document

Subnetwork

D1

D2

D3

D4

Bayesian Network Retrieval Model
bayesian network retrieval model24
Bayesian Network Retrieval Model

Probability distributions:

Marginal Distributions (root term nodes):

(M being the number of terms in the collection)

bayesian network retrieval model25
Bayesian Network Retrieval Model

Conditional Distributions (term nodes with parents):

(based on Jaccard´s coefficient)

Conditional Distributions (document nodes):

Probability functions

bayesian network retrieval model26
Bayesian Network Retrieval Model

Retrieval:

TqQ  Relevant

p(dj|Q)??

But... Due to the complexity of the whole network we can not run an exact propagation algorithm.

Solution:

PROPAGATION + EVALUATION

bayesian network retrieval model27
Bayesian Network Retrieval Model
  • Propagation:

Running the exact Pearl´s propagation algorithm in the polytree (term subnetwork), p(ti|Q), Ti, are computed.

  • Evaluation:

Evaluation of a probability function in the Document Subnetwork, computing p(dj|Q), Dj, incorporating p(ti|Q).

bayesian network retrieval model28
Bayesian Network Retrieval Model

Adding document relationships

Given a document, Dj:

  • Compute p(dj|di), Di.
  • Select those documents with greatest probability of relevance with respect to Dj.
  • Link Dj with all these documents.
bayesian network retrieval model29

Term Subnetwork

D1

D2

D3

D4

D5

D6

D7

D`1

D`2

D`3

D´4

D´5

D´6

D´7

Bayesian Network Retrieval Model

But... Instead of linking the documents in the document subnetwork...

bayesian network retrieval model30
Bayesian Network Retrieval Model
  • We don´t have to restimate probability distributions in the document nodes.
  • Propagation: Evaluation of a probability function in the second document layer  Efficiency.

Advantages of this topology:

bayesian network retrieval model31

Where Sj is a normalising constant

Bayesian Network Retrieval Model
  • Compute p(dj|Q), Dj

(1st document layer)

  • Compute p(d´j|Q), D´j

(2nd document layer)

Retrieval?

bayesian network retrieval model32
Bayesian Network Retrieval Model
  • Reducing the propagation time in the Term Subnetwork:
  • Representing only the best relationships among terms.
  • Modifying Pearl´s propagation algorithm.
  • Changing the Term subnetwork topology.
bayesian network retrieval model33
Bayesian Network Retrieval Model

1. Representing only the best term relationships

Problems:

  • Automatically learning the relationships among terms could imply that some relationships are not strong enough.

Retrieval effectiveness could be damaged

  • If the number of terms is very high, the learning stage could be time-consuming.
bayesian network retrieval model34

Classification algorithm

Non-selected terms

Selected terms

Polytree learning

Bayesian Network Retrieval Model

Solution:

Selection of best terms

Collection

bayesian network retrieval model35
Bayesian Network Retrieval Model
  • Advantages:
    • Reduction of learning time
    • Representation of the best relationships among terms
    • Faster propagation.
bayesian network retrieval model36
Bayesian Network Retrieval Model
  • Classification algorithm:
      • K-means, with Euclidean distance
  • Objects:
      • Terms
  • Attributes:
      • Term discrimination value (tdv)
      • Inverse Document Frequency (idf)
  • Classes:
      • Good terms: higher tdv, and medium-high idf.
      • Rest of the terms.
bayesian network retrieval model37
Bayesian Network Retrieval Model

2. Modifying Pearl´s algorithm.

In large polytrees, the belief of a great number of terms, those furthest from query terms, will not be updated after propagating.

So...Why is the propagation

algorithm still running?

bayesian network retrieval model40
Bayesian Network Retrieval Model

3. Changing the Term Subnetwork topology.

In certain cases, the polytree topology of the Term subnetwork, even using the term selection approach, could not be very appropriate.

An alternative topology:

Two term layers

  • Preserving accuracy of term relationships represented in the graph.
  • Providing an efficient inference mechanism.
bayesian network retrieval model41

T1

T2

T3

T4

T5

T6

T7

T´1

T´2

T´3

T´4

T´5

T´6

T´7

Document Subnetwork

Bayesian Network Retrieval Model
bayesian network retrieval model42
Bayesian Network Retrieval Model
  • Relationships ara captured using the coocurrences among terms.
  • The probability of relevance in the second term layer is computed by means of:
relevance feedback in b n models
Relevance Feedback in B.N. Models
  • Inference and Belief Network Models:
    • Modifying link matrices and adding new links (and also new document nodes in the second).
  • Bayesian Network Model:
    • Inclusion of new evidences from the inspection of the document ranking using partial evidences.
    • (Advantage: neither graph structure modification nor probability matrix re-estimation).
other applicatio ns
Other applications:
  • Indexing
  • Hypertext
  • User profiling
  • WWW
  • Structured documents
  • Image retrieval
  • Document classification
  • Filtering
bibliography
Bibliography
  • Bruza, P. & van de Gaag, L.C. (1996). Index Expression Belief Network for Information Disclosure. International Journal of Expert Systems. 7(2), 107-138.
  • de Campos, L.M.; Fernández-Luna, J.M. & Huete, J.F. (2000). Building Bayesian network-based information retrieval systems. Proc. of the 2nd LUMIS Workshop. 543-550.
  • de Campos, L.M.; Fernández-Luna, J.M. & Huete, J.F. (2001). Relevance Feedback in the Bayesian Network Retrieval Model: An Approach Based on Term Instantiation. Lecture Notes in Computer Science. 2189. 13 – 23.
  • de Campos, L.M.; Fernández-Luna, J.M. & Huete, J.F. (2001). Document Instantiation for relevance feedback in the Bayesian Network Retrieval model. Proceedings of the SIGIR’01 Workshop on Mathematical and Formal Models in Information Retrieval. 10-18
  • de Campos, L.M.; Fernández-Luna, J.M. & Huete, J.F. (2002). A layered Bayesian Network Model for Document Retrieval. Proceedings of the ECIR’2002 Colloquium. Lecture notes in Computer Science, 2291, 169 – 182.
bibliography46
Bibliography
  • Luis M. de Campos, Juan M. Fernández-Luna, Juan F. Huete. Reducing term to term relationships in an extended Bayesian network retrieval model. Proceedings of the Ninth International IPMU Conference (Information Processing and Mangement of Uncertainty in Knowledge-based Systems) Conference, Vol. 2, 1195-1202 (ISBN Vol. 2: 2-9516453-2-5), 2002. ESIA – Université de Savoie (Editor).
  • Luis M. de Campos, Juan M. Fernández-Luna, Juan F. Huete. Two terms layer: An alternative topology for representing term relationships in the Bayesian Network Retrieval Model. Electronic Proceeding of the Seventh Online World Conference on Soft Computing in Industrial Applications (wsc7.ugr.es).
  • Luis M. de Campos, Juan M. Fernández-Luna, Juan F. Huete. Reducing Propagation Effort in Large Polytree: An application to Information Retrieval. To appear in Proceedings of the Workshop on Probabilistic and Graphical Models. Cuenca (SPAIN), 2002.
  • Crestani, F., Lalmas, M., van Rijsbergen, C.J., Campbell, L. (1998). Is this Document Relevant?… Probably: A Survey of Probabilistic Models in Information Retrieval. Computing Survey. 30(4). 528-552.
bibliography47
Bibliography
  • Fernández-Luna, J.M. (2001). Modelos de Recuperación de Información basados en Redes de Creencia. Ph.D. Thesis (in Spanish). University of Granada.
  • Frisse M. & Cousins, S.B. (1989). Information Retrieval from Hypertext: Update on the Dynamic Medical Handbook Project. Proceedings of the Hypertext’89 Conference. 199-212.
  • Ghazfan , D., Indrawan, M. & Srinivasan, B. (1996). Towards meaningful Bayesian networks. IPMU’96 Conference. 841-846.
  • Haines, D. & Croft W.B. (1983). Relevance Feedback and Inference Networks. 20th ACM-SIGIR Conference. 119-128.
  • Pearl, J. (1988). Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference. Morgan and Kaufmann. San Mateo, California.
  • Reis, I. (2000). Bayesian Networks for Information Retrieval. Ph.D. Thesis. Universidad Federal de Minas Gerais.
  • van Rijsbergen, C.J. (1971). Information Retrieval. 2nd Edition. Butter Worths.
  • van Rijsbergen, C.J., Harper, D.J., & Porter, M.F. (1981). The selection of good search terms. Information Processing & Management. 17, 77-91.
bibliography48
Bibliography
  • Sahami, M. (1998). Using Machine Learning to Improve Information Access. Ph.D. Thesis. Stanford University.
  • Savoy, J. & Desbois, D. (1991). Information Retrieval in Hypertext Systems: An Approach using Bayesian Networks. Electronic Publishing. 42(2), 87-108.
  • Turtle, H.R., & Croft, W.B. (1991). Evaluation of an Inference Network-based Retrieval Model. Information Systems. 9(3), 189-224.
  • Turtle, H.R., & Croft, W.B. (1997). Uncertainty in Information Systems. In Uncertainty Management in Information System: From needs to solutions. Kluver Academic. 189-224.
  • Tzeras K. & Hartman, S. (1993). Automatic Indexing Based on Bayesian Inference Netoworks. 16th ACM-SIGIR Conference. 22-35.
the end

The end...

Thank you very much