Discovering key concepts in verbose queries
Download
1 / 26

Discovering Key Concepts in Verbose Queries - PowerPoint PPT Presentation


  • 164 Views
  • Uploaded on

Discovering Key Concepts in Verbose Queries. Michael Bendersky and W. Bruce Croft University of Massachusetts SIGIR 2008. Objective. “Discovering Key Concepts in Verbose Queries”. Objective. “Discovering Key Concepts in Verbose Queries” <num> Number 829 <title> Spanish Civil War support

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about ' Discovering Key Concepts in Verbose Queries' - jadzia


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Discovering key concepts in verbose queries

Discovering Key Concepts in Verbose Queries

Michael Bendersky and W. Bruce Croft

University of Massachusetts

SIGIR 2008


Objective
Objective

  • “Discovering Key Concepts in Verbose Queries”


Objective1
Objective

  • “Discovering Key Concepts in Verbose Queries”

  • <num> Number 829

    <title> Spanish Civil War support

    <desc> Provide information on all kinds of material international support provided to either side in the Spanish Civil War


Objective2
Objective

  • “Discovering Key Concepts in Verbose Queries”

  • <num> Number 829

    <title> Spanish Civil War support

    <desc> Provide information on all kinds of material international support provided to either side in the Spanish Civil War


Objective3
Objective

  • “Discovering Key Concepts in Verbose Queries”

  • Use of key concepts?


Objective4
Objective

  • “Discovering Key Concepts in Verbose Queries”

  • Use of key concepts?

  • Combine with current IR model


Retrieval model
Retrieval Model

  • Conventional Language Model:

    score(q,d) = p(q|d) =


Retrieval model1
Retrieval Model

  • Conventional Language Model:

    score(q,d) = p(q|d) =

  • New Model:

    score(q,d) = p(q|d) = =



Final retrieval function1
Final Retrieval Function

score(q,d) =

Language Model


Final retrieval function2
Final Retrieval Function

score(q,d) =

Key Concepts


What is a concept
What is a Concept?

  • Noun phrase in a query


What is a concept1
What is a Concept?

  • Noun phrase in a query

  • <num> Number 829

    <title> Spanish Civil War support

    <desc> Provide information on all kinds of material international support provided to either side in the Spanish Civil War


What is a concept2
What is a Concept?

  • Noun phrase in a query

  • <num> Number 829

    <title> Spanish Civil War support

    <desc> Provide information on all kinds of material international support provided to either side in the Spanish Civil War


Finding key concepts
Finding ‘Key’ Concepts

  • Rank concepts by p(ci|q)


Finding key concepts1
Finding ‘Key’ Concepts

  • Rank concepts by p(ci|q)

  • Compute p(ci|q) by frequency?

  • <num> Number 829

    <title> Spanish Civil War support

    <desc> Provide information on all kinds of material international support provided to either side in the Spanish Civil War


Finding key concepts2
Finding ‘Key’ Concepts

  • Approximate p(ci|q) by machine learning

  • h(ci) is ci’s query-independent importance score

  • p(ci|q) = h(ci) / ciqh(ci)

AdaBoost.M1

h(ci)

ci


Features of a concept
Features of a Concept

  • is_cap : is capitalized

  • tf : in corpus

  • idf : in corpus

  • ridf : idf modified by Poisson model

  • wig : weighted information gain; change in entropy from corpus to retrieved data

  • g_tf : Google term frequency

  • qp : number of times the concept appears as a part of a query in MSN Live

  • qe : number of times the concept appears as exact query in MSN Live



Exp 1 identifying key concept
Exp 1: Identifying Key Concept

  • Cross-validation on corpus

  • Each fold has 50 queries

  • Check whether the top concept is a key concept

  • Assume 1 key concept per query during annotation



Exp 1 identifying key concept2
Exp 1: Identifying Key Concept

  • Better than idf ranking


Exp 2 information retrieval
Exp 2: Information Retrieval

score(q,d) =

  • Use only the top 2 concepts for each query

  • q is the entire <desc> section

  •  = 0.8


Exp 2 information retrieval1
Exp 2: Information Retrieval

  • KeyConcept[2]<desc> : author’s method

  • SeqDep<desc> : include all bigrams in query



What to take home
What to take home?

  • Singling out key concepts improves retrieval


ad