Learning user interaction models for predicting web search result preference
This presentation is the property of its rightful owner.
Sponsored Links
1 / 23

Learning User Interaction Models for Predicting Web Search Result Preference PowerPoint PPT Presentation


  • 38 Views
  • Uploaded on
  • Presentation posted in: General

Learning User Interaction Models for Predicting Web Search Result Preference. Eugene Agichtein et al. Microsoft Research SIGIR ‘06. Objective. Provide a rich set of features for representing user behavior Query-text Browsing Clickthough Aggregate various feature RankNet.

Download Presentation

Learning User Interaction Models for Predicting Web Search Result Preference

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Learning user interaction models for predicting web search result preference

Learning User Interaction Models for Predicting Web Search Result Preference

Eugene Agichtein et al.

Microsoft Research

SIGIR ‘06


Objective

Objective

  • Provide a rich set of features for representing user behavior

    • Query-text

    • Browsing

    • Clickthough

  • Aggregate various feature

    • RankNet


Browsing feature

Browsing feature

  • Related work

  • The amount of reading time could predict

    • interest level on news articles

    • rating in recommender system

  • The amount of scrolling on a page also have strong relationship with interest


Browsing feature1

Browsing feature

  • How to collect browsing feature?

    • Obtain the information via opt-in client-side instrumentation


Browsing feature2

Browsing feature

  • Dwell time


Browsing feature3

Browsing feature

  • Average & Deviation

  • Properties of the click event


Clickthrough feature

Clickthrough feature

  • 1. Clicked VS. Unclicked

    • Skip Above (SA)

    • Skip Next (SN)

  • Advantage

    • Propose preference pair

  • Disadvantage

    • Inconsistency

    • Noisiness of individual


Clickthrough feature1

Clickthrough feature

  • 2. Position-biased


Clickthrough feature2

Clickthrough feature


Clickthrough feature3

Clickthrough feature


Clickthrough feature4

Clickthrough feature

  • Disadvantage of SA & SN

    • User may click some irrelevant pages


Clickthrough feature5

Clickthrough feature

  • Disadvantage of SA & SN

    • User often click part of relevant pages


Clickthrough feature6

Clickthrough feature

  • 3. Feature for learning


Feature set

Feature set


Feature set1

Feature set


Evaluation

Evaluation

  • Dataset

    • Random sample 3500 queries and their top 10 results

    • Rate on a 6-point scale manually

    • 75% training, 25% testing

    • Convert into pairwise judgment

    • Remove tied pair


Evaluation1

Evaluation

  • Pairwise judgment

  • Input

    • UrlA, UrlB

  • Outpur

    • Positive: rel(UrlA) > rel(UrlB)

    • Negative: rel(UrlA) ≤ rel(UrlB)

  • Measurement

    • Average query precision & recall


Evaluation2

Evaluation

1. Current

  • Original rank from search engine

  • 2. Heuristic rule without parameter

    • SA, SA+N

  • 3. Heuristic rule with parameter

    • CD, CDiff, CD + CDiff

  • 4. Supervised learning

    • RankNet


  • Evaluation3

    Evaluation


    Evaluation4

    Evaluation


    Evaluation5

    Evaluation


    Conclusion

    Conclusion

    • Recall is not a important measurement

    • Heuristic rule

      • very low recall and low precision

    • Feature set

      • Browsing features have higher precision


    Discussion

    Discussion

    • Is user interaction model better than search engine

      • Small coverage

      • Only pairwise judgment

    • Given the same training data, which one is better, traditional ranking algorithm or user interaction?

    • Which feature is more useful?


  • Login