Learning user interaction models for predicting web search result preference
Sponsored Links
This presentation is the property of its rightful owner.
1 / 23

Learning User Interaction Models for Predicting Web Search Result Preference PowerPoint PPT Presentation


  • 47 Views
  • Uploaded on
  • Presentation posted in: General

Learning User Interaction Models for Predicting Web Search Result Preference. Eugene Agichtein et al. Microsoft Research SIGIR ‘06. Objective. Provide a rich set of features for representing user behavior Query-text Browsing Clickthough Aggregate various feature RankNet.

Download Presentation

Learning User Interaction Models for Predicting Web Search Result Preference

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Learning User Interaction Models for Predicting Web Search Result Preference

Eugene Agichtein et al.

Microsoft Research

SIGIR ‘06


Objective

  • Provide a rich set of features for representing user behavior

    • Query-text

    • Browsing

    • Clickthough

  • Aggregate various feature

    • RankNet


Browsing feature

  • Related work

  • The amount of reading time could predict

    • interest level on news articles

    • rating in recommender system

  • The amount of scrolling on a page also have strong relationship with interest


Browsing feature

  • How to collect browsing feature?

    • Obtain the information via opt-in client-side instrumentation


Browsing feature

  • Dwell time


Browsing feature

  • Average & Deviation

  • Properties of the click event


Clickthrough feature

  • 1. Clicked VS. Unclicked

    • Skip Above (SA)

    • Skip Next (SN)

  • Advantage

    • Propose preference pair

  • Disadvantage

    • Inconsistency

    • Noisiness of individual


Clickthrough feature

  • 2. Position-biased


Clickthrough feature


Clickthrough feature


Clickthrough feature

  • Disadvantage of SA & SN

    • User may click some irrelevant pages


Clickthrough feature

  • Disadvantage of SA & SN

    • User often click part of relevant pages


Clickthrough feature

  • 3. Feature for learning


Feature set


Feature set


Evaluation

  • Dataset

    • Random sample 3500 queries and their top 10 results

    • Rate on a 6-point scale manually

    • 75% training, 25% testing

    • Convert into pairwise judgment

    • Remove tied pair


Evaluation

  • Pairwise judgment

  • Input

    • UrlA, UrlB

  • Outpur

    • Positive: rel(UrlA) > rel(UrlB)

    • Negative: rel(UrlA) ≤ rel(UrlB)

  • Measurement

    • Average query precision & recall


Evaluation

1. Current

  • Original rank from search engine

  • 2. Heuristic rule without parameter

    • SA, SA+N

  • 3. Heuristic rule with parameter

    • CD, CDiff, CD + CDiff

  • 4. Supervised learning

    • RankNet


  • Evaluation


    Evaluation


    Evaluation


    Conclusion

    • Recall is not a important measurement

    • Heuristic rule

      • very low recall and low precision

    • Feature set

      • Browsing features have higher precision


    Discussion

    • Is user interaction model better than search engine

      • Small coverage

      • Only pairwise judgment

    • Given the same training data, which one is better, traditional ranking algorithm or user interaction?

    • Which feature is more useful?


  • Login