Learning user interaction models for predicting web search result preference
Download
1 / 23

Learning User Interaction Models for Predicting Web Search Result Preference - PowerPoint PPT Presentation


  • 57 Views
  • Uploaded on
  • Presentation posted in: General

Learning User Interaction Models for Predicting Web Search Result Preference. Eugene Agichtein et al. Microsoft Research SIGIR ‘06. Objective. Provide a rich set of features for representing user behavior Query-text Browsing Clickthough Aggregate various feature RankNet.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha

Download Presentation

Learning User Interaction Models for Predicting Web Search Result Preference

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Learning User Interaction Models for Predicting Web Search Result Preference

Eugene Agichtein et al.

Microsoft Research

SIGIR ‘06


Objective

  • Provide a rich set of features for representing user behavior

    • Query-text

    • Browsing

    • Clickthough

  • Aggregate various feature

    • RankNet


Browsing feature

  • Related work

  • The amount of reading time could predict

    • interest level on news articles

    • rating in recommender system

  • The amount of scrolling on a page also have strong relationship with interest


Browsing feature

  • How to collect browsing feature?

    • Obtain the information via opt-in client-side instrumentation


Browsing feature

  • Dwell time


Browsing feature

  • Average & Deviation

  • Properties of the click event


Clickthrough feature

  • 1. Clicked VS. Unclicked

    • Skip Above (SA)

    • Skip Next (SN)

  • Advantage

    • Propose preference pair

  • Disadvantage

    • Inconsistency

    • Noisiness of individual


Clickthrough feature

  • 2. Position-biased


Clickthrough feature


Clickthrough feature


Clickthrough feature

  • Disadvantage of SA & SN

    • User may click some irrelevant pages


Clickthrough feature

  • Disadvantage of SA & SN

    • User often click part of relevant pages


Clickthrough feature

  • 3. Feature for learning


Feature set


Feature set


Evaluation

  • Dataset

    • Random sample 3500 queries and their top 10 results

    • Rate on a 6-point scale manually

    • 75% training, 25% testing

    • Convert into pairwise judgment

    • Remove tied pair


Evaluation

  • Pairwise judgment

  • Input

    • UrlA, UrlB

  • Outpur

    • Positive: rel(UrlA) > rel(UrlB)

    • Negative: rel(UrlA) ≤ rel(UrlB)

  • Measurement

    • Average query precision & recall


Evaluation

1. Current

  • Original rank from search engine

  • 2. Heuristic rule without parameter

    • SA, SA+N

  • 3. Heuristic rule with parameter

    • CD, CDiff, CD + CDiff

  • 4. Supervised learning

    • RankNet


  • Evaluation


    Evaluation


    Evaluation


    Conclusion

    • Recall is not a important measurement

    • Heuristic rule

      • very low recall and low precision

    • Feature set

      • Browsing features have higher precision


    Discussion

    • Is user interaction model better than search engine

      • Small coverage

      • Only pairwise judgment

    • Given the same training data, which one is better, traditional ranking algorithm or user interaction?

    • Which feature is more useful?


    ad
  • Login