Learning user interaction models for predicting web search result preference
Download
1 / 23

Learning User Interaction Models for Predicting Web Search Result Preference - PowerPoint PPT Presentation


  • 72 Views
  • Uploaded on

Learning User Interaction Models for Predicting Web Search Result Preference. Eugene Agichtein et al. Microsoft Research SIGIR ‘06. Objective. Provide a rich set of features for representing user behavior Query-text Browsing Clickthough Aggregate various feature RankNet.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about ' Learning User Interaction Models for Predicting Web Search Result Preference' - justina-wilcox


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Learning user interaction models for predicting web search result preference

Learning User Interaction Models for Predicting Web Search Result Preference

Eugene Agichtein et al.

Microsoft Research

SIGIR ‘06


Objective
Objective

  • Provide a rich set of features for representing user behavior

    • Query-text

    • Browsing

    • Clickthough

  • Aggregate various feature

    • RankNet


Browsing feature
Browsing feature

  • Related work

  • The amount of reading time could predict

    • interest level on news articles

    • rating in recommender system

  • The amount of scrolling on a page also have strong relationship with interest


Browsing feature1
Browsing feature

  • How to collect browsing feature?

    • Obtain the information via opt-in client-side instrumentation


Browsing feature2
Browsing feature

  • Dwell time


Browsing feature3
Browsing feature

  • Average & Deviation

  • Properties of the click event


Clickthrough feature
Clickthrough feature

  • 1. Clicked VS. Unclicked

    • Skip Above (SA)

    • Skip Next (SN)

  • Advantage

    • Propose preference pair

  • Disadvantage

    • Inconsistency

    • Noisiness of individual


Clickthrough feature1
Clickthrough feature

  • 2. Position-biased




Clickthrough feature4
Clickthrough feature

  • Disadvantage of SA & SN

    • User may click some irrelevant pages


Clickthrough feature5
Clickthrough feature

  • Disadvantage of SA & SN

    • User often click part of relevant pages


Clickthrough feature6
Clickthrough feature

  • 3. Feature for learning




Evaluation
Evaluation

  • Dataset

    • Random sample 3500 queries and their top 10 results

    • Rate on a 6-point scale manually

    • 75% training, 25% testing

    • Convert into pairwise judgment

    • Remove tied pair


Evaluation1
Evaluation

  • Pairwise judgment

  • Input

    • UrlA, UrlB

  • Outpur

    • Positive: rel(UrlA) > rel(UrlB)

    • Negative: rel(UrlA) ≤ rel(UrlB)

  • Measurement

    • Average query precision & recall


Evaluation2
Evaluation

1. Current

  • Original rank from search engine

  • 2. Heuristic rule without parameter

    • SA, SA+N

  • 3. Heuristic rule with parameter

    • CD, CDiff, CD + CDiff

  • 4. Supervised learning

    • RankNet





  • Conclusion
    Conclusion

    • Recall is not a important measurement

    • Heuristic rule

      • very low recall and low precision

    • Feature set

      • Browsing features have higher precision


    Discussion
    Discussion

    • Is user interaction model better than search engine

      • Small coverage

      • Only pairwise judgment

    • Given the same training data, which one is better, traditional ranking algorithm or user interaction?

    • Which feature is more useful?


    ad