Michael hahsler andreas geyer schulz
1 / 17

Michael Hahsler Andreas Geyer-Schulz - PowerPoint PPT Presentation

  • Uploaded on

Evaluation of Recommender Algorithms for an Internet Information Broker based on Simple Association Rules and on the Repeat-Buying Theory WEBKDD 2002 Edmonton, Alberta, Canada. Michael Hahsler Andreas Geyer-Schulz. Introduction. Research questions

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation

PowerPoint Slideshow about ' Michael Hahsler Andreas Geyer-Schulz' - gregory-vaughn

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Michael hahsler andreas geyer schulz

Evaluation of Recommender Algorithms for an Internet Information Broker based on Simple Association Rules and on theRepeat-Buying TheoryWEBKDD 2002Edmonton, Alberta, Canada

Michael Hahsler

Andreas Geyer-Schulz

Introduction Information Broker

  • Research questions

    • Are frequent itemsets (association rules) useful to build recommender systems for information brokers?

    • How does a recommender based on association rules perform compared to a recommender based on the repeat-buying theory known from marketing research?

  • Framework for evaluation

  • Item-to-item correlation type recommender

  • Association rules / Repeat-buying theory

  • Experimental setup

  • Results and conclusion

Framework for evaluation
Framework for Evaluation Information Broker

(Fayyad et al. 1996)


Performance measures for recommender algorithms
Performance Measures for Recommender Algorithms Information Broker

  • From Machine Learning

    • Accuracy

    • Coverage

    • MAE

  • From Information Retrieval

    • Precision / recall

    • F-measure (F1) / E-measure

  • Other measures

    • Receiver Operating Characteristic (ROC)

    • Mobasher's R

Item to item correlation type of recommender
Item-to-item Correlation Type of Recommender Information Broker

  • Recommendations can be based on many sources of information (e.g. Ansari et al 2000):

    • Expert evaluations

    • Expressed preferences

    • Individual characteristics

    • Item characteristics (content based)

    • Expressed preferences of other people (collaborative)

  • We analyze non-personalized transactions where items are used together.

explicit, observations

Association rules agrawal et al 1993
Association Rules (Agrawal et al, 1993) Information Broker

  • Support-Confidence Framework

  • Model assumptions: none

  • We use only Rules with 1 item in the antecedent for recommendations

  • Criticism: confidence ignores the frequency of the consequent. Other measures of interestingness are

    • Lift - symmetric measure of co-occurrence (deviation from independence)

    • Conviction - deviation of the implication from independence

Repeat buying theory ehrenberg 1988
Repeat-buying Theory (Ehrenberg 1988) Information Broker

  • Model assumptions:

    • Stationary (in the analyzed time period)

    • "Purchases" of items follow independent Poisson processes

    • Distribution of the parameters follows a Gamma-distribution

    • Used for panel data for consumer products (scanner data)

  • LSD (NBD) gives the probability that an item (combination of items) is "purchased" 1,2,...,r times by pure chance.

Repeat buying theory recommender
Repeat-buying Theory (Recommender) Information Broker

  • Users "purchase" some items more often together than predicted

  • Filter outliers as potential recommendations


Co-occurence of

2 Web pages

from out Dept.'s

home page in the

Web Server's log

Experimental setup info broker
Experimental Setup (Info. Broker) Information Broker

  • Educational Internet Information Broker for lecture notes, research material, enrollment information,...

Experimental setup data set
Experimental Setup (Data Set) Information Broker

  • Available Data:

    6 months of transaction data (January - June 2002)

    Transactions: 25522

    Items: 3774

    Co-occurrences: 114128

  • Data Set:

    co-occurrence lists for 300 items randomly selected

    9 users from the target group evaluated 1661 co-occurrences

    561 "recommend" - 1100 "don't recommend"

Results support for association rules
Results (Support for Association Rules) Information Broker

  • Low support necessary for reasonable recall

Results lift conviction
Results (Lift / Conviction) Information Broker

  • Lift / conviction don't perform sig. better than confidence.

Results precision accuracy
Results (Precision, Accuracy) Information Broker

  • Perform better than a random choice

  • Support-Confidence sensitive to misspecification (conf)

Results mae
Results (MAE) Information Broker

  • Support-Confidence sensitive to misspecification (conf)

  • Repeat-buying seems more robust

Results parameter of the repeat buying recommender
Results (Parameter of the Information BrokerRepeat-buying Recommender)

  • Expected error II rate is below the avg. rate (exception thresholds 0, 0.1, 0.2)

Conclusion Information Broker

  • Frequent itemsets represent useful recommendationsAccuracy >70%, Precision 60-90%

  • Both algorithms perform very similar with adequate parameters.

  • Repeat-buying algorithm seems more robust to misspecification

  • Model assumptions of the repeat-buying theory need to be checked.

  • Datasets from other applications are needed to confirm findings