1 / 17

Michael Hahsler Andreas Geyer-Schulz

Evaluation of Recommender Algorithms for an Internet Information Broker based on Simple Association Rules and on the Repeat-Buying Theory WEBKDD 2002 Edmonton, Alberta, Canada. Michael Hahsler Andreas Geyer-Schulz. Introduction. Research questions

gregory-vaughn
Download Presentation

Michael Hahsler Andreas Geyer-Schulz

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Evaluation of Recommender Algorithms for an Internet Information Broker based on Simple Association Rules and on theRepeat-Buying TheoryWEBKDD 2002Edmonton, Alberta, Canada Michael Hahsler Andreas Geyer-Schulz

  2. Introduction • Research questions • Are frequent itemsets (association rules) useful to build recommender systems for information brokers? • How does a recommender based on association rules perform compared to a recommender based on the repeat-buying theory known from marketing research? • Framework for evaluation • Item-to-item correlation type recommender • Association rules / Repeat-buying theory • Experimental setup • Results and conclusion

  3. Framework for Evaluation (Fayyad et al. 1996) added

  4. Performance Measures for Recommender Algorithms • From Machine Learning • Accuracy • Coverage • MAE • From Information Retrieval • Precision / recall • F-measure (F1) / E-measure • Other measures • Receiver Operating Characteristic (ROC) • Mobasher's R

  5. Item-to-item Correlation Type of Recommender • Recommendations can be based on many sources of information (e.g. Ansari et al 2000): • Expert evaluations • Expressed preferences • Individual characteristics • Item characteristics (content based) • Expressed preferences of other people (collaborative) • We analyze non-personalized transactions where items are used together. explicit, observations

  6. Association Rules (Agrawal et al, 1993) • Support-Confidence Framework • Model assumptions: none • We use only Rules with 1 item in the antecedent for recommendations • Criticism: confidence ignores the frequency of the consequent. Other measures of interestingness are • Lift - symmetric measure of co-occurrence (deviation from independence) • Conviction - deviation of the implication from independence

  7. Repeat-buying Theory (Ehrenberg 1988) • Model assumptions: • Stationary (in the analyzed time period) • "Purchases" of items follow independent Poisson processes • Distribution of the parameters follows a Gamma-distribution • Used for panel data for consumer products (scanner data) • LSD (NBD) gives the probability that an item (combination of items) is "purchased" 1,2,...,r times by pure chance.

  8. Repeat-buying Theory (Recommender) • Users "purchase" some items more often together than predicted • Filter outliers as potential recommendations Outliers Co-occurence of 2 Web pages from out Dept.'s home page in the Web Server's log

  9. Experimental Setup (Info. Broker) • Educational Internet Information Broker for lecture notes, research material, enrollment information,...

  10. Experimental Setup (Data Set) • Available Data: 6 months of transaction data (January - June 2002) Transactions: 25522 Items: 3774 Co-occurrences: 114128 • Data Set: co-occurrence lists for 300 items randomly selected 9 users from the target group evaluated 1661 co-occurrences 561 "recommend" - 1100 "don't recommend"

  11. Experimental Setup (Evaluated Data)

  12. Results (Support for Association Rules) • Low support necessary for reasonable recall

  13. Results (Lift / Conviction) • Lift / conviction don't perform sig. better than confidence.

  14. Results (Precision, Accuracy) • Perform better than a random choice • Support-Confidence sensitive to misspecification (conf)

  15. Results (MAE) • Support-Confidence sensitive to misspecification (conf) • Repeat-buying seems more robust

  16. Results (Parameter of the Repeat-buying Recommender) • Expected error II rate is below the avg. rate (exception thresholds 0, 0.1, 0.2)

  17. Conclusion • Frequent itemsets represent useful recommendationsAccuracy >70%, Precision 60-90% • Both algorithms perform very similar with adequate parameters. • Repeat-buying algorithm seems more robust to misspecification • Model assumptions of the repeat-buying theory need to be checked. • Datasets from other applications are needed to confirm findings

More Related