Hybrid Web Recommender Systems

1. Hybrid Web Recommender Systems Robin Burke Presentation by Jae-wook Ahn 10/04/05

2. References • Entrée system & dataset • Burke, R. (2002). Semantic ratings and heuristic similarity for collaborative filtering. AAAI Workshop on Knowledge-based Electronic Markets 2000. • Feature augmentation, mixed hybrid example • Torres, R., McNee, S., Abel, M., Konstan J., & Riedl J. (2004). Enhancing Digital Libraries with TechLens+. Proceedings of the 2004 Joint ACM/IEEE Conference on Digital Libraries. • Hybrid recommender system UI issue • Schafer, J. (2005). DynamicLens: A Dynamic User-Interface for a Meta-Recommendation System. Workshop: Beyond Personalization 2005, IUI’05. • Collaborative filtering algorithm • Sarwar, B., Karypis, G., Konstan, J., & Riedl, J. (2001). Item-based collaborative filtering recommendation algorithms. In Proceedings of the 10th international conference on World Wide Web. Hybrid Web Recommender Systems

3. Concepts and Techniques

4. Hybrid Recommender Systems • Mix of recommender systems • Recommender system classification – knowledge source • Collaborative (CF) • User’s ratings “only” • Content-based (CN) • Product features, user’s ratings • Classifications of user’s likes/dislikes • Demographic • User’s ratings, user’s demographics • Knowledge-based (KB) • Domain knowledge, product features, user’s need/query • Inferences about a use’s needs and preferences Hybrid Web Recommender Systems

5. CF vs. CN • User-based CF • Searches for similar users in user-item “rating” matrix • Item-based CF • Searches for similar items in user-item “rating” matrix • CN • Searches for similar items in item-feature matrix • Example – TF*IDF term weight vector for news recommendation Items Ratings Users Hybrid Web Recommender Systems

6. Recommender System Problems • Cold-start problem • Learning based techniques • Collaborative, content-based, demographic  Hybrid techniques • Stability vs. plasticity problem • Difficulty to change established user’s profile • Temporal discount – older rating with less influence • KB – fewer cold start problem (no need of historical data) • CF/Demographic – cross-genre niches, jump outside of the familiar (novelty, serendipity) Hybrid Web Recommender Systems

7. Strategies for Hybrid Recommendation • Combination of multiple recommendation techniques together for producing output • Different techniques of different types • Most common implementations • Most promise to resolve cold-start problem • Different techniques of the same type • Ex) NewsDude – naïve Bayes + kNN Hybrid Web Recommender Systems

8. Seven Types of Recommender Systems • Taxonomy by Burke (2002) • Weighted • Switching • Mixed • Feature combination • Feature augmentation • Cascade • Meta-level Hybrid Web Recommender Systems

9. Weighted Hybrid • Concept • Each component of the hybrid scores a given item and the scores are combined using a linear formula • When recommenders have consistent relative accuracy across the product space • Uniform performance among recommenders (otherwise  other hybrids) Hybrid Web Recommender Systems

10. Weighted Hybrid Procedure • Training • Joint rating • Intersection – candidates shared between the candidates • Union – case with no possible rating  neutral score (neither liked nor disliked) • Linear combination Hybrid Web Recommender Systems

11. Mixed Hybrid • Concepts • Presentation of different components side-by-side in a combined list • If lists are to be combined, how are rankings to be integrated? • Merging based on predicted rating or on recommender confidence • Not fit with retrospective data • Cannot use actual ratings to test if right items ranked highly • Example • CF_rank(3) + CN_rank(2)  Mixed_rank(5) Hybrid Web Recommender Systems

12. Mixed Hybrid Procedure • Candidate generation • Multiple ranked lists • Combined display Hybrid Web Recommender Systems

13. Switching Hybrid • Concepts • Selects a single recommender among components based on recommendation situation • Different profile different recommendation • Components with different performance for some types of users • Existence of criterion for switching decision • Ex) confidence value, external criteria Hybrid Web Recommender Systems

14. Switching Hybrid Procedure • Switching decision • Candidate generation • Scoring • No role for unchosen recommender Hybrid Web Recommender Systems

15. Feature Combination Hybrid • Concepts • Inject features of one source into a different source for processing different data • Features of “contributing recommender” are used as a part of the “actual recommender” • Adding new features into the mix • Not combining components, just combining knowledge source Hybrid Web Recommender Systems

16. Feature Combination Hybrid Procedure • Feature combination  In training stage • Candidate generation • Scoring Hybrid Web Recommender Systems

17. Feature Augmentation Hybrid • Concepts • Similar to Feature Combination • Generates new features for each item by contributing domain • Augmentation/combination – done offline • Comparison with Feature Combination • Not raw features (FC), but the result of computation from contribution (FA) • More flexible to apply • Adds smaller dimension Hybrid Web Recommender Systems

18. Feature Augmentation Hybrid Procedure Hybrid Web Recommender Systems

19. Cascade Hybrid • Concepts • Tie breaker • Secondary recommender • Just tie breaker • Do refinements • Primary recommender • Integer-valued scores – higher probability for ties • Real-valued scores – low probability for ties • Precision reduction • Score: 0.8348694  0.83 Hybrid Web Recommender Systems

20. Cascade Hybrid Procedure • Procedure • Primary recommender • Ranks • Break ties by secondary recommender Hybrid Web Recommender Systems

21. Meta-level Hybrid • Concepts • A model learned by contributing recommender  input for actual recommender • Contributing recommender completely replaces the original knowledge source with a learned model • Not all recommenders can produce the intermediary model Hybrid Web Recommender Systems

22. Meta-level Hybrid Procedure • Procedure • Contributing recommender  Learned model • Knowledge Source Replacement • Actual Recommender Hybrid Web Recommender Systems

23. Experiments

24. Testbed – Entrée Restaurant Recommender • Entrée System • Case-based reasoning • Interactive critiquing dialog • Ex) Entry Candidates  “Cheaper”  Candidates  “Nicer”  Candidates  Exit • Not “narrowing” the search by adding constrains, but changing the focus in the feature space Hybrid Web Recommender Systems

25. Testbed – Entrée Restaurant Recommender (cont’d) • Entrée Dataset • Rating • Entry, ending point – “positive” rating • Critiques – “negative” rating • Mostly negative ratings • Validity test for positive ending point assumption – strong correlation between original vs. modified (entry points with positive ratings) • Small in size Hybrid Web Recommender Systems

26. Evaluation Methodology • Measures • ARC (Average Rank of the Correct recommendations) • Accuracy of retrieval • At different size retrieval set • Fraction of the candidate set (0 ~ 1.0) • Training & Test set • 5 fold cross validation – random partition of training/test set • “Leave one out” methodology – randomly remove one item and check whether the system can recommend it • Sessions Sizes • Single visit profiles – 5S, 10S, 15S • Multiple visit profiles – 10M, 20M, 30M Hybrid Web Recommender Systems

27. Baseline Algorithms • Collaborative Pearson (CFP) • Pearson’s correlation coefficient for similarity • Collaborative Heuristic (CFH) • Heuristics for calculating distances between critiques • “nicer” and “cheaper”  dissimilar • “nicer” & “quieter”  similar • Content-based (CN) • Naïve Bayes algorithm – compute probability that a item is “liked” / “disliked” • Too few “liked” items  modified candidate generation • Retrieve items with common features with the “liked” vector of the naïve Bayes profile • Knowledge-based (KB) • Knowledge-based comparison metrics of Entrée • Nationality, price, atmosphere, etc. Hybrid Web Recommender Systems

28. Baseline Evaluations • Techniques vary in performance on the Entrée data • Content-based(CN) – weak • Knowledge-based (KB) – better on single-session than multi-session • Heuristic collaborative (CFH) – better than correlation-based (CFP) for short profiles • Room for improvement • Multi-session profiles Hybrid Web Recommender Systems

29. Baseline Evaluations Hybrid Web Recommender Systems

30. Hybrid Comparative Study • Missing components • Mixed hybrid • Not possible with retrospective data • Demographic recommender • No demographic data Hybrid Web Recommender Systems

31. Results – Weighted • Hybrid performance better in only 10 of 30 • CN/CFP – consistent synergy (5 of 6) • Lacks uniform performance • KB, CFH • Linear weighting scheme assumption – fault Hybrid Web Recommender Systems

32. Results – Switching • KB hybrids – best switching hybrids Hybrid Web Recommender Systems

33. Results – Feature Combination • CN/CFH, CN/CFP • Contributing CN • Identical to CFH, CFP • CFH maintains accuracy with reduced dataset • CF/CN Winnow – modest improvement Hybrid Web Recommender Systems

34. Results – Feature Augmentation • Best performance so far • Particularly CN*/CF* • Good for multi-session profiles Hybrid Web Recommender Systems

35. Results – Cascade • CFP/KB, CFP/CN • Great improvement • Also good for multi-profile sessions Hybrid Web Recommender Systems

36. Results – Meta-level Hybrids • CN/CF, CN/KB, CF/KB, CF/CN • Not effective • No synergy • Weakness of KB/CN in Entrée dataset • Both components should be strong Hybrid Web Recommender Systems

37. Discussion • Dominance of the hybrids over basic recommenders • Synergy was found under • Smaller profile size • Sparse recommendation density •  hybridization conquers cold start problem Hybrid Web Recommender Systems

38. Discussion (cont’d) • Best hybrids • Feature augmentation, cascade • FA allows a contributing recommender to make a positive impact • without interfering with the performance of the better algorithm Hybrid Web Recommender Systems

39. Conclusions • Knowledge-based recommendation is not limited • Numerously combined to build hybrids • Good for secondary or contributing components • Cascade hybrids are effective • Though rare in literatures • Effective for combining recommender with different strengths • Different performance characteristics • Six hybridization techniques • Relative accuracy & consistency of hybrid components Hybrid Web Recommender Systems

40. System Example & Related Issues

41. System Example – TechLens+ • Hybrid recommender system • Recommenders – CF, CN • Hybrid algorithms – CF/CN FA, CN/CF FA, Fusion (Mixed) • Corpus • CiteSeer • Title, abstract (CN), citations (CF) • Methodology • Offline experiment, Online user study with questionnaire (by asking satisfaction on the recommendation) • Results • Fusion was the best • Some FA were not good due the their sequential natures • Different algorithms should be used for recommending different papers • Users with different levels of experiences perceive recommendations differently Hybrid Web Recommender Systems

42. Meta-recommender – DynamicLens • Can user provided information improve hybrid recommender system output? • Meta-recommender • Provide users with personalized control over the generation of a recommendation list from hybrid recommender system • MetaLens • IF (Information Filtering), CF Hybrid Web Recommender Systems

43. Meta-recommender – DynamicLens (cont’d) • Dynamic query • Merges preference & recommendation interfaces • Immediate feedback • Discover why a given set of ranking recommendations were made Hybrid Web Recommender Systems

44. Questions & Comments