Temporal Diversity in Recommender Systems

1. Temporal Diversity in Recommender Systems Neal Lathia, Stephen Hailes, Licia Capra, and Xavier Amatriain SIGIR 2010 April 6, 2011 Hyunwoo Kim

2. Outline Introduction Why Temporal Diversity? Evaluating for Diversity Promoting Temporal Diversity Conclusion

3. Introduction Collaborative Filtering [Kim, ECRA2010]

4. Introduction • User’s interest changes over time [Zheng, ESWC2011] baby health education in 2006 Alice in 2011

5. Introduction • A problem with current evaluation techniques • No temporal characteristics of the produced recommendations • In this work, • Diversity of top-N lists over time

6. Why Temporal Diversity? • Two perspectives • Changes that CF data undergoes over time • How surveyed users respond to recommendations with varying levels of diversity • Changes over time • Continuous rating of content • Recommender systems have to make decisions based on INCOMPLETE and CHANGING data • A list at any particular time is likely to be different with previous list • Do these changes translate into different recommendations over time?

7. Why Temporal Diversity? • User survey • Popular movies from

8. Why Temporal Diversity? In S3, some users commented: “appeared to very random” “varied widely” “avoided box office hits” … In S1, some users commented: “lack of diversity persisted” “toonaïve” “not working” “decreased interest” … Users are responding to the impression of the recommender system!! • Usersurvey • S1: popular movies with no diversity • S2: popular movies with diversity • S3: randomly selected movies

9. Why Temporal Diversity? • Qualities in recommendations • ACCURATE recommendations • CHANGE OVER TIME • NEW recommendations

10. Evaluating for Diversity • How diverse CF algorithms are over time • Baseline: item’s mean rating • Item-based k-Nearest Neighbor (kNN) • Matrix factorization approach based on Singular Value Decomposition (SVD) • Dataset • Netflix prize dataset • To improve the accuracy of predictions about how much someone is going to enjoy a movie based on their movie preferences • $1,000,000 grand prize on September 21, 2009

11. Evaluating for Diversity Diversity = 1/5 This week’s list Last week’s list Diversity and novelty

12. Evaluating for Diversity Novelty = 2/5 This week’s list Previous recommendations Diversity and novelty

13. Evaluating for Diversity • Diversity results and analysis • Baseline produces little to no diversity • Factorization and nearest neighbor approaches increment diversity

14. Evaluating for Diversity • Novelty results and analysis • Novelty values are lower than diversity values • When different a recommendation appears, it is a recommendation at some point in the past

15. Evaluating for Diversity • How diversity relates to accuracy • RMSE: Root Mean Squared Error • Different algorithms often overlap and kNN CF is sometimes less accurate than the baseline

16. Promoting Temporal Diversity • Diversity comes at the cost of accuracy • When promoting diversity, we must continue to take into account users’ preferences • Three methods • Temporal switching • Temporal user-based switching • Re-ranking frequent visitors’ lists

17. Promoting Temporal Diversity 5th 1st 2nd 3rd 4th kNN SVD kNN SVD kNN kNN SVD kNN SVD kNN user login user login user login Temporal switching Temporal user-based switching

18. Promoting Temporal Diversity • Temporal switching from a system

19. Promoting Temporal Diversity Temporal user-based switching

20. Promoting Temporal Diversity Diversity 40% Top-5 list Full list Re-ranking list Re-ranking frequent visitors’ lists

21. Promoting Temporal Diversity • Re-ranking frequent visitors’ lists • Only a single CF algorithm is used

22. Conclusion • What we found • State-of-the-art CF algorithms produce low temporal diversity • They repeatedly recommend the same top-N items to users • What we did • A metric to measure temporal diversity • A fine-grained analysis of the factors that may influence diversity • Future work • How novel items find their way into recommendations • How user rating patterns can be used to improve recommender system’s resilience to attack