Adapting Recommendation Diversity to Openness to Experience: A Study of Human Behaviour

1. Adapting Recommendation Diversity to Openness to Experience: A Study of Human Behaviour Nava Tintarev, Matt Dennis and Judith Masthoff University of Aberdeen

2. UMAP’2013. Rome, Italy Outline • Personality and recommender systems • Experiment – openness to experience and diversity • Results • Limitations • Implications for recommender systemsdesign

3. UMAP’2013. Rome, Italy Personality traits • Generally it is assumed that: • a) traits are relatively stable over time, • b) traits differ among individuals (for instance, some people like to try new things while others prefer to stick to known options), and • c) traits influence behaviour (e.g. ordering familiar food at a restaurant). • Five factor model (Big Five): Extraversion, Agreeableness, Conscientiousness, Emotional Stability, and Openness to Experience • Openness to Experience: active imagination, aesthetic sensitivity, attentiveness to inner feelings, preference for variety, and intellectual curiosity

4. UMAP’2013. Rome, Italy Personality + Recsys == TRUE? • Recommendation is not all about accuracy. • The tailoring of recommender systems to personality has been found to • improve accuracy for sparse data sets and new users (Hu and Pu 2011), • to help predict choices for presidential candidates (Nunes 2008), • to positively impact the acceptance of a system and recommendations (Hu and Pu 2009,Wu et al 2013) • Openness to experience may make users more receptive to more diverse and potentially serendipitous recommendations.

5. UMAP’2013. Rome, Italy So what makes for “good” diversity

6. UMAP’2013. Rome, Italy Diversity != Serendipity • Unexpected and helpful (Ge et al 2010) • Topic diversification approach based on taxonomy-based dissimilarity (Ziegler et al. ,2005). Impacted accuracy negatively. • Re-rank a list of top items was found to improve diversity without a great loss in accuracy (Adomavicius and Kwon, 2011) • Users preferred recommendations from a diversified set of clusters(categorical diversity?), rather than within clusters (thematic diversity?). (Abbassi et al., 2012)

7. UMAP’2013. Rome, Italy Research Questions • 1) there may be a difference in preference for the degree of diversity in recommendations among users, • and • 2) within category vs across category diversity in recommendations has not received a great deal of weight in previous literature and would benefit from empirical testing with users.

8. UMAP’2013. Rome, Italy Experiment: User-as-wizard

9. UMAP’2013. Rome, Italy Experiment - participants • Amazon Mechanical Turk • 120 participants (128 excluded) • 57% female, 41% male, 2% undisclosed • Openness to Experience within range for the normal population (TIPI). • Average completion time 5 minutes (up to 30)

10. UMAP’2013. Rome, Italy Meet Oliver (Dennis et al 2010)

11. UMAP’2013. Rome, Italy Procedure • Recommend three items (books) • Vary along three dimensions • author (0,1) Same or different • genre (0,0.3, 1) Same, similar or different • themes (0,0.3,1) Almost all themes in common, some themes in common, or no themes in common • Had to justify their choice before moving on to the next recommendation.

12. UMAP’2013. Rome, Italy

13. UMAP’2013. Rome, Italy Results

14. UMAP’2013. Rome, Italy Results • No statistically significant effect of story • Effect of order in sequence • But a tendency toward a difference in application of thematic and categorical diversity

15. UMAP’2013. Rome, Italy Effect of story • 2-3 things changed! • Slightly higher for openness_high • Difference not reliable – possible ceiling effect • No correlation between the participants (aggregated TIPI score on) openness to experience and diversity.

16. UMAP’2013. Rome, Italy Order Book2 >div Book2 Book3 >div Book3 p < 0.01 (Bonferroni corrected) Openness_low starts lowest but `catches up’ by Book3!

17. UMAP’2013. Rome, Italy Ways of applying diversity • Tendency toward more genre diversity for openness_high • Tendency toward more theme diversity for openness_low • We need to repeat this study!

18. UMAP’2013. Rome, Italy Results (again) • No statistically significant effect of story • Effect of order in sequence • But a tendency toward a difference in application of thematic and categorical diversity

19. UMAP’2013. Rome, Italy Limitations • Domain • Is this what people need or what people do? • Predicting openness to experience

20. UMAP’2013. Rome, Italy Possible implications for recommender systems design • Start narrow go broader • Shift focus toward more thematic diversity (within cluster) for people who are low on openness to experience. • Diversity across genres (across clusters) is still relevant for the majority of users • Worth looking into predicting openness to experience

21. UMAP’2013. Rome, Italy Wrap-up • We studied the effect of openness to experience on rec diversity • People like to expand each other’s horizons • They start narrow and go wide • Do not really consider personality • But tends toward more thematic diversity for low OE vs more categorical diversity for low OE.

22. UMAP’2013. Rome, Italy Questions? • n.tintarev@abdn.ac.uk • This research has been funded by the Engineering and Physical Sciences Research Council (EPSRC, UK), grant ref. EP/J012084/1