Jiajun Bu, Shulong Tan, Chun Chen, Can Wang, Hao Wu, Lijun Zhang and Xiaofei He

1. Music Recommendation by Unified Hypergraph: Combining Social Media Information and Music Content Jiajun Bu, Shulong Tan, Chun Chen, Can Wang, Hao Wu, Lijun Zhang and Xiaofei He Zhejiang University

2. Multi-type Media Fusion • Content analysis • text • Image • Audio • Video • …… • Social analysis • Friendship • Interest group • Resource collection • Tag • …… Hypergraph

3. Outlines • Music Recommendation • Social media information • Unified Hypergraph Model • Music Recommendation on Hypergraph (MRH) • Experimental results

4. Music Recommendation • We have huge amount of music available in music social communities • It is difficult to find music we would potentially like • Music Recommendation is needed! Recommended music by the Last.fm.

5. Traditional Music Recommendation • Traditional music recommendation methods only utilize limited kinds of social information • Collaborative Filtering (CF) only uses rating information • Acoustic-based method only utilizes acoustic features • Hybrid method just combines these two

6. Music Social Community [www.pandora.com] Social activities between users Actions which users can do on resources

7. Introduction to Last.fm Users can listen to music. These are music tracks this user likes best. Users can bookmark resources by tags. Users can also make friends. Users can join in groups

8. Social Media Information in Last.fm Friendships Memberships Tagging relations Listening relations Inclusion relations

9. Social Media Information • The rich social media information is valuable for music recommendation. • To build the users’ preference profiles. • To predict users’ interests from their friends. • To recommend music tracks by albums or artists. • …

10. How About Graph Model? • Use traditional graph to model social media information but fail to keep high-order relations in social media information • (u1, t1, r1) • (u1, t2, r2) • (u2, t2, r1) It is unclear whether u2bookmarks r1, r2, or both. ?

11. Unified Hypergraph Model • Using a unified hypergraph to model multi-type objects and the high-order relations • Each edge in a hypergraph, called a hyperedge, is an arbitrary non-empty subset of the vertex set • Modeling each high-order relation by a hyperedge, so hypergraphscan capture high-order relations naturally The high-order relations among the three types of objects can be naturally represented as triples. • (u1, t1, r1) • (u1, t2, r2) • (u2, t2, r1)

12. Unified Hypergraph Construction Each type of relations corresponds to a certain type of hyperedges in the unified hypergraph. tag album tag tag album The six types of objects form the vertex set of the unified hypergraph.

13. Hyperedges Construction Details :a hyperedge corresponding to each pairwise friendship :a hyperedge corresponding to each tagging relation : a hyperedge for each track-track similarity relation :a hyperedge corresponding to each group :a hyperedge for each album or artist : a hyperedge for each user-track listening relation

14. Ranking on Unified Hypergraph Setting a user as the query Tracks have more strong “hyperpaths” to the query user will get higher ranking scores … Track List

15. Notation • A unified hypergraph • : Vertex-hyperedge incidence matrix

16. Notation-2 • : the degree of a hyperedge is the number of vertices in the hyperedge: • : the degree of a vertex is the weight sum of all hyperedges the vertex belongs to: • Dv ,De and W : diagonal matrices consisting of hyperedge degrees, vertex degrees and hyperedge weights

17. Problem Definition • Given some query vertices from , rank the other vertices on the unified hypergraph according to their relevance to the queries. • : the ranking score of the i-th object • : the vector of ranking scores • : the query vector

18. Cost Function Vertices contained in many common hyperedges should have similar ranking scores Obtained ranking scores should be similar to pre-given labels The optimal ranking result is achieved when Q(f) is minimized

19. Matrix-vector Form

20. Optimal Solution Requiring that the gradient of Q(f) vanish gives the following this equation We define Note: all the matrices are highly sparse!

21. Music recommendation on Hypergraph (MRH) • The offline training phase: • Constructing matrix H and W • Computing matrix Dv and De • Calculating , where • The online recommendation phase: • Building the query vector y • Computing the ranking results f* • Recommending top tracks which not listened

22. General Ranking Framework Setting a user as the query … For friend recommendation User List … Group List For group recommendation … Tag List For topic recommendation … Track List For music recommendation … Album List For album recommendation … Artist List For artist recommendation

23. Personalized Tag Recommendation Personalized Tag recommendation for the target user and resource … Tag List Setting a user and an resource as the queries

24. Objects and Relations in Our Dataset Objects Relations

25. Compared Algorithms • R1: friendship relations • R2: membership relations • R3: listening relations • R4: tagging relations on tracks • R5: tagging relations on albums • R6: tagging relations on artists • R7: track-album inclusion relations • R8: album-artist inclusion relations • R9: similarities between tracks

26. Performance Comparison Comparison of recommendation algorithms in terms of MAP and F1. Comparison of recommendation algorithms in terms of NDCG. It is clear that our proposed algorithm significantly outperforms the other recommendation algorithms

27. Precision-Recall Curves Comparing to MRH-social, MRH uses similarity relations among tracks additionally. We find that using this acoustic information can improve the recommendation result, especially when we only care top ranking music tracks. The superiority of MRH over RUG indicates that the hypergraph is indeed a better choice for modeling complex relations in social media information CF algorithm does not work well too. This is probably because the user-track matrix in our data set is highly sparse Acoustic-based (AB) method works worst. That is because acoustic-based method incurs the semantic gap and similarities based on acoustic content are not always consistent with human knowledge Our proposed method alleviates these problems. MRH-hybrid only uses similarity relations among music tracks and listening relations, but it works much better than AB and CF

28. Social Information Contribution MRH using listening relations and tagging relations MRH using listening relations, and social relations MRH using listening relations and inclusion relations The baseline is MRH only using listening relations Comparison of MRH on different subsets of social media information in terms of MAP and F1. There is a little improvement at lower ranks obtained by social relations. Intuitively, the users’ tastes may be inferred from friendship and membership relations. Tagging relations do not improve the performance. That is because there is a strong correlation between listening relations and tagging relations, and thus the usage of tagging relations is limited Using inclusion relations among resources, we can recommend music tracks in the same or similar albums, as well as the tracks performed by the same or similar artists. So the performance is improved greatly.

29. An Example Top 5 Recommended Tracks for No.793 Dirty Window Deer Dance Spit It Out Know No. 793 System of a Down War? From: User No. 793 joins in the groups about metal and named Slipknot. Users in these groups are fans of Metallica(one of the four most popular heavy metal band. ) andSlipknot System of a Down From: The reason these three tracks are recommended is that user No. 793 likes music come from System of a Down best. Metallica From: From: System of a Down From: Slipknot

30. Conclusion • We use the unified hypergraph model to fuse multi-type media, includes multi-type social media information and music content. • Social media information is very useful for music recommendation. • Hypergraphs can accurately capture the high-order relations among various types of objects.

31. Thank You !