Music Recommendation Systems: A Progress Report

1. Music Recommendation Systems:A Progress Report Adam Berenzweig April 19, 2002

2. Music Recommendation Is: • Music IR for the masses • Kids in candy stores • Querying is hard; people can’t describe music • Recommendation can be integrated into players, streaming services, music stores, etc. • Break major label/retail monopoly on choice!!

3. Music Recommendation Is: • Set-based IR • “Find me items similar to this set, in the way that the set is similar to itself” • Set = collection, or playlist extension. • Be sensitive to themes or aspects of the user’s collection. • All about similarity

4. Background I: IR/Statistics • Collaborative Filtering • Latent Semantic Analysis (Deerwester & al., ‘90) • SVD to find hidden meaning • Probabilistic LSA (Hofmann, ‘99) • EM to find hidden meaning • Latent Class Models (Hofmann, ‘99)

5. Background II: Audio IR • Artist classification • Whitman & Lawrence; Berenzweig, Ellis & Lawrence • Genre classification • Tzanetakis • Fingerprinting, query-by-example • What features??? • What is it I like about the music that I like?

6. Artist Classification Using Vocals • Anchor Models • Similarity Metrics

7. Artist Classification Using Vocals • Are vocal segments more easy to identify than instrumental segments? • “Using Voice Segments to Improve Artist Classification of Music”, Berenzweig, Ellis & Lawrence, to appear AES 22.

8. Segmented Posteriograms

9. Segmented Posteriograms

10. Experiment at-a-glance Frame labels Song labels Audio input Cepstra (MFCC, PLP) Artist Classifier Confidence Weighting Vox/Music Classifier

11. Results

12. The Album Effect • Testing on different album than trained hurts performance by 30-40% relative. • Is it production effects or style?

13. Future Work • Album Effect: production or style? • Better segmentation • Further analysis of posteriograms • song structure: change detection, clustering • another level of classification? leads to...

14. Artist Classification Using Vocals • Anchor Models • Similarity Metrics

15. Anchor Models • Dual Motivation: • Scalable artist classification • Induced artist similarity metric • Technique from speaker identification literature (Reynolds, Sturim & al.)

16. Anchor Models Anchor Models

17. Anchor Space • n-dimensional Euclidean space • Distance metric is simple • Dimensions have meaning

18. Anchor Models • Basically doing dimensionality reduction or feature extraction, where • nonlinear mapping to low-D feature space is learned • mapping is musically relevant • but no theoretical justification like PCA

19. Anchor Space • Artists are distributions, not points. • Model with GMMs • Each frame of audio (32 milliseconds) is a point. • Each song is a cloud, too. • Distance is KL-divergence • estimate with total likelihood under GMM.

20. Artist Classification Using Vocals • Anchor Models • Similarity Metrics

21. Searching for Ground Truth • Does a single “correct” similarity metric exist? • Subjective, relative, mood-dependent. • Aspects of similarity - Tversky ‘77 • (Psychological) similarity is not a metric. • A “dynamic interplay between classification & similarity”

22. Similarity is not a metric? An ellipse is like a circle. A circle is like an ellipse. No Triangle Inequality Asymmetry

23. Salient Aspects • Distance in big Euclidean space may not have any meaning! • Goal: find big Euclidean space, then analyze salient dimensions of collections. • Directly answers the question: what is it I like about the music that I like?

24. Sources of Opinion Ask directly? Preference Data: Spidering opennap lists. Expert Opinion: Allmusic Guide “Similar Artist” sections. Semantic Similarity: Whitman & Lawrence Searching for Ground Truth

25. “Community Metadata”. (Whitman and Lawrence) Web spider collects terms. Treats artists like documents Semantic Similarity

26. Expert Opinion

27. Completing the Erdos Numbers Incomplete Graph Complete Erdos Distance

28. Want many judgements, but full matrix not likely Problem of relativity, drift Ask for relative judgements Game and Survey mode Problem of unknown artists Use total history Human Evaluation

29. Musicseer

30. Evaluation: Ranking

31. Thanks!