1 / 26

Visually and Acoustically Exploring the High-Dimensional Space of Music

Visually and Acoustically Exploring the High-Dimensional Space of Music. Lukas Bossard Michael Kuhn Roger Wattenhofer SocialCom 2009 Vancouver, Canada. organization by album. History. Storage media Vinyl records Compact cassettes Compact discs

Download Presentation

Visually and Acoustically Exploring the High-Dimensional Space of Music

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Visually and Acoustically Exploring the High-Dimensional Space of Music Lukas Bossard Michael Kuhn Roger Wattenhofer SocialCom 2009 Vancouver, Canada

  2. organization by album History • Storage media • Vinyl records • Compact cassettes • Compact discs • An Album is stored on a single physical storage medium • Sequence of songs given by album • Album is typically listened to as a whole Michael Kuhn, ETH Zurich @ SocialCom 2009

  3. Music today • Huge offer, easily available • filesharing, iTunes, amazon, etc. • Large collections • The entire collection is stored on a single electronic storage medium • Organization by albums (and other lists) is no longer appropriate organize by similarity Michael Kuhn, ETH Zurich @ SocialCom 2009

  4. Organization by Similarity • Our Goals • Mobile application (portable player) • Play songs the user likes • Overview of a collection • Problems on mobile devices • Limited input • Limited output • Limited processing power • Limited memory • Solution • Use song coordinates provided by www.musicexplorer.org Michael Kuhn, ETH Zurich @ SocialCom 2009

  5. Which songs are similar? • Goussevskaia et al., WI 2008: • Each song is positioned in a Euclidean „Map of Music“ • Similar songs are close to each other in this Euclidean space Michael Kuhn, ETH Zurich @ SocialCom 2009

  6. The Map of Music • Based on usage data • „behaviour of the crowd“ • Gathered from social music platform (last.fm) • NO audio-analysis! • Underlying similarity measure • Item-to-item collaborative filtering (Amazon)[Linden et al., IEEE Internet Computing] • „users who listen to song A also listen to song B“ • Coordinates available through webservice • www.musicexplorer.org Michael Kuhn, ETH Zurich @ SocialCom 2009

  7. rock Hey Jude Praise you Imagine Galvanize electronic I want it that way My Prerogative pop Using the Map • Similar songs are close to each other • Quickly find nearest neighbors • Span (and play) volumes • Create smooth playlists by interpolation • Visualize a collection • Low memory footprint • Well suited for mobile domain convenient basis to build music software Michael Kuhn, ETH Zurich @ SocialCom 2009

  8. That‘s easy – is it? 10 dimensional! Michael Kuhn, ETH Zurich @ SocialCom 2009

  9. Contributions Proof-of-concept application for Android devices („Google-phone“) Visual and acoustic guide to the high-dimensional music galaxy Michael Kuhn, ETH Zurich @ SocialCom 2009

  10. Visual Exploration Visual Exploration Michael Kuhn, ETH Zurich @ SocialCom 2009

  11. fast happy sad slow The Reference: SensMe (Sony Ericsson) Create playlist by selecting areas Based on audio-analysis Michael Kuhn, ETH Zurich @ SocialCom 2009

  12. Global Overview Orientation Local Overview Requirements Our problem: 10 dimensions! Michael Kuhn, ETH Zurich @ SocialCom 2009

  13. Lens Metaphor Detailed view in the center Few details in the border rings Michael Kuhn, ETH Zurich @ SocialCom 2009

  14. Lens: Recursive Clustering Few details in the border regions High resolution in the center Michael Kuhn, ETH Zurich @ SocialCom 2009

  15. Cake Metaphor Used to represent song clusters Michael Kuhn, ETH Zurich @ SocialCom 2009

  16. The Visual Exploration Interface • Browsing • Touch cluster to bring it to the center • Playlist Generation • Select a number of seed songs • Playlist will consist of songs around these seeds • Similar to SensME (but songs are selected in a different interface) Touch to make this area the new center Michael Kuhn, ETH Zurich @ SocialCom 2009

  17. Evaluation (1) • User Experiment • 9 participants • Collection (1400 songs) • 5 minutes to create playlist of 20 songs (for both systems) • Evaluation: Participants had to... • ...rate each individual song in the playlists • ...fill in a questionaire vs. Michael Kuhn, ETH Zurich @ SocialCom 2009

  18. Evaluation (2) • Average song rating (scale: 0..10): • 5.5 (SensMe) • 6.3 (this paper) • Questionaire (scale: 1..5): Trade-off: Accurracy of high-dimensional space versus simplicity of interface Michael Kuhn, ETH Zurich @ SocialCom 2009

  19. Acoustic Exploration Michael Kuhn, ETH Zurich @ SocialCom 2009

  20. Shuffle (play songs in random order) Idea Can we do better? Yes! Idea: Learn on the fly which songs the user likes! Skip = bad song Listen = good song Michael Kuhn, ETH Zurich @ SocialCom 2009

  21. Realization Supposed to be the user‘s region of interest Basic algorithm: Voronoi Tesselation First song was skipped Michael Kuhn, ETH Zurich @ SocialCom 2009

  22. Improvements • Weighting • Account for strong/weak feedback • Aging • Allows to adapt to changing mood • Centering • Border regions are risky => go to center • Escaping • Sometimes play random song to avoid getting stuck somewhere Rating bar (left = skip, right = good) Michael Kuhn, ETH Zurich @ SocialCom 2009

  23. References • Random shuffling (e.g. iPod-Shuffle) • Pampalk et al. (ISMIR, 2005) • Designed for (Euclidean) audio feature spaces If there are songs with dg < db: select such song with smallest dg Else: select song with largest ratio dg/db dg db Michael Kuhn, ETH Zurich @ SocialCom 2009

  24. Evaluation • 9 Participants • Song ratings are used as input and for evaluation Ratings clearly better than random Diversity clearly better than Pampalk Michael Kuhn, ETH Zurich @ SocialCom 2009

  25. www.musicexplorer.org Conclusion Michael Kuhn, ETH Zurich @ SocialCom 2009

  26. Questions? Michael Kuhn, ETH Zurich @ SocialCom 2009

More Related