1 / 41

Music Information Retrieval

Speaker Associate Professor Ning-Han Liu. Music Information Retrieval. What’s MIR. Music information retrieval ( MIR ) is the interdisciplinary science of retrieving information from music. MIR is a small but growing field of research with many real-world applications. Applications of MIR.

ember
Download Presentation

Music Information Retrieval

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Speaker Associate Professor Ning-Han Liu Music Information Retrieval

  2. What’s MIR • Music information retrieval (MIR) is the interdisciplinary science of retrieving information from music. • MIR is a small but growing field of research with many real-world applications.

  3. Applications of MIR • Recommender systems • Track separation and instrument recognition • Automatic score creation • Automatic categorization • Music generation

  4. Methods used in MIR-Data Source • Music Content • Symbolic Data Formats • Scores • MIDI • Digital Audio Formats • WAV • Mp3 • Metadata • Song name • Singer • Performer • tags

  5. Methods used in MIR-Feature Representation • Mel-Frequency Cepstral Coefficient (MFCC) • is a measure of the timbre of a piece of music. • Chords • Harmonics • Melody • Main pitch • Beats per minute

  6. Methods used in MIR-Statistics and Machine Learning • Music analysis and knowledge representation • Classification, clustering and modeling

  7. Famous Application • iTunes: is a media player computer program, used for playing, downloading, saving, and organizing digital music and video files on desktop or laptop personal computers.

  8. Famous Application • KKBOX

  9. Famous Application • Nike+iPod

  10. Previous research issue I • Query music efficiently • Music representation • ‘U’,’D’,’S’ • Rhythm • Chords • Index structure • Link list • Suffix tree • Query process • Exact matching • Approximate matching

  11. Previous research issue I • String Matching • Exact String Matching • E.g. sol-mi-mi-fa-re-re-do-re-mi-fa-sol-sol-sol • Query: do-re-mi • To find whether data contain the query melody string • Algorithms: KMP, Boyer-Moore • Problem • Cannot expect users to precisely specify music query • Need to retrieve all entire melody string

  12. Previous research issue I • String Matching • Approximate String Matching • Edit distance is often used as similarity measure • E.g. sol-mi-mi-fa-re-re-do-re-mi-fa-sol-sol-sol • Query: do-mi matching do-re-mi with edit distance=1

  13. Previous research issue I • String Matching Algorithm Using Index • Arbee L.P. Chen (ICMCS 1999)

  14. Previous research issue I • Approximate Matching

  15. Previous research issue I • Query by Music Segments • Arbee L.P. Chen (ICME 2000)

  16. Previous research issue I • Query by Music Segments • Index: Suffix Tree

  17. Previous research issue II • Music Recommendation • Content based filtering (CBF) • predicts user preferred data items by matching the representations of the data items relevant to the user • Collaborative filtering (CF) • uses the correlations between users on the basis of their ratings to predict items for users • Mixed method • Combine CBF and CF

  18. Previous research issue II • A music recommendation system based on music data grouping and user interests • Arbee (ICME 2000)

  19. Previous research issue II

  20. Novel research topic I • Playlist Generation • An Intelligent Music Playlist Generator based on the Time Parameter with Artificial Neural Networks (Liu, 2010) N.H. Liu, S.J. Hsieh, C.F. Tsai, "An Intelligent Music Playlist Generator based on the Time Parameter with Artificial Neural Networks," Expert systems with applications. Volume 37, Issue 4, April (2010), Pages 2815-2825 . (SCI, impact factor=2.9, EI)

  21. Novel research topic I • The structure of the mixed artificial neural networks

  22. Novel research topic II • JoMP: A Mobile Music Player Agent for Joggers based on User Interest and Pace (Liu, 2009) N.H. Liu, H.Y. Kung, "JoMP: A Mobile Music Player Agent for Joggers based on User Interest and Pace," IEEE Transactions on Consumer Electronics, Volume 55, No. 4, Nov, (2009 (SCI, EI)

  23. Novel research topic II • System prototype

  24. Novel research topic III • Intelligent Music Player for Bike Sport Using EEG & GPS Sensors • Brainwave earphone (NeuroSky™ Minset) and wearing method. N.H. Liu, H.M. Hsu, H.C. Chu, and S.H. Hsu "Intelligent Music Player for Bike Sport Using Electroencephalogram and Global Positioning System Sensors" Sensor Letters, Volume 11, Number 5, pp. 772-780 (2013) (SCI).

  25. Novel research topic III • The system is based on the Fuzzy Inference System (FIS), combined with EEG and GPS sensors to measure the sport data of cyclists.

  26. Novel research topic IV • Intelligent car audio system in cloud N.H. Liu "Design of an Intelligent Car Radio and Music Player System," accepted by Multimedia Tools and Applications. (2013).(SCI, IF=1.014) DOI: 10.1007/s11042-013-1467-z

  27. Novel research topic V • Car audio system with doze prevention

  28. Novel research topic V • Car audio system with doze prevention N.H. Liu, C.Y. Chiang, H.M. Hsu, "Improving Driver Alertness through Music Selection Using a Mobile EEG to Detect Brainwaves"  Sensors, 13(7):8199-8221 (2013)(SCI, IF=1.953).

  29. Novel research topic V • Car audio system with doze prevention

  30. Novel research topic VI • Distance estimation methods • Methods to calculate a personalized distance measure between different pieces of music based on user preferences. • The questions ask the user to rate the similarity between songs selected by the system. N.H. Liu "Comparison of Content-based Music Recommendation Using Different Distance Estimation Methods," Applied Intelligence, Volume 38, issue 2, pp. 160-174, Mar. (2013).(SCI, IF=1.853)

  31. Novel research topic VI • Weighted squared Euclidean distance function generated from maximum likelihood estimation

  32. Novel research topic VI • Weighted squared Euclidean distance function generated from genetic algorithm Define a fitness function to evaluate the chromosome performance

  33. Novel research topic VI • Distance function generated by genetic programming The fitness function is defined as follows:

  34. Novel research topic VII • Music Selection Interface for Car Audio System • In the architecture, users are able to select a genre of music or a playlist from through a 2D interface. Self-organizing map, depending on a personalized distance function and music contents, is utilized to map music tracks to the interface.

  35. Novel research topic VII • Prototype in car N.H.Liu, "Music Selection Interface for Car Audio System Using SOM with Personal Distance Function,"  EURASIP Journal on Audio, Speech, and Music Processing,  2013:20. (2013) (SCI)

  36. Novel research topic VIII • Query by Singing/Humming • When a user cannot remember the title of a song, or its related details, the most direct and convenient method to search for the song is by humming a section of it. • The background of the user often influences the genres of the songs being searched. • We use the information from a user’s search history, as well as the properties of genres common to users with similar backgrounds, to estimate the genre or style the current user may be interested in based on a probability calculation.

  37. Novel research topic VIII • System flow chart

  38. Novel research topic VIII • The Singing/Humming signals process

  39. Novel research topic VIII • Ranking by User’s Preference

  40. Novel research topic VII • Ranking through Similar Users’ Records naïve Bayesian prediction

  41. Novel research topic VII • User Interface N.H. Liu, "Effective Results Ranking for Mobile Query by Singing/Humming Using a Hybrid Recommendation Mechanism," IEEE Transactions on Multimedia, Aug (2014)(SCI, IF=1.754)

More Related