1 / 33

Content-based Music Retrieval from Acoustic Input (CBMR)

Content-based Music Retrieval from Acoustic Input (CBMR). Outline. What is CBMR? Methods Signal processing Similarity comparison Experiment results Demo Future work. What is CBMR?. CBMR : Content-based Music Retrieval Traditional database query : Text-based or SQL-based

vic
Download Presentation

Content-based Music Retrieval from Acoustic Input (CBMR)

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Content-based Music Retrieval from Acoustic Input (CBMR)

  2. Outline • What is CBMR? • Methods • Signal processing • Similarity comparison • Experiment results • Demo • Future work

  3. What is CBMR? • CBMR : • Content-based Music Retrieval • Traditional database query : • Text-based or SQL-based • Our goal : • Music retrieval by singing/humming

  4. Related Work • Query by humming by Ghias,Loga and Chamberlin in 1995 • Autocorrelation pitch detection • 183 songs in database • MELDEX system by New Zealand Digital Library Project in 1996 • Gold/Rabiner Algorithm (800 songs) • Sing ‘la’ or ‘ta’ when transposition • Karaoke song recognizer by J.F. Wang in 1997 • Novel pitch detection • 50 songs in database

  5. Flowchart On-line processing Microphone Signal Input Filtering Pitch Tracking Post Signal Processing Sampling 11KHz Mid-level Representation Similarity Comparison QueryResults (Ranked SongList) Midi message Extraction Off-line processing Songs Database

  6. Original Wave Input 小雨中的回憶 11025 Hz 8 Bits Mono

  7. Single Frame Overlap Zoom in Frame 512 points/frame 340 points overlap

  8. Pitch Tracking • Range • E2 - C6 • 82 Hz - 1047 Hz ( - ) • Method • Auto-correlation

  9. Auto-correlation without Clipping

  10. Center Clipping Clipping limits are set to r% of the absolute maximum of the auto-correlation data 0 0 0 (a) (b) (c)

  11. Auto-correlation with Clipping

  12. Pitch Contour

  13. Signal Process • Remove violent point & short notes • Down sampling & smoothing • Frequency to semitone • Semitone : A music scale based on A440

  14. Pitch Contour (After Smoothing)

  15. Mid-level Representation

  16. Mid-level Representation without Rest

  17. Similarity Comparison • Goal • Find the most similar Midi file • Challenge • Tempo variance • Dynamic time warping (DTW) • Tune variance • Key transposition

  18. Compare by DTW Wave File DTW Mid File

  19. Dynamic Time Warping (DTW) j window r(j) r(j-1) window i t(i-1) t(i)

  20. DTW (cont.) j dist(i,j) = |t(i)-r(j)| if ( t(i) = Rest && r(j) = Rest ) dist(i,j) = 0; elseif ( t(i) = Rest || r(j) = Rest) dist(i,j) = restWeight; i

  21. Example of DTW

  22. Key Transposition • Mean sift • Binary search in the searching area • O( N) --> O (log N) Mean Searching Area

  23. Example of Key Transposition

  24. Score Function • m : length of match string • n : length of input string • e : DTW distance • A = 0.8 • B = 0.6

  25. Experiment Environment • 290 wave files • Wave length : 5 - 8 sec • Wave format : PCM, 11025Hz, 8bits, Mono • Environment • Celeron 450 with 128Mb RAM under Matlab 5.3 • Database • 493 midi files

  26. Experiment Result (Histogram)

  27. Experiment Result (Pie) Total time: 4589 sec (15.8 sec/per-wave)

  28. Experiment Result (Pie) - With Rest Total time : 7893 sec (27.2 sec/per-wave)

  29. How to Accelerate? • Branch and bound • O(N) -> O(lnN) • Triangle inequality • d(a,b) + d(b,c) ≧ d(a,c) • Hierarchical • 2 phase • 3/32 sec • 2/32 sec

  30. Experiment Result (Pie) - 3/32 sec Total time : 2358 sec (8.9 sec/per-wave)

  31. Experiment Result (Pie) - 2 Phase Total time: 3006 sec (11.2 sec/per-wave)

  32. Error Analysis • Midi error • Singing error • Low pitch • Broken vocalism • Noise

  33. Future Work • Time consuming • Better similarity comparison • Different comparison unit • Hardware acceleration • Better searching algorithm • Steadier pitch tracking algorithm • Noise handle

More Related