1 / 18

Onset Detection, Tempo Estimation, and Beat Tracking

Understand the importance of onset detection in music analysis, its function, and the steps involved in generating onset strength curves. Discover ways to utilize onset data for tempo estimation and beat tracking in audio processing. Dive into performance indices for evaluating beat tracking accuracy.

jlola
Download Presentation

Onset Detection, Tempo Estimation, and Beat Tracking

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Onset Detection, Tempo Estimation, and Beat Tracking J.-S Roger Jang (張智星) http://mirlab.org/jang MIR Lab, CSIE Dept. National Taiwan University

  2. What Are Onsets? • Music instrument event is usually divided into 4 stages of ADSR (attack, decay, sustain, release) based on its energy profile. • Onset is the time when the slope is the highest, during the attack time. Quiz!

  3. Characteristics of Onset Detection • Music types • Monophonic (single sound source)  Easier • Polyphonic (multiple sound sources)  Harder • Instrument types • Percussive instruments  hard onsets which are easier to detect • String instruments  Soft onsets which are harder to detect

  4. Why Onset Detection is Important? • It is a fundamental step in audio music analysis • Music transcription (from wave to midi) • Music editing (Song segmentation) • Tempo estimation and beat tracking • Musical fingerprinting (the onset trace can serve as a robust feature for fingerprinting)

  5. Onset Detection Function • ODF (onset detection function) creates a curve of onset strength, aka • Onset strength curve (OSC) • Novelty curve • Most ODFs are based on time-frequency representation • Magnitude of STFT  spectrogram • Phase of STFT • Mel-band of STFT • Constant-Q transform Short-time Fourier transform

  6. Spectrogram • wave2specGram.m X(n, k)

  7. ODF: Spectral Flux • Concept • Sum the positive change in each frequency bin Quiz!

  8. Flowchart of OSC • Steps of OSC • Spectrogram • Mel-band spectrogram (optional) • Spectral flux • Smoothed OSC via Gaussian smoothing • Trend of OSC via Gaussian smoothing • Trend-subtracted OSC • Check out wave2osc.m to see these steps.

  9. Mel-freq Spectrogram • 40 filters in Mel-freq filter bank • Spectrograms spec1 Linear freq Mel freq spec2 Matrix M spec2=M*spec1 melBinPlot.m

  10. Mel-freq Representation • About mel-freq spectrogram • Advantage: More correlated to human perception(just like MFCC in speech recognition) • The degree of effectiveness is yet to be verified

  11. Spectral Flux • spec2oscPlot.m magSpec rawOsc rawOsc=mean(max(0, diff(magSpec, 1, 2))); Order 1 Dim 2 Order 1

  12. Gaussian Smoothing • s is small  Smoothed OSC • s is large  Trend of OSC

  13. Example of OSC • wave2osc.m

  14. What Can You Do With OSC... • OSC  onsets • Pick peaks to have onsets • OSC  tempo (BPM, beats per minute) • Apply ACF (or other PDF) to find the BPM • OSC  beat tracking • Pick equal-spaced peaks to have beat positions Quiz!

  15. Example of Beat Tracking • beatTrack.m Identified beat pos. Max BPM=250 Min BPM=50 Tempo estimation via ACF 8 candidate sets for beat positions

  16. Demo of Beat Tracking • Demo link • http://mirlab.org/demo/beatTracking • Examples • 陪我看日出 • 分享

  17. Performance Indices in Information Retrieval Figure source: http://bryannotes.blogspot.tw/2015/06/precisionrecall.html • References • F1-score • Precision & recall • Information retrieval Desired Retrieved Quiz!

  18. Performance Indices ofBeat Tracking • Many performance indices of BT • Check out audio beat tracking task of MIREX • Mostly adopted ones • Precision, recall, f-measure, accuracy • Try simSequence.m in SAP toolbox Precision = tp/(tp+fp)=3/(3+4) = 0.428571 Recall = tp/(tp+fn)=3/(3+2) = 0.600000 F-measure = tp/(tp+(fn+fp)/2)=3/(3+(2+4)/2) = 0.500000 Accuracy = tp/(tp+fn+fp)=3/(3+2+4) = 0.333333 Quiz!

More Related