1 / 23

Adapted representations of audio signals for music instrument recognition

Adapted representations of audio signals for music instrument recognition. Pierre Leveau Laboratoire d’Acoustique Musicale, Paris - France GET - ENST (Télécom Paris), France. Summary.

kaiser
Download Presentation

Adapted representations of audio signals for music instrument recognition

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Adapted representations of audio signals for music instrument recognition Pierre Leveau Laboratoire d’Acoustique Musicale, Paris - France GET - ENST (Télécom Paris), France

  2. Summary • Master Thesis: Music instrument recognition on solo performances with signal segmentation (transient part / release part) • Ph. D. Thesis: Structured and sparse decompositions: application to audio indexing Pierre Leveau - ENST - LAM

  3. Music instrument recognition on solo performances with signal segmentation (transient part / release part) Music Instrument Recognition • Basic Scheme Training DB (manually indexed) Feature extraction Classification model File to analyze Feature extraction Comparison to the model decision Pierre Leveau - ENST - LAM

  4. Music instrument recognition on solo performances with signal segmentation (transient part / release part) Feature Extraction on frames of fixed size (30 ms) Feature Extraction Analysis Frames Pierre Leveau - ENST - LAM

  5. Music instrument recognition on solo performances with signal segmentation (transient part / release part) Music Note Scheme Ex: strong attack instrument energy time Pierre Leveau - ENST - LAM

  6. Music instrument recognition on solo performances with signal segmentation (transient part / release part) Interest of transients for Music Instrument Recognition piano trumpet flute cello Pierre Leveau - ENST - LAM

  7. Music instrument recognition on solo performances with signal segmentation (transient part / release part) Chosen Method • Signal segmentation into transient part / release part • Approximation: fixed length transients • Need of an automatic onset detection algorithm. • Study of solo performances Pierre Leveau - ENST - LAM

  8. Music instrument recognition on solo performances with signal segmentation (transient part / release part) Onset Detection • Detection function (ex: high frequency content, spectral difference, phase deviation…) • Peak-picking Pierre Leveau - ENST - LAM

  9. Music instrument recognition on solo performances with signal segmentation (transient part / release part) Evaluation of Onset Detection • Necessity of an reference onset database • ROC Curves good detections % false alarms % Pierre Leveau - ENST - LAM

  10. Music instrument recognition on solo performances with signal segmentation (transient part / release part) Sound Onset Labelization spectrogram Reference Onset and Sound Databases Signal plot Pierre Leveau - ENST - LAM Sound listening and labels positioning

  11. Music instrument recognition on solo performances with signal segmentation (transient part / release part) Onset Database Annotation precision depending on the file type Detection function evaluation must take it into account Pierre Leveau - ENST - LAM

  12. Music instrument recognition on solo performances with signal segmentation (transient part / release part) Annotation precision: examples trumpet cello Pierre Leveau - ENST - LAM

  13. Music instrument recognition on solo performances with signal segmentation (transient part / release part) Developed Detection Function Complex Spectral Difference: Delta Complex Spectral Difference: guitar violin Pierre Leveau - ENST - LAM

  14. Music instrument recognition on solo performances with signal segmentation (transient part / release part) Detection Function comparison TROC = Topt Tolerance window TROC = 100 ms Pierre Leveau - ENST - LAM

  15. Music instrument recognition on solo performances with signal segmentation (transient part / release part) Signal segmentation R R T R T R T T Analysis Frames Pierre Leveau - ENST - LAM

  16. Music instrument recognition on solo performances with signal segmentation (transient part / release part) Music Instrument recognition on transients - Results • Music instrument recognition only on transients implies: • big decrease of the learning database size • for a fixed duration of the test signal, less data to take a decision. Results worse than for a recognition on all frames Pierre Leveau - ENST - LAM

  17. Music instrument recognition on solo performances with signal segmentation (transient part / release part) Perspectives • Increase the onset database size for a more robust evaluation • Improve the robustness of the Onset detection algorithm • Merge decisions on transients and steady part, compare to the classical static recognition. • Select features adapted for each part of the notes. Pierre Leveau - ENST - LAM

  18. Ph. D. Thesis Subject: Sparse and structured decompositions: application to audio indexing Under supervision of Gaël Richard (GET - ENST, Paris) and Laurent Daudet (Laboratoire d’Acoustique Musicale, Paris) Pierre Leveau - ENST - LAM

  19. Sparse and structured decompositions: application to audio indexing SparseRepresentations • Classical representations: Orthogonal transform (ex: Fourier Transform, STFT, MDCT, Wavelet Transform…) • Redundant representations: : Redundant dictionnary Sparse representations (only on N terms): Pierre Leveau - ENST - LAM

  20. Sparse and structured decompositions: application to audio indexing Dictionary Example C: MDCT basis (useful to represent tonal parts of signals) W: DWT basis (useful to represent transient parts of signals) Pierre Leveau - ENST - LAM

  21. Sparse and structured decompositions: application to audio indexing Algorithms • Matching Pursuit (and its variants): • Greedy algorithms • Based on an iterative search • Faster algorithm needs a suboptimal search • Molecular Matching Pursuit: • Gives structured, perceptually relevant organizations of the atoms (by grouping significant coefficients) • Faster than standard MP • Fast varying frequencies (ex: vibrato) cannot be efficiently represented Pierre Leveau - ENST - LAM

  22. Sparse and structured decompositions: application to audio indexing Application to music instrument recognition Classical Music Instrument Recognition Feature Extraction Comparison to statistical models Signal features Decision Music Instrument Recognition with sparse decomposition MMP Feature Extraction (which features?) Comparison to statistical models (which models?) Signal features Decision Structured Representation Pierre Leveau - ENST - LAM

  23. To be continued… Thank you for your attention. Pierre Leveau - ENST - LAM

More Related