1 / 48

Adaptive Sinusoidal Modeling of Percussive Musical Instrument Sounds

Adaptive Sinusoidal Modeling of Percussive Musical Instrument Sounds. Marcelo Caetano, George Kafentzis , Athanasios Mouchtaris , Yannis Stylianou FORTH-ICS, Greece – CSD University of Crete , Greece – Orange Labs , France. Overview. Introduction and Motivation

ahava
Download Presentation

Adaptive Sinusoidal Modeling of Percussive Musical Instrument Sounds

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Adaptive SinusoidalModeling of Percussive Musical Instrument Sounds Marcelo Caetano, George Kafentzis, AthanasiosMouchtaris, Yannis Stylianou FORTH-ICS, Greece– CSD University of Crete, Greece – Orange Labs, France

  2. Overview Introduction and Motivation Adaptive Sinusoidal Modeling Percussive Musical Instrument Sounds Evaluation Conclusions and Future Work Marcelo Caetano, George Kafentzis, Athanasios Mouchtaris, and Yannis Stylinou

  3. Introduction and Motivation Marcelo Caetano, George Kafentzis, Athanasios Mouchtaris, and Yannis Stylinou

  4. Introduction • Percussive musical instrument sounds • Sharp onset • Highly nonstationary attack transients • Stationary sinusoids • Slowly-varying inside analysis window • Smear sharp onsets (pre-echo) • Fail to capture attack transients Marcelo Caetano, George Kafentzis, Athanasios Mouchtaris, and Yannis Stylinou

  5. Motivation • Extended adaptive Quasi-Harmonic Model (eaQHM) • Adaptation of sinusoids inside the analysis window • Nonstationary sinusoids capture • Quasi-stationary oscillations (partials) • Transients (amplitude and frequency modulations) Marcelo Caetano, George Kafentzis, Athanasios Mouchtaris, and Yannis Stylinou

  6. Adaptive Sinusoidal Modeling Marcelo Caetano, George Kafentzis, Athanasios Mouchtaris, and Yannis Stylinou

  7. Sinusoidal Modeling Analysis: sound being modeled Synthesis: sinusoidal model Residual: modeling error Marcelo Caetano, George Kafentzis, Athanasios Mouchtaris, and Yannis Stylinou

  8. Sinusoidal Modeling • Signal processing steps Marcelo Caetano, George Kafentzis, Athanasios Mouchtaris, and Yannis Stylinou

  9. Sinusoidal Modeling Peak picking Marcelo Caetano, George Kafentzis, Athanasios Mouchtaris, and Yannis Stylinou

  10. Sinusoidal Modeling Partial tracking Marcelo Caetano, George Kafentzis, Athanasios Mouchtaris, and Yannis Stylinou

  11. Sinusoidal Modeling Partial tracking Marcelo Caetano, George Kafentzis, Athanasios Mouchtaris, and Yannis Stylinou

  12. Sinusoidal Modeling • Resynthesis • Overlap-Add (OLA) • Additive synthesis • OLA: Stationary frequencies and amplitudes • Additive synthesis: polynomial interpolation Marcelo Caetano, George Kafentzis, Athanasios Mouchtaris, and Yannis Stylinou

  13. Adaptive Sinusoidal Modeling • Signal processing steps Marcelo Caetano, George Kafentzis, Athanasios Mouchtaris, and Yannis Stylinou

  14. Adaptive Sinusoidal Modeling • Extended Adaptive Quasi-Harmonic Model (eaQHM) • Quasi-Harmonic Model • Least squares • Frequency correction • Adaptation • Parameter interpolation • Nonstationary basis functions Marcelo Caetano, George Kafentzis, Athanasios Mouchtaris, and Yannis Stylinou

  15. Quasi-Harmonic Model (QHM) Least squares Fit parameters of template sound by least squares Template is harmonically related sinusoids Parameter estimation includes frequency correction Marcelo Caetano, George Kafentzis, Athanasios Mouchtaris, and Yannis Stylinou

  16. Quasi-Harmonic Model (QHM) Analysis equation Marcelo Caetano, George Kafentzis, Athanasios Mouchtaris, and Yannis Stylinou

  17. Quasi-Harmonic Model (QHM) Frequency correction Marcelo Caetano, George Kafentzis, Athanasios Mouchtaris, and Yannis Stylinou

  18. Quasi-Harmonic Model (QHM) Synthesis equation Marcelo Caetano, George Kafentzis, Athanasios Mouchtaris, and Yannis Stylinou

  19. Quasi-Harmonic Model (QHM) Frequency correction Marcelo Caetano, George Kafentzis, Athanasios Mouchtaris, and Yannis Stylinou

  20. Quasi-Harmonic Model (QHM) Frequency correction Marcelo Caetano, George Kafentzis, Athanasios Mouchtaris, and Yannis Stylinou

  21. Quasi Harmonic Model (QHM) Iteratively corrects amplitude and frequency estimations Stationary inside the analysis and synthesis window Marcelo Caetano, George Kafentzis, Athanasios Mouchtaris, and Yannis Stylinou

  22. Extended Adaptive Quasi Harmonic Model (eaQHM) • Adapts to temporal variations inside the analysis window • Time varying frequencies and amplitudes • Captures amplitudes and frequency modulations Marcelo Caetano, George Kafentzis, Athanasios Mouchtaris, and Yannis Stylinou

  23. Extended Adaptive Quasi Harmonic Model (eaQHM) • Analysis equation Marcelo Caetano, George Kafentzis, Athanasios Mouchtaris, and Yannis Stylinou

  24. Extended Adaptive Quasi Harmonic Model (eaQHM) • Synthesis equation Marcelo Caetano, George Kafentzis, Athanasios Mouchtaris, and Yannis Stylinou

  25. Frequency Adaptation Marcelo Caetano, George Kafentzis, Athanasios Mouchtaris, and Yannis Stylinou

  26. Extended Adaptive QHM Percussive musical instrument sound Marcelo Caetano, George Kafentzis, Athanasios Mouchtaris, and Yannis Stylinou

  27. Extended Adaptive QHM Initialization: user input values Marcelo Caetano, George Kafentzis, Athanasios Mouchtaris, and Yannis Stylinou

  28. Extended Adaptive QHM Estimation of amplitudes Marcelo Caetano, George Kafentzis, Athanasios Mouchtaris, and Yannis Stylinou

  29. Extended Adaptive QHM Correction of frequency estimations (QHM) Marcelo Caetano, George Kafentzis, Athanasios Mouchtaris, and Yannis Stylinou

  30. Extended Adaptive QHM Projection onto nonstationary basis functions Marcelo Caetano, George Kafentzis, Athanasios Mouchtaris, and Yannis Stylinou

  31. Percussive Musical Instrument Sounds Marcelo Caetano, George Kafentzis, Athanasios Mouchtaris, and Yannis Stylinou

  32. Percussive Musical Instruments • eaQHM renders a representation • Perceptually closer than SM • Same model (synthesis) complexity • Adaptation • Sharp onsets (no pre-echo) • Amplitude and frequency modulations • Transient modeling Marcelo Caetano, George Kafentzis, Athanasios Mouchtaris, and Yannis Stylinou

  33. Comparison • Model complexity Marcelo Caetano, George Kafentzis, Athanasios Mouchtaris, and Yannis Stylinou

  34. Percussive Onsets Marcelo Caetano, George Kafentzis, Athanasios Mouchtaris, and Yannis Stylinou

  35. Evaluation Marcelo Caetano, George Kafentzis, Athanasios Mouchtaris, and Yannis Stylinou

  36. Aim of Evaluation Compare representation between SM and eaQHM Objective evaluation: SRER Subjective evaluation: perceptual similarity Marcelo Caetano, George Kafentzis, Athanasios Mouchtaris, and Yannis Stylinou

  37. Parameters Pitch class C: from C3 (f0≈131 Hz) to C7 (f0≈2093 Hz) Dynamics: mezzo forte and forte Duration is less than 2s Window size (3*T0), FFT (2048), Fs (16k), hop (2ms) Marcelo Caetano, George Kafentzis, Athanasios Mouchtaris, and Yannis Stylinou

  38. Musical Instrument Sounds Marcelo Caetano, George Kafentzis, Athanasios Mouchtaris, and Yannis Stylinou

  39. Objective Evaluation Signal to Reconstruction Error Ratio (SRER) Marcelo Caetano, George Kafentzis, Athanasios Mouchtaris, and Yannis Stylinou

  40. Objective Evaluation Local SRER: window before the onset Global SRER: whole duration Marcelo Caetano, George Kafentzis, Athanasios Mouchtaris, and Yannis Stylinou

  41. Objective Evaluation • Results Marcelo Caetano, George Kafentzis, Athanasios Mouchtaris, and Yannis Stylinou

  42. Listening Test Which model is perceptually closer to original sounds? Forced preference 13 plucked strings and 6 percussion instruments 51 listeners Marcelo Caetano, George Kafentzis, Athanasios Mouchtaris, and Yannis Stylinou

  43. Listening Test Marcelo Caetano, George Kafentzis, Athanasios Mouchtaris, and Yannis Stylinou

  44. Results of the Listening Test Marcelo Caetano, George Kafentzis, Athanasios Mouchtaris, and Yannis Stylinou

  45. Results of the Listening Test Participants refer to instruments by name Listeners used attack to tell difference Sharpness of onset was mentioned as difference Marcelo Caetano, George Kafentzis, Athanasios Mouchtaris, and Yannis Stylinou

  46. Conclusions and Future Work Marcelo Caetano, George Kafentzis, Athanasios Mouchtaris, and Yannis Stylinou

  47. Conclusions & Perspectives eaQHM represents well sharp onsets and attack transients QHM provides good initial estimation of parameters Adaptation corresponds to projection of signal onto nonstationary functions (AM/FM modulation) Adaptation represents variations inside the analysis window Marcelo Caetano, George Kafentzis, Athanasios Mouchtaris, and Yannis Stylinou

  48. Questions? • http://www.csd.uoc.gr/~kafentz/listest/ Marcelo Caetano, George Kafentzis, Athanasios Mouchtaris, and Yannis Stylinou

More Related