1 / 22

Tabla Gyan

Tabla Gyan. Realtime tabla recognition and resynthesis Parag Chordia (GTCMT) Alex Rae (GTCMT ). Overview. What : Stroke type. When: Stroke timing. Resynthesis. Transformation: Timbre Rhythm. Video Demo. The Drum. Dayan – treble drum Bayan – bass drum. Tabla Language.

irina
Download Presentation

Tabla Gyan

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Tabla Gyan Realtime tabla recognition and resynthesis Parag Chordia (GTCMT) Alex Rae (GTCMT)

  2. Overview What : Stroke type When: Stroke timing Resynthesis Transformation: Timbre Rhythm

  3. Video Demo

  4. The Drum • Dayan – treble drum • Bayan – bass drum

  5. Tabla Language

  6. Recognition Architecture Input music Onset detection Rhythm Training data Statistical Model SVM Bayesian Neural Net ke te dhe Stroke Label dha tun ge

  7. Build Model: Training Data • Several Datasets • Professional musician • Home recording • Audio recordings manually edited and labeled

  8. Build Model: Target Mapping • Standardize idiosyncratic traditional naming conventions • Map timbrally similar (or identical) strokes to the same category

  9. Build Model: Feature Extraction Spectral Features • MFCCs (24) • Centroid • Variance • Skewness • Kurtosis • Slope • Roll-off Spectral centroid F1 F2 F3 . . . Fn Variance Kurtosis Feature Vector

  10. Build Model: Trained Model • WEKA machine learning package • Support Vector Machine • Models trained on different datasets can be saved for future use

  11. Audio: Input • Live audio is taken from a close-mic’dtabla • Stereo signal provides partial separation of drums

  12. Audio: Segmentation • Onset detection done in Max using bonk~ • More recent parallel project uses spectral flux algorithm in Java • End of stroke marked by next onset (1 sec buffer size) • Onset times stored

  13. Audio: Feature Extraction Spectral centroid F1 F2 F3 . . . Fn Variance Kurtosis Feature Vector

  14. Output: Classification • Feature vector is fed to previously trained model • Single category label returned feature vector SVM label

  15. Output: Symbolic Score • Stroke label combined with timing and amplitude information • Score stored in temporary buffer in Max patch .3204 .9665 2 .3527 .5715 6 .3031 .3648 6 .3325 .9827 6 .2970 .4762 2 .3865 .5928 1 .3496 .6603 8 .7046 .4621 1 .3144 .5024 6 .7152 .2990 6 .3387 .8891 2 .2902 .7342 6 .3868 .9051 7 .3049 .5727 1

  16. Output: Timbre Remapping Stroke labels can be flexibly remapped

  17. Output: Conditional Repetition

  18. Output: User Interface

  19. Dangum

  20. Future Directions • Beat tracking • Modeling specific types of improvisational forms (e.g. qaida, tihai …) • Automate transformations • Improve interface so it can be “played” • Tracking of expressive parameters (e.g. bayan pitch modulation)

  21. Conclusions • Shown a realtime tabla interaction system • Implemented as Max java external using machine learning to identify strokes • Supports flexible transformations • Foundation for more general improvisation system

More Related