1 / 21

SEMA

SEMA. SEmantic description of Musical Audio with applications in audio-mining, interactive multimedia, and brain research. Benoit Catteau. Prof. LEMAN Marc Faculty of Letters and Philosophy. Prof. MARTENS Jean-Pierre Faculty of Applied Sciences. SEMA. Introduction SEMA The mission

shada
Download Presentation

SEMA

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. SEMA SEmantic description of Musical Audio with applications in audio-mining, interactive multimedia, and brain research. Benoit Catteau Prof. LEMAN Marc Faculty of Letters and Philosophy Prof. MARTENS Jean-Pierre Faculty of Applied Sciences

  2. SEMA Introduction SEMA The mission My research topics Research resources The auditory model MAMI project Conclusion

  3. SEMA Introduction SEMA The mission My research topics Research resources The auditory model MAMI project Conclusion

  4. SEMAIntroduction • Music is important in a human’s life • People that have a big music collection need efficient search strategies • Searching for a song now restricted to • Title • Artist • Genre

  5. … … SEMAIntroduction • See music collection as a databaseOwner queries the database • New search algorithms need • Input from the user (textual, auditive imitation) • Automatic annotation of the database Annotations Audio Audio

  6. SEMA • Introduction • SEMA • The mission • My research topics • Research resources • The auditory model • MAMI project • Conclusion

  7. SEMAThe mission Engineers audio structures Syntactic analysis Statistical models Subjective appreciation Musical experts Syntactic Annotation Semantic Annotation

  8. SEMA • Introduction • SEMA • The mission • My research topics • Research resources • The auditory model • MAMI project • Conclusion

  9. SEMAMy topics: structures • Extraction of melody lines (bass line, dominant melody, …) and tonality (key) • Reveal the musical structure: chords, harmonic progressions, rythmic patterns, … • Identification of voices (vocal parts, musical instruments)

  10. Class I Supervised Learning Algorithm Syntactic Elements Tracks Class … … … SEMAMy topics: semantics • Need for manually annotated databases for training and evaluation of algorithms

  11. SEMA Introduction SEMA The mission My research topics Research resources The auditory model MAMI project Conclusion

  12. BPF audio Cochlear preprocessing Neural firing patterns SEMAresearch resources: auditory model • Prepocessing the input: give the machine an artificial ear • Time/frequency analysis in peripheral ear

  13. SEMAresearch resources: auditory model • Further analysis of neural firing patterns syntactic elements BPF audio Cochlear preprocessing Central auditory processing (???) Ear model Processing module

  14. SEMA Introduction SEMA General The mission Research resources The auditory model MAMI project Conclusion

  15. SEMAresearch resources: MAMI project • MAMI = “Musical Audio MIning” • Goal: querying the database by humming, whistling, singing, playing the melody (monophonic!) • Method: melody transcription of query, comparing this transcription with MIDI scores of target melodies

  16. SEMAresearch resources: MAMI project • Needed central auditory processing pitch pattern BPF audio Cochlear preprocessing Central auditory processing (???) Ear model Processing module

  17. SEMAresearch resources: MAMI project • Phase 1 : Time-based algorithm (AMPEX) • developed mainly for speech • limited frequency range (< 400 Hz) • Phase 2 : Frequency-based algorithm (SHS) • analysis of spectral energy distributions • search for harmonics and harmonic ratios • Phase 3 : Combine the two algorithms

  18. audio BPF AMPEX Cochlear preprocessing Frequency Splitter Combination pitch Ear model Filtering/Sampling SHS SEMAresearch resources: MAMI project

  19. SEMAresearch resources: MAMI project • Results of the pitch detection for singing without lyrics

  20. SEMA Introduction SEMA General The mission Research resources The auditory model MAMI project Conclusion

  21. SEMAconclusion • We will extend the work of MAMI: • Eliminate the use of MIDI in QBH • Learn how to process polyphonic music • Develop semantic classification algorithms that can handle a wide class of musical pieces • Use these semantic classifications in advanced search methods

More Related