1 / 56

Computer Music Researches in NTUEE

Computer Music Researches in NTUEE. Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication Engineering National Taiwan University. Research Areas of Shyh-Kang Jeng. Computer Music (JCMG) Agent Systems and Multimedia (JAMG)

Faraday
Download Presentation

Computer Music Researches in NTUEE

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Computer Music Researchesin NTUEE Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication Engineering National Taiwan University

  2. Research Areas of Shyh-Kang Jeng • Computer Music (JCMG) • Agent Systems and Multimedia (JAMG) • Electromagnetic Theory and Applications (JEMG)

  3. Computer Music

  4. Outline • Sound Output and Separation • Sound Synthesis • Music Information Retrieval • Music Perception

  5. Advantages of Interactive 3-D Sound System • Sound Localization • Spatial Impression • Fast Computer & DSP • It‘s very cheap!!!

  6. What is HRTF ? • HRTF(Head Related Transfer Functions) 在自由聲場下,聲音傳遞至人耳耳道的轉移函數,為達成聽覺虛擬實境最重要的關鍵. • 3D Sound=HRTF*(Mono Sound Source) • Getting HRTF -- By Measurement or Simulation

  7. 3D Tracker

  8. 仰角 0 ° 清大的 HRTF 麻省理工 HRTF 方位角 0 °和 可以辨認 ：一人 可以辨認 ：四人 180 °時的比較 不能辨認 ：四人 不能辨認 ：一人 方位角 0 °時 可以辨認 ：四人 可以辨認 ：四人 不能辨認 ：一人 不能辨認 ：一人 方位角 90 °時 均可辨認 均可辨認 方位角 180 °時 可以辨認 ：四人 可以辨認 ：四人 不能辨認 ：一人 不能辨認 ：一人 方位角 270 °時 均可辨認 均可辨認 HRTF的聽覺比較

  9. Crosstalk的消除

  10. 10 unit : m 8 6 z - a x i s 4 2 0 10 8 6 4 2 0 -2 -4 10 8 6 -6 4 2 0 -8 -2 -4 -6 -10 -8 y - a x i s -10 x - a x i s Virtual Concert Hall

  11. Shooting and Bouncing of a Ray Tube

  12. BRIR in Concert Hall Model

  13. VIESS Architecture irc2 simulation Room environment HRTFL filter Sound source (*.wav file) RTF filter HRTFR filter

  14. Weak Strong VIESS Demonstration Room environment irc2 simulation Sound Source (*.wav file) RTF filter Original

  15. 歌曲人聲消除 • 原音 軟體消音 自製消音

  16. Outline • Sound Output and Separation • Sound Synthesis • Music Information Retrieval • Music Perception

  17. Physical Modeling Synthesis • Synthesize sound of a musical instrument through computation based on its sound-generation model • The artificial instrument can have the same parameters as the real instrument • Expressivity of control is unlimited

  18. Ideal Vibrating String

  19. Ideal, Lossless Digital Waveguide

  20. 吹管樂器樂音之物理模式合成 • Titanic

  21. 硬體設計圖

  22. 實際燒錄

  23. 實際燒錄(cont’d)

  24. Outline • Sound Output and Separation • Sound Synthesis • Music Information Retrieval • Music Perception

  25. Music Retrieval Service Key-Melody Notes Key-MelodySearching Unit MIDI Files The Best Match Desired Music MIDI Sequencer

  26. MPEG-7

  27. Example of Melody Contour DS(1st Measure of “Moon River”) <MelodyContour> <Contour> 67 4 –1 –1 –1 –1 1</Contour> <Meter> <Numerator>3</Numerator> <Denominator>4<Denominator> <Meter> <Beat> 1 4 5 7 8 9 10</Beat> <MelodyContour>

  28. Edit Distance Computation • p={0 0 -4 2 0 0 -3} q={0 0 0 2 2 -2}

  29. Music Query System Prototype Put the note into the piano bar Choose what kinds of notes Message Board

  30. Music Query System Prototype

  31. Outline • Sound Output and Separation • Sound Synthesis • Music Information Retrieval • Music Perception

  32. Rhythm Recognition Pitch Recognition Timbre Recognition Chord Recognition Time-frequency analysis Pattern recognition Integrated Recognition Model STFT Constant-Q Wavelet

  33. Spectral Shapes of Flutes and Trumpets

  34. Slope of the Attack Violin Flute AcousticPiano Trumpet String Ensembles AcousticGuitar

  35. Results of Timbre Recognition Unit in Monophonic Environments Total Recognition rate = 80.56%

  36. Results of Timbre Recognition Unit in Polyphonic Environments Total Recognition rate = 49.07%

  37. Cochlea Cerebral Cortex Perception Neural Network Wavelet Transform Recognized Chords Models of Chord Classification

  38. Results of Chord Recognition Unit

  39. Transcription Example Original Transcription Result

  40. 2.5 2 Magnitude 1.5 1 0.5 0 0 200 400 600 800 1000 1200 1400 Frequency(Hz) 5 4.5 4 3.5 3 Magnitude 2.5 2 1.5 1 0.5 0 0 200 400 600 800 1000 1200 1400 Frequency (Hz) The Octave Problem

  41. Digital audio signal Constant-Q transform Spectral peaks 64 Spectral peaks for every 64 samples Grouping Series G1 Series i Series C8 sustained or silent Onset/offset detection off on Series i Octave detector for series i Pitch(es) clear Pitch memory for series i Pitch Detection System

  42. Octave Detection Example Original Test Results

  43. 鋼琴和絃 wav to midi 轉換

  44. Beat Tracking System • Off-line method • Input signal sampling rate : 8 kHz • Platform : MATLAB

  45. Variance Diagram

  46. Piano Music

  47. A Pop Song

  48. Polyphonic Music

  49. 爵士鼓節拍校正

  50. Conductor Program

More Related