610 likes | 764 Views
Korean Brain Neuroinformatics Research Program and Artificial Auditory Systems. Soo-Young Lee Brain Science Research Center Department of BioSystems Korea Advanced Institute of Science and Technology http:// bsrc.kaist.ac.kr. Korean Brain Neuroinformatics Research Program
E N D
KoreanBrain Neuroinformatics Research Programand Artificial Auditory Systems Soo-Young Lee Brain Science Research Center Department of BioSystems Korea Advanced Institute of Science and Technology http://bsrc.kaist.ac.kr
Korean Brain Neuroinformatics Research Program Artificial Auditory Systems- Auditory Model- Binaural Processing- Selective Attention Contents Department of BioSystems andBrain Science Research Center, KAIST
Brain: The Final Frontier! Intelligence to Machine! Freedom to Mankind! From Biology to Hardware! Department of BioSystems andBrain Science Research Center, KAIST
Feasibility study in early 1997 Announced by Ministry of Science and Technology on September 1997 Inauguration of Brain Science Research Center at Korea Advanced Institute of Science and Technology on December 1997 Brain Research Promotion Law on May 1998 Started on November 1998 for 10 years. Braintech’21 Department of BioSystems andBrain Science Research Center, KAIST
뇌기능의 응용 (신경회로망) Scope of Braintech’21 Brain Science & Engineering Research Program Neuroinformatics Application (Engineering) Macroscopic (Function) Under- standing (Science) Mimicking (Bioelectronics) Creating (Artificial Life) Neurobiology Protecting (Medical) Microscopic (molecules) Biomedical Brain Research Program Department of BioSystems andBrain Science Research Center, KAIST
Phase 1 (1998-2000)- Build-up Foundation- 2 programs (Brain Science & Engineering, Brain Biomedical) Phase 2 (2001-2003)- Extend the Foundation- 3 programs (Brain Neurobiology, Brain Neuroinformatics, Brain Biomedical) Phase 3 (2004-2007)- Apply to Real-World Problems 3 Phases for 10 Years Department of BioSystems andBrain Science Research Center, KAIST
Phase 1 VLSI Self-adapt Viosn & Auditory Cog. Sci. Modeling Phase 2 Auditory Vision Cog/Infer. Behavior fMRI Brain Eng. Cog. NS Phase 3 fMRI etc. Brain-like Systems (Artificial Brain) Phase 2: Neuroinformatics Program Department of BioSystems andBrain Science Research Center, KAIST
Brain Neuroinformatics Brain-like Functional Systems Neuromorphic Chips Mathematical Model Analysis Software Neuroscience Data Functional Groups Measurement Technology Measurement Group Department of BioSystems andBrain Science Research Center, KAIST
Selection and Concentration Integration- Database- Neuromorphic Chips- Brain-like Systems Multidisciplinary Team- Neuroscientists- Mathematicians- Engineers Korean Approaches Department of BioSystems andBrain Science Research Center, KAIST
Pattern Recognition Cognition & Inference Vision Inference Brain-like System SelectiveAttention Learning/ Memory/ Language Behavior Recogn./ Tracking Feature Extraction Sen- sor Individual/ Group Behavior Perception/ Planning Body Control Feature Extraction Recog./ Understand Sen- sor Fusion Selective Attention Smell Olf. Touch Auditory Speech Recognition Brain Functions Department of BioSystems andBrain Science Research Center, KAIST
Noise-Robust Speech Processing Cocktail party problem Artificial Auditory System Department of BioSystems andBrain Science Research Center, KAIST
Auditory model • Binaural processing • Selective attention • Speech processing chip Artificial Auditory System • Based on Human Cognitive Mechanism • Develop mathematical model and auditory chip • Develop Continuous speech recognition system 95% recognition in 10dB SNR Department of BioSystems andBrain Science Research Center, KAIST
Auditory Central Nerve System Department of BioSystems andBrain Science Research Center, KAIST
Object Path Attention Spatial Path Cochlea Nonlinear Features Cross Correlation Bottom-Up Attention Time Adaptation Sound Localization TD/BU Attention Time-Freq. Masking Speech Enhancement Speaker Separation Complex Sound Top-Down Attention Aud. Cortex Higher-Order Functions Parser Speech Rec. System Research Scopes Auditory Pathway Department of BioSystems andBrain Science Research Center, KAIST
Human Auditory System Department of BioSystems andBrain Science Research Center, KAIST
Auditory Model - bandpass filtering (logarithmic scale in frequency) at basilar membrane - Nonlinear processing at inner hair cell (short-term adaptation, synchronization, etc.) Department of BioSystems andBrain Science Research Center, KAIST
Auditory Model BM OHC IHC Synapse / Auditory Nerve Automatic Gain Control of Outer Hair Cell Department of BioSystems andBrain Science Research Center, KAIST
Auditory model S Cochlear Filter 1 Auditory Nerve Fibers Gain & Q Control Frequency Information Cochlear Filter i Zero-Crossing Detector Time-Freq. Histogram x(t) y(t,f) Amplitude Integrator Nonlinear Transform Intensity Information Cochlear Filter M Auditory Nerve Fibers Basilar Membrane Frequency Selectivity Mechanical-to-neural Transduction Department of BioSystems andBrain Science Research Center, KAIST
100.0 90.0 80.0 Recognition Rates 70.0 60.0 50.0 40.0 30.0 20.0 10.0 0.0 Clean 25 20 15 10 5 SNR (dB) Recognition Results with Noise White Gaussian Noise Car Noise 10 5 0 -5 -10 SNR (dB) LPCC MFCC SBCOR PLP EIHC AudModel Department of BioSystems andBrain Science Research Center, KAIST
100.0 90.0 80.0 70.0 60.0 50.0 40.0 30.0 20.0 10.0 0.0 Recognition Results with Noise Factory Noise Military Operations Room Noise Recognition Rates Clean 25 20 15 10 5 Clean 25 20 15 10 5 SNR (dB) SNR (dB) LPCC MFCC SBCOR PLP EIHC AudModel Department of BioSystems andBrain Science Research Center, KAIST
ICA-based Speech Features : Time Department of BioSystems andBrain Science Research Center, KAIST
ICA-based Speech Features : Freq. Department of BioSystems andBrain Science Research Center, KAIST
Time-Frequency Masking • Lateral Inhibition Department of BioSystems andBrain Science Research Center, KAIST
Masking using Lateral Inhibition Department of BioSystems andBrain Science Research Center, KAIST
Isolated Word Recognition Rates Department of BioSystems andBrain Science Research Center, KAIST
Left Ear Z-1 Z-1 Z-1 Right Ear Z-1 Z-1 Z-1 Filter Banks (f) Interaural Time Difference (t) Binaural Hearing • Summation for all BP filters • Winner-Take-All on ITD Department of BioSystems andBrain Science Research Center, KAIST
Audio Noise Mixed Signal Speech Signal Unknown Noise Mic 1 Command ANC + BSS Mixed Signal Audio Noise Mic 2 Noise Reference Signal Noise Car Audio Simultaneous ANC and BSS Department of BioSystems andBrain Science Research Center, KAIST
Cochlea Cochlear Nucleus Auditory Cortex Higher-Order Brain Function (Recognition) Superior Olive Superior Olive Auditory Cortex Cochlear Nucleus Cochlea Auditory Pathway • Improvement of Cochlea model • Binaural hearing model • Top-Down attention • Speech-recognition chip Department of BioSystems andBrain Science Research Center, KAIST
Unmixed Signal u1(t)= s1(t) Mixed Signal x1(t) Speech s1(t)=s(t) y1(t) + + A21 W21 W12 A12 Noise s2(t)=n(t) y2(t) Mixed Signal x2(t) Unmixed Signal u2(t)= s2(t) Blind Signal Separation Department of BioSystems andBrain Science Research Center, KAIST
Speech+ Convolved Noise x(t)=s(t)+n0(t) Enhanced Speech u(t) Speech s(t) y(t) + -Wl Wl Noise Nl(t) Noise nl(t) + Active Noise Canceling Department of BioSystems andBrain Science Research Center, KAIST
LMS: second-order statistics only ICA: statistical independence LMS algorithm SNR (dB) ICA-based algorithm Car Speech Music 45 40 35 30 25 20 15 10 5 0 Independent Component Analysis Department of BioSystems andBrain Science Research Center, KAIST
4 input buffer, 6 weight vector 12 KHz, 12 bit sampling 512 time delays Memory Requirement Input buffer : 2 x [ 2 x 512 x 12 ] : 12 MHz Weight vector : 2 x [ 3 x 512 x 30 ] : 36 MHz Function 4 ANC 2 BSS + 4 ANC 3 BSS Gate count 20,000 NESS processor Department of BioSystems andBrain Science Research Center, KAIST
Artificial mixing using room response filter Simulation (1) Department of BioSystems andBrain Science Research Center, KAIST
Simulation Tasks Simulation (2) Department of BioSystems andBrain Science Research Center, KAIST
TASK II : 4 ch ANC M1 O1 S1 N1 N2 N3 N4 SNR Department of BioSystems andBrain Science Research Center, KAIST
Simulation Result Department of BioSystems andBrain Science Research Center, KAIST
Known Signal : 4 MUSICs Unknown Signal : CAR Demo System Department of BioSystems andBrain Science Research Center, KAIST
4 audio music 1 male voice 1 female voice MIC1 MIC2 OUT1 OUT2 2-BSS + 4-ANC Channels Department of BioSystems andBrain Science Research Center, KAIST
Training TI DIGIT training set : men only 794 utterance Clean DB only ESR700 Development Tool Network size : automatically constructed by ESR700 tool Input : 256 Hidden : 20 Output : 10 Recognition Test (1) Department of BioSystems andBrain Science Research Center, KAIST
Test TI DIGIT test set : men only 1267 utterance Artificial mixing by room response 2 BSS + 4 ANC BSS : Test DB + car noise [NOISEX CD] ANC : 4 MUSICs Recognition Test (2) Department of BioSystems andBrain Science Research Center, KAIST
Case Example SNR : - 10 dB Recognition Test (3) MIC1 MIC2 OUT1 OUT2 Department of BioSystems andBrain Science Research Center, KAIST
without ANC/BSS with ANC/BSS Recognition Rate(%) 100 80 60 40 20 0 Clean 25 20 15 10 5 0 -5 -10 SNR(dB) Recognition Rates Department of BioSystems andBrain Science Research Center, KAIST
Recognition Test (4) Department of BioSystems andBrain Science Research Center, KAIST
ICA-based ANC with Filter Bank Department of BioSystems andBrain Science Research Center, KAIST
Filter bank design Eight-channel oversampled filter bank Prototype filter with 192 taps Learning curves Signal and noise: speech Mixing filter A measured filter in a normal office room with 1000 taps Performance limitation of the freq. domain approach The filter bank approach Much faster convergence speed than the time domain approach ANC Results withFilter Bank Department of BioSystems andBrain Science Research Center, KAIST
Internal Cue External Cue Attended Output Classifier Output Bottom-Up Recognition Top-Down Attention Attended Features Bottom-Up Attention Input Features Brain Environment Input Stimulus Selective Attention • Bottom-Up: - Change - Masking - ICA • Top-Down: - MLP - HMM Department of BioSystems andBrain Science Research Center, KAIST
Confusing/Occluded patterns Overlapped patterns Top-Down Selective Attention Department of BioSystems andBrain Science Research Center, KAIST
1958: Broadbent, “Early filter” theoryThe brain temporarily retained information about all stimuli but that the information faded, unless attention had been turned quickly to a particular memory trace. 1960: Treisman”the filter merely attenuates” 1971: Broadbent“selection on the basis of semantic properties” 1988: Fukushima: “Neocognitron” modelselective attention and switching for the recognition of superimposed binary patterns Background Department of BioSystems andBrain Science Research Center, KAIST
S. Treue & J.C.M. Trujilo, Nature 1999, pp. 575-579. Feature Similarity Gain Model • The regulation of the gain of a sensory neuron reflects the similarity of the features of the currently behaviourally relevant target and the sensory selectivity of the neuron. • The relevant target features include the spatial location, direction of motion and presumably others. Department of BioSystems andBrain Science Research Center, KAIST
^ ^ ^ ^ h1 h2 h3 hK x1 x2 x3 xN a1 a2 a3 aN Wkn y1 y2 y3 yM Vmk x1 x2 x3 xN ^ x a x W h V y Proposed Neural Model • Early filtering: “feature-based”attention • Attention cue at test phase: external cue to a class • (focus at recognition only, not training) • Top-Down expectation: Gradient-based EBP • algorithm using trained knowledge in an MLP Department of BioSystems andBrain Science Research Center, KAIST