ECE 598: The Speech Chain

1. ECE 598: The Speech Chain Lecture 10: Auditory Physiology

2. Today • Outer Ear: Sound Localization • Middle Ear: Impedance Matching • Basilar Membrane: Frequency Analysis • Mechanical Principles • Frequency Response of Auditory Filters • Nonlinearity of Basilar Membrane Response • Mechano-Electric Transduction • Inner and Outer Hair Cells • Neuro-transmitter Uptake Models • Neural Activation Thresholds

3. Auditory Anatomy: Overview

4. Localization of Sound: Inter-Aural Time Delay (ITD) Wavefronts (lines of constant pressure) rcosq q r(p/2-q) r Diffusion of sound around the head Wave traveling direction ITD = (r/c)(p/2-q+cosq)

5. Localization of Sound: Inter-Aural Amplitude Difference • f < c/4r ~ 1kHz: head << l=c/f; sound diffuses around head • f > c/2r ~ 2kHz: head > l, so sound is blocked by the head High frequency, Short wavelength (shown: wave “troughs”, i.e., pressure minima) Low frequency, Long wavelength (shown: wave “troughs”, i.e., pressure minima) Sound diffuses around the head; no shadow Head shadow: Sound unable to diffuse around large obstacle

6. Localization of Sound: Echoes from the Pinna and Shoulders Direct Sound Pinna Echo Shoulder Echo

7. Localization of Sound, Summary: Head-Related Transfer Function • Source: s(t) = cos(wt) • Received at near ear: xR(t) = AR(w) cos(wt+fR(w)) • Received at far ear: xL(t) = AL(w) cos(wt+fL(w)-wtITD) • Near ear frequency response: HR(w) = AR(w)ejfR(w) • Far ear frequency response: HL(w) = AL(w)ej(fL(w)-wtITD)

8. Middle Ear Functions • Impedance Matching: • Sound transmission in water (inner ear) requires much higher pressure than sound transmission in air (outer ear). • Without middle ear, sound incident on oval window would bounce away (g=1) • Middle ear reduces g so that not all sound is reflected • Reduce Exposure to Loud Environments • Strap muscles loosen in loud environments, reducing the amplitude of sound transmitted to inner ear • Effect is relatively slow (hundreds of milliseconds), so not useful for adaptation to rapid sounds (gunshots)

9. Impedance Mis-Match Between Water and Air: Without a Middle Ear, What Would You Hear? Auditory Canal, contents: air Oval Window Scala Vestibuli, contents: perilymph ≈ sodium water • Continuity of pressure at the boundary: (pa++pa-) = (pw++pw-) • Continuity of volume velocity at the boundary: (A/raca) (pa+-pa-) = (A/rwcw) (pw+-pw-) • Densities: ra = 0.001 g/cc, rw = 1 g/cc • Speeds of Sound: ca = 354m/s, cw=1000m/s • Suppose pw-=0, meaning that the only input sound is pa+ • Then… The reflected sound is: pa- = (rwcw-raca)/(rwcw+raca) pa+ = 0.9994 pa+ The transmitted sound is: pw+ = 2raca/(rwcw+raca) pa+ = 0.0006 pa+ pw+ pa+ A pw- pa-

10. Hammer-Anvil-Stirrup = Lever-Based Impedance Matching System Eardrum Velocity = (1/raca)(pa+-pa-) Pressure = (pa++pa-) • Lever system reduces the effective input impedance (z=p/v) of water by a factor of L2 • Resulting reflection coefficient g = (rwcw-L2raca)/(rwcw+L2raca) ~ 0.98-0.99 < 1 Oval Window Velocity = (1/Lraca)(pa+-pa-) Pressure = L (pa++pa-) L units length 1 unit length

11. Acoustic Impedance of Ear Canal Informative About Middle Ear Function Outer Ear, contents: air pa+ • Remember how to calculate impedance? • Impose a Boundary Condition at Far End pa-=gpa+ • Calculate z=p/v at Near End z = p/v = rc (pa+e-jkx+pa-ejkx)/(pa+e-jkx-pa-ejkx) = rc (e2jkL + g)/(e-2jkL - g) • So by measuring the acoustic input impedance of the auditory canal very precisely, it’s possible to deduce g at different frequencies, and thus to learn something about health of the middle ear (product: Mimosa Acoustics) Eardrum pa- x 0 -L

12. Inner Ear Anatomy(image courtesy Alec Salt, Otolaryngology, Washington University)

13. Inner Ear Anatomy: Charged Fluids(image courtesy of Alec Salt, Otolaryngology, Washington University)

14. Cross-Section of the Basilar Membrane(image courtesy wikipedia)

15. Frequency Selectivity of Places on the Basilar Membrane Basilar Membrane (separates scala media & scala tympani) Unroll Oval Window Apex, x~3cm: Low Stiffness High Mass fc = (k/m)1/2/2p ~ 40Hz Base, x=0mm: High Stiffness Low Mass fc = (k/m)1/2/2p ~ 16000Hz In between: Each position, x, is tuned to a different mechanical resonance x(fc) ~ 30mm – (11mm) ln(1 + 46fc/(fc+14700))

16. Frequency Selectivity of Places on the Basilar Membrane Scala Vestibuli Oval Window Wave pw+e-jwx/c propagates forward at c=1000m/s until… Round Window Scala Tympani Wave energy is absorbed by oscillation of the basilar membrane at x(fc=w/2p)

17. Frequency Selectivity of Places on the Basilar Membrane Traveling waves in the cochlea. “Concerning the pleasures of observing, and the mechanics of the inner ear,” Nobel Lecture, 1961, Georg von Békésy (courtesy of Pacific Biosciences Research Center Hawaii)

18. Bandwidth of the Auditory Filters: 100Hz at fc<500Hz, 0.2fc at fc>500Hz(image courtesy Julius Smith and Jonathan Abel, CCRMA, Stanford) Equivalent Rectangular Bandwidth (ERB) = Bandwidth of an ideal BPF that passes the same total energy as the basilar membrane section at the same fc

19. Velocity of Basilar Membrane Causes Inner Hair Cell Follicles to Bend

20. Velocity of Basilar Membrane Causes Inner Hair Cell Follicles to Bend

21. Bending of Follicles Causes Depolarization of IHC Scala Media (+80mV) Cations enter through follicle tips when follicles bend Organ of Corti (0mV)

22. Depolarization of IHC Causes Release of Neurotransmitter Cations enter through follicle tips when follicles bend Neurotransmitter released Synapse, afferent neuron

23. Neurotransmitter Dynamics(Three-Store Model: Meddis, JASA 1986) Neurotransmitter Re-uptake (several ms) Neurotransmitter release (instant) Neurotransmitter binding (several ms) Result: probability of neuron firing is a smoothed (lowpass filtered) version of the IHC voltage

24. Signal Processing in the Inner Ear (Simulated)

25. Neural Response to a Synthetic Vowel(Cariani, 2000)