Biologically Inspired Intelligent Systems

1. Biologically Inspired Intelligent Systems Lecture 6 Roger S. Gaborski

2. Auditory System

3. Auditory Pathways • Auditory • Identify objects by their sound (Temporal lobe) • Direct our movements by sound (Posterior Parietal Region) • Phone rings in the dark, you form your hand into the correct shape to pick up the phone (similar to a visual image directs your hand to form a particular shape)

4. Notes taken from several sources, including: • http://openlearn.open.ac.uk/mod/resource/view.php?id=263180 • www.cartage.org.lb/.../PressureWave.htm

5. Sound Wave • Pattern of high pressure and low pressure regions moving through a medium • Detector: Human ear or mechanical transducer • Detects change in pressure • Detect high pressure (compression) • Detect normal pressure • Detect low pressure (rarefaction )

6. Pressure Waves, not to be confused with electromagnetic light waves www.cartage.org.lb/.../PressureWave.htm

7. Auditory System accomplishes three tasks: • Deliver the acoustic stimulus to the receptors • It must transduce the stimulus from pressure changes into electrical signals • It must process the electrical signals: • Qualities of the sound source: • pitch • Loudness • location.

8. The Human Ear • the outer ear, which is responsible for gathering sound energy and funneling it to the eardrum • the middle ear, which acts as a mechanical transformer • the inner ear, where the auditory receptors (hair cells) are located.

9. Tympanic Membrane

10. Outer Ear:Tympanic Membrane • Sound, consists of compressions and rarefactions of air particles • The sound pulls and pushes at the membrane moving it inwards and outwards at the same frequency as the incoming sound wave. • This vibration ultimately leads to the perception of sound. • Greater amplitude of the sound waves causes greater deflection of the membrane. • The higher the frequency of the sound, the faster the membrane vibrates.

11. Middle Ear • Three inter locking bones • Transmit vibrations of the membrane to the inner ear

12. Inner Ear

13. Inner Ear • Three parts: • the semicircular canals -sense of balance • the vestibule-sense of balance • the cochlea- key component responsible for auditory perception

14. Frequency Coding • Cells code frequencies based on their location on the basilar membrane • Cells at base of cochlea are most sensitive to high frequency sounds • Cells at apex are displaced by low frequency sounds

15. Basilar Membrane hyperphysics.phy-astr.gsu.edu

16. Cochlear Nerve • Formed by axons of bipolar cells • Each bipolar cell is connected to only one hair cell and therefore contain information on the place on the basilar membrane that is being excited • Not a single frequency response, but a range of responses

17. Tuning (Similar to cone responses in retina)

18. Example Extract for analysis www-ccrma.stanford.edu/~unjung/mylec/mfcc.html

19. Extracted Signal Blue signal original, Red signal original signal processed with Hamming window

20. www-ccrma.stanford.edu/~unjung/mylec/mfcc.html

21. Cochlear ImplantInspired by Cochlear Frequency Filtering

22. Mel-Frequency Cepstrum (MFC) • Short-term power spectrum of a sound • Frequency bands are equally spaced on the mel scale which approximates the human auditory system's response • Results in better audio compression • Used as features for speech recognition and speaker identification • Use for music classification (Cognitive Component Analysis preprocessing) http://en.wikipedia.org/wiki/Mel-frequency_cepstrum

26. Tonotopic Map in Auditory Cortex

27. Topographical Maps • Visual – V1 has a topographical map • Somatic – S1 has a topographical map • Auditory – A1 has a topographical map • Orthogonal to frequency axis is a striped arrangement of binaural properties • One stripe excited by both ears • Next stripe excited by one ear and inhibited by the other ear (ocular dominance columns in V1) • Unclear what happens in higher levels

28. Ice-cube model of the auditory cortex

29. AudiogramsHow Does Our Hearing Compare? More Energy Less Energy www.owlnet.rice.edu

30. Hearing Like a Dolphin? Frequency Transformation Mapping g = f(x) Wide Range Acoustic Sensor Speaker Other issues: Sensitivity

31. Sound Localization Use both ears to detect time of arrival differences for low frequency sounds (30 usec) (ITD - interaural timedifferences ) High frequency sounds are blocked by the head, so they are attenuated and this difference in intensity is used to locate the source of the sound www.unmc.edu/Physiology/Mann/mann8.html

32. Sound Localization • Detection of the location of a sound is function of neurons in the superior olive and the trapezoid body of the brainstem

33. Observations • “Development of Sound Localization Mechanisms in the Mongolian Gerbil Is Shaped by Early Acoustic Experience,” (Armin H. Seidl and Benedikt Grothe Max Planck Institute of Neurobiology, D-82152 Martinsried, Germany) • ITD sensitivityin gerbils undergoes a developmental maturation after hearingonset • Development can be disruptedby altering the animal's acoustic experience during a criticalperiod • Animals that had been exposed to omnidirectionalwhite noise during a restricted time period right after hearingonset, ITD tuning did not develop normally • Animals that had been exposed to omnidirectional noise as adultsdid not show equivalent abnormal ITD tuning

34. Understanding Sound Patterns-cortex neurons • Music – right temporal lobe specialization • Speech – left temporal lobe • Little knowledge of how music and information is processed at the cellular level

35. Chomsky’s and Pinker’s Argument • All languages have common structural characteristics • WHY?? • Reason

36. Chomsky’s and Pinker’s Argument • All languages have common structural characteristics • WHY?? • Reason- genetically determined constraint on the nature of human language • 1. Justifications for statement??? • 2. • 3 • 4.

37. Chomsky’s and Pinker’s Argument • All languages have common structural characteristics • Reason- genetically determined constraint on the nature of human language • Language is universal in human populations • Complexity of language not related to the complexity of the group’s culture • Language of technically primitive cultures are as complex and elegant as languages of industrialized cultures • Language is learned early in life without effort –12 months words ->3 yrs rich language ability – environment influenced • Languages have basic structure components – verbs, subjects – different order, etc.

38. Speech Areas • Wernicke’s Area-temporal lobe on the left side of the brain • Language comprehension (area where spoken words understood) • Language Recognition • Broca’s Area-located in the left frontal lobe • Controls facial neurons • Involved in language processing • Speech production and comprehension

39. Damage to Wernicke’s Area • Loss of ability to understand language • Can speak clearly, but order of words do not make sense

40. www.indiana.edu/~pietsch/hemianopsia.html

41. Broca and Wernicke’s Areas www.indiana.edu/~pietsch/hemianopsia.html

42. Damage to Broca’s area • Person understand language, but cannot produce speech • Words not properly formed • Speech slow and slurred

43. Understanding Wernicke’s area (contains sound images of words) Comprehend Word Heard Spoken Word A1

44. Speaking a ‘Thought’ Broca’s Area (stores motor programs for speaking words) Wernicke’s Area “Thought” Cranial nerves Facial area of motor cortex “Speak”

45. Model Model of Cochlea Acoustic Model Word Model Language Model

46. Research Areas: Projects, Thesis and Independent Study • Biologically plausible/natural methods for temporal data processing • Clustering/classification techniques for temporal data • Feature extraction and representations of temporal data • Temporal information processing and fusion • Representation/analysis/recognition of actions and events from temporal data

47. Research Areas: MS Projects, Thesis and Independent Study • Time-series modeling/forecasting • Temporal information indexing and retrieval • Temporal data mining and knowledge discovery

48. Cognitive Component Analysis Hansen, Larson, et. al.