1 / 53

The Neural Basis of Speech Perception – a view from functional imaging

The Neural Basis of Speech Perception – a view from functional imaging. Sophie Scott Institute of Cognitive Neuroscience, University College London. This approach to speech perception. Speech is an auditory signal

candie
Download Presentation

The Neural Basis of Speech Perception – a view from functional imaging

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. The Neural Basis of Speech Perception – a view from functional imaging Sophie Scott Institute of Cognitive Neuroscience, University College London

  2. This approach to speech perception • Speech is an auditory signal • It is possible to address the neural processing of speech within the framework of auditory cortical processing. • This is not synonymous with the entire language system. • If one is a skilled speaker of a language, then speech perception is obligatory.

  3. Functional imaging • Where neural activity occurs, blood is directed. • Measure neural activity by tracking these changes in local blood flow. • Thus measuring mass synaptic activity • Poor temporal resolution • Essentially a comparison of blood flow changes across conditions - so the baseline comparisons are critical

  4. Listening Wise et al, Lancet, 2001

  5. Neuroanatomy of speech Speech production Speech perception

  6. CM A1 MC CP medial ML RM R AL caudal RT RTM Tpt RTL A1 CORE sts dorsal RP Ins paAlt lateral BELT rostral TS3 sts PARABELT TS2 TS1 Pro Scott and Johnsrude, 2003, from Romanski et al, 1999

  7. Dorsal prearcuate (8a) STGc AI CL Dorsal principal sulcus (46) R ML Inferior convexity (12) AL RT Orbitalpolar Prefrontal cortex Core Belt Parabelt CBP RBP STGr From Kaas and Hackett, 1999

  8. Spatial representations tonotopy bandwidth Conspecific vocalisations

  9. Anterior Posterior STP Tpt HG PT Assoc STP STS Ventral C B PB Human Assoc STS Monkey

  10. Scott and Johnsrude, 2003 anterior medial lateral posterior PA STA ALA LP AA MA LA A1

  11. Scott and Johnsrude, 2003 Amplitude modulated noise against unmodulated noise: Giraud et al, 1999 Sounds with harmonic structure against pure tones: Hall, Johnsrude et al., 2002 anterior medial lateral posterior Frequency modulated tones against unmodulated tones: Hall, Johnsrude et al., 2002 Spectral change against steady state sounds: Thivard et al, 2000

  12. Hierarchical processing • Structure in sound is computed beyond primary auditory cortex • More complex structure (e.g. spectral change) processed further from PAC • How does this relate to speech processing?

  13. speech rotated speech noise vocoded speech rotated noise vocoded speech

  14. (Sp + VCo + RSp) - RVCo (Sp + VCo + RSp) - RVCo Left hemisphere -60 -4 -10 Z = 6.6 -64 -38 0 Z = 5.7 1 1 0 0 -1 -1 -2 -2 Sp VCo RSp RVCo Sp VCo RSp RVCo Anterior (Sp + VCo) - (RSp + RVCo) (Sp + VCo) - (RSp + RVCo) -54 +6 -16 Z = 4.7 -62 -12 -12 Z = 5.5 2 1 1 0 0 -1 -1 -2 Scott, Blank, Rosen and Wise, 2000 Sp VCo RSp RVCo Sp VCo RSp RVCo

  15. Right hemisphere Anterior (Sp + RSp) - (VCo + RVCo) +66 -12 0 Z = 6.7 2 1 0 -1 Sp VCo RSp RVCo Scott, Blank, Rosen and Wise, 2000

  16. Intelligibility

  17. Plasticity within this system Naïve subjects were scanned before they could understand noise vocoded speech, then they were trained, then scanned again.

  18. Flexibility in speech perception: learning to understand noise vocoded speech Activity to noise vocoded speech after a training period, relative to prior activity to NVC before the training period. Narain, Wise, Rosen, Matthews, Scott, under review. As well as left lateralised STS, there is involvement of left premotor cortex and the left anterior thalamus (which receive projections from the belt and parabelt).

  19. Spectrograms of the stimuli (speech) 16 8 4 3 2 1 (rotated speech) 16R 3R

  20. Intelligibility - behavioural data

  21. Z=5.6 x=-62 y=-10 z=80 Z=4.52 x=-64 y=-28 z=8 Left 1 2 3 4 8 16 3R 16R 1 2 3 4 8 16 3R 16R Right Z=5.96 x=64 y=-4 z=-2 Z=4.73 x=-48 y=-16 z=-16 1 2 3 4 8 16 3R 16R 1 2 3 4 8 16 3R 16R Scott, Rosen, Lang and Wise, 2006

  22. Amplitude modulated noise against unmodulated noise: Giraud et al, 1999 Sounds with harmonic structure against pure tones: Hall, Johnsrude et al., 2002 Frequency modulated tones against unmodulated tones: Hall, Johnsrude et al., 2002 Spectral change against steady state sounds: Thivard et al, 2000 Scott and Johnsrude, 2003 anterior medial lateral posterior Peak responses to Intelligibility (Scott et al, 2006)

  23. Speech specific processing • Does not occur in primary auditory cortexd • Begins early in auditory cortex - in areas that also respond to AM • As we move forward down the STS, the responses become less sensitive to acoustic structure - resembles behavioural profile

  24. Speech comprehension - The role of context • e.g., words recognised more easily in sentences • “The ship sailed the sea” > “Paul discussed the dive”. • Can we identify the neural basisofthis contextual modulation of speech comprehension? • (Miller et al., 1951; Boothroyd and Nittrouer, 1988; Grant and Seitz, 2000;Stickney and Assmann, 2001; Davis et al., 2005)

  25. (noise vocoding:Shannon et al., 1995 predictability: Kalikow et al., 1977)

  26. Low predictability:log increase with more channels …‘Sue was interested in the bruise’… 27 jonas obleser

  27. High predictability:influence at intermediate number of channels Behav 2 low+high …‘Sue was interested in the bruise’… …‘He caught the fish in his net’… 28 jonas obleser

  28. Bottom-up processes:correlations with number of channels (cf. e.g. Binder et al. 2000; Scott et al., 2000; Davis & Johnsrude 2003; Zekveld et al., 2006) Obleser, Wise, Dresner, & Scott, 2007 RFX p<0.005 uncorrected, k>30

  29. Left-hemispheric array of brain regions when context affects comprehension Lateral Prefrontal (BA 8) Medial Prefrontal (BA 9) Angular Gyrus (BA 39) Ventral IFG (BA 47) Posterior Cingulate (BA 30) RFX p<0.005 uncorrected, k>30 Obleser, Wise, Dresner, & Scott, 2007

  30. findings • A range of brain areas outwith auditory cortex contribute to ‘top down’ semantic influences on speech perception • Further studies will be able to dissociate the contributions of different linguistic factors

  31. Words are not the only things we say

  32. Non speech sounds? x=54 Regions inredrespond to noises and rotated noises Regions inyellowrespond to noises and rotated noises

  33. Right hemisphere Anterior (Sp + RSp) - (VCo + RVCo) +66 -12 0 Z = 6.7 2 1 0 -1 Sp VCo RSp RVCo

  34. What drives lateral asymmetry? • Previous studies have not generally used ‘speech-like’ acoustic modulations • We aimed to manipulate speech stimuli to vary the amplitude and spectral properties of speech independently • Control for intelligibility • Do we see additive effects of amplitude and spectral modulations? • Are these left lateralised?

  35. Steady spectrum, steady amplitude Steady spectrum, varying amplitude Varying spectrum, steady amplitude Varying spectrum, varying amplitude

  36. Significantly more activated by stimuli with both AM and SpM Similar response to AM and SpM Down for flat amplitude and spectrum Ideal additive effects Effect size

  37. Additive effects Flat AM SpM SpMAM PET scanning, 16 runs, N=13, thresholded at p<0.0001, 40 voxels Flat AM SpM SpMAM

  38. Additive effects Flat AM SpM SpMAM PET scanning, 16 runs, N=13, thresholded at p<0.0001, 40 voxels Flat AM SpM SpMAM

  39. But… • Is there a problem - were these stimuli really processed as speech? • To address this, 6 of the 13 subjects were pretrained on speech exemplars, and the speech stimuli were included as a 5th condition.

  40. A B C D E speech

  41. A B C D E speech

  42. Flat AM SpM SpMAM Flat AM SpM SpMAM Flat AM SpM SpMAM Speech conditions N=6, thresholded at p<0.0001, 40 voxels

  43. Speech conditions Flat AM SpM SpMAM Flat AM SpM SpMAM N=6, thresholded at p<0.0001, 40 voxels

  44. Asymmetries in speech perception • Exist! • Are not driven by simple properties of the speech signal • Right - preferentially processes speech-like sounds - voices? • Left - processes linguistically relevant information

  45. Posterior auditory areas • In primates, medial posterior areas show auditory and tactile responses • What do these areas do in speech processing in humans?

  46. Wise et al, 2001, Brain Speaking and mouthing Wise, Scott, Blank, Murphy, Mummery and Warburton, 2001 This region, in the left posterior temporal-parietal junction, responds when subject repeat a phrase, mouth the phrase silently, or go ‘uh uh’, over mentally rehearsing the phrase

  47. Amount of DAF (0, 50, 125, 200ms) Listening over silence

  48. DAF peak on right 0 50 125 200

More Related