Effects of attention on auditory scene analysis.

1. Bob Carlyon MRC Cognition and Brain Sciences Unit, Cambridge, England Effects of attention on auditory scene analysis. With more than a little help from: Rhodri Cusack, Matt Davis, John Deeks, Jessica Foxton, Olaf Hauk, Antje Heinrich, Ingrid Johnsrude, Friedemann Pulvermuller, Yury Shtyrov, Sarah Thompson

2. Auditory scene analysis and attention • In everyday life we listen to sounds such as speech in the presence of interfering sounds • The auditory system has to: • Determine which parts of the stimulus belong to the same source (cf. grouping or binding in vision) • Deal with instances where portions of the target voice are masked • I’ll describe two phenomena that reflect how the auditory system performs these tasks, & discuss the role of attention in each. • Compare results between: • Perform some task on the stimulus of interest throughout • Perform a demanding competing task on another stimulus

3. Fused (galloping rhythm - “horse”) B A A Segregated (individual pulsing rhythms - “morse”) Frequency Frequency Time Auditory Streaming Tone sequences can be perceived as either fused or segregated: B A A “Morse” percepts increase with freq separation, rate, and build up over time

4. The “horse morse” task Our question: does build-up of streaming still occur if you’re not attending to the tones? Not our question: Is attention needed for all grouping processes?

5. Baseline Condition “Horse morse” sequence presented for 20 sec to left ear (over headphones). Frequency separation between tones: 4,6,8, or 10 semitones Task: press button 1 or 2 depending on whether you hear “horse” or “morse”. Every time your percept changes, press the other button Experiment 1 Two-task condition As above, except that during first 10 seconds a sequence of noise bursts presented to right ear. Each noise burst chosen at random to sound like it was either “approaching” or “departing” Task: respond “approach” or “depart” to each burst. After 10 sec, a message flashes on screen, noise bursts stop, and subject reverts to the “horse morse” task. One task condition Noise bursts are present, but subject ignores them and does “horse-morse” task throughout

6. Results When attention is diverted, build-up of streaming is reduced Or..the act of switching attention to tones “resets” streaming

7. Attention & Streaming: Further findings Not due to switching tasks: if subjects do an orthogonal task on tones during 1st half of sequence, and then start making streaming judgements, streaming has built up A similar effect can be obtained by requiring subjects to count backwards in 3s silently, or perform a visual competing task

8. Left ear: 13.5-sec ABA tone sequence, target early or late in sequence, delay on B tone: Attention & Streaming: Objective measure Build-up & attention effects can be measured objectively using task that can only be done when sequence heard as 1 stream (Thompson et al): Right ear: Approach-depart noises throughout. Attend to tones throughout or Attend to noises for 6.75 sec, then attend to tones

9. The continuity illusion (“CI”) When a sound is turned on and off briefly, and the silent gap is filled by an “inducing sound”, it can be heard as continuous. Allows us to perceive a sound as “whole” even when portions of it are masked. We’ve previously shown a neural correlate of the illusion using an EEG measure (“Mismatch Negativity”; Micheyl et al) Here we study the effects of CI on perception of speech sounds: Behavioral study (Carlyon et al, 2002, Acta Acustica) Neural (fMRI) correlate (Heinrich et al, 2008, JCN) Effect of attention on neural correlate (Heinrich et al, brand new)

10. Simultaneous: Alternating: FREQ--> Illusion: TIME--> CI and vowel identification Carlyon et al (2002) measured vowel identification under 3 main conditions: Performance in Illusion condition > Alternating condition

11. CI and vowel identification: fMRI study Main comparison is between “illusion” condition and a new condition, “illusion break”, in which formant level is increased & noise level decreased: Stimuli broadly similar to those in behavioural expt. Previous finding: primary auditory areas show more activation to stimuli with lots of onsets & offsets (Cusack, Carlyon, Johnsrude, & Epstein, 2001) We predicted an interaction: “Primary” areas more activated by “illusion break” “Speech sensitive” areas more activated by “illusion”

12. CI and vowel identification: fMRI study “Illusion” & “Illusion break” conditions, plus 6 others designed to give speech or non-speech percepts: 8.4-sec sequences of 10 sounds. Each sequence= 1 condition 19 RH Subjects detect rare “soft” sounds Siemens 3T Tim Trio system; head gradient coil Sparse imaging procedure

13. CI and vowel identification: fMRI study 1) Define “Speech area” (Speech-Non Speech conditions) 2) Define “General Sound” area (Non Speech-Silence) 3) Compare activation of illusion vs illusion break conditions in the 2 areas Speech area: Bilateral posterior MTG General Sound area: Bilateral middle STG

14. CI and vowel identification: fMRI study Speech area: Illusion> Illusion Break General Sound Area: Break > Illusion

15. Effect of attention on neural correlate of CI Repeat Heinrich et al study with 4 stimulus conditions (Vowel75, Vowel300, Illusion, Break). In all conditions the sounds were accompanied by Sequence of noise bursts in a remote frequency region Sequence of visual stimuli (oval balls) presented on a screen 3 Attention conditions: Listen for occasional attenuated sound in vowel-like sequence Detect an occasional visual deviant (“broken ball”) Detect occasional noise deviant (“depart sound”) Hypothesis: If CI depends on attention, then activation in speech region should be reduced when attention diverted in the “Illusion” condition, & more so than in Vowel75 and Vowel300 conditions

16. RH: Attention affects illusory and “real” vowels similarly LH: Attention doesn’t affect illusory vowels Effect of attention on neural correlate of CI Replicate location of Speech-sensitive region Attending to auditory or visual stimuli increased activity in auditory & visual sensory areas, respectively. Effects of attention on Vowel-Break and Illusion-Break Voxel with largest sound X attention interaction in each hemisphere, restricted to “speech” region from Heinrich et al (2008): …no evidence for affect of attention on the Continuity Illusion

17. Summary • Attention has a strong effect on auditory streaming. Either: • Diverting attention prevents build-up. • Switching attention back to sequence “resets” streaming. • Effects can be obtained with: • Auditory, visual, or non-sensory competing tasks • Subjective or performance-based measures • No evidence for an effect of attention on continuity illusion • When studying effects of attention on auditory scene analysis: • Need to use a demanding competing task • Effect of attention unlikely to be the same for all aspects of scene analysis (cf. harmonicity; Alain et al)