Connecting Sound with the Mind’s Eye: Multisensory Interactions in Music Conductors

1. Connecting Sound with the Mind’s Eye: Multisensory Interactions in Music Conductors W. David Hairston, Ph.D Advanced Neuroscience Imaging Research Lab Department of Radiology; Wake Forest University School of Medicine, Winston-Salem, NC 27157

2. Multisensory enhancement within a number of paradigms, including: • Simple reaction times • Saccadic response latencies • Signal detection • Orientation/localization behavior in cat General conditions for enhancements to be observed -spatial alignment -temporal congruence -minimal efficacy (“inverse effectiveness”)

3. What about human localization ability Stimuli: Vis (LED) Aud (broadband) Vis-Aud 50 ms, 30 reps Hairston et al, J. Cog. Neuro, 2003

4. Evaluating Precision Precision ~ Std Dev

6. Hmmmm…. Why? The total amount of gain/ enhancement observed is determined by the relative contribution of each sense, based upon its own perceptual acuity How do you increase auditory acuity?

7. Conductors as auditory localizers Daily, career experience requires rapid and accurate assessment of auditory scene Also requires coordination of multisensory information – “who” played “what” wrong note, etc A# 20 conductors, min 7 years “podium experience” (ave 10.2) Matched on age, education, sex, etc (Hodges, Hairston & Burdette ’06) A

8. Non-musicians Visual and multisensory performance very similar

9. 40 30 20 10 Improved auditory performance Multisensory performance enhanced

10. Conclusions from this… • While untrained subjects gain little from an additional auditory cue, music conductors appear to benefit from additional auditory signals, for which their spatial acuity is much better. • The degree of multisensory gain in this unique population appears to be tied to their improved auditory performance • These results suggest that the specialized training and experience of these individuals has a profound affect within both the auditory and multisensory realms

11. But what if… • Multisensory “enhancements” are beneficial when the task can make use of additional information • BUT - When do not match or are not relevant – can be detrimental or inhibiting…. -Slower RT -Biased localization -Illusions and misperceptions Many times the task at hand required focusing on one sense and ignoring other.

12. “Tuning out” irrelevant information…

13. Current studies… • Physiological effects of “tuning in” to one sense over another • “Cross-modal deactivation”: Decreased activity within one sensory cortex related to stimulation of another E.g., decreased BOLD signal in occipital cortex during auditory task • Some evidence for relation to selective attention Unclear whether the extent of deactivation observed is related to the difficulty of the task at hand

14. Methods Goals: Does changing task difficulty affect cross-modal deactivation? Does extreme acuity within one modality, and unique multisensory training affect this process? Subjects - Non-musicians – various careers, no formal musical training, 25-40 y/o - Conductors – min. 5 yrs podium experience Matched on age, gender, education, etc Why conductors? - Heightened auditory acuity - Conducting activities require NOT inhibiting other (e.g., visual) information

15. 660 Hz 660 Hz Base Test 440 Hz 456 Hz Easier 440 Hz 443 Hz Difficult (Time) 440 Hz Easier 60 ms 440 Hz Difficult 20 ms Methods: Tasks Pitch Discrimination Temporal Discrimination Subjects’ thresholds for each task acquired prior to fMRI scanning

16. Methods: Thresholding Acquired threshold Allows compensation for variability in perceptual acuity between subjects and groups 2 down/1 up rule

17. Non-Musicians Conductors Perceptual acuity, non-musicians vs. conductors A# Base comparison A Non-musicians need a significantly larger difference to discriminate tones than conductors.

18. Trial Trial Trial TR silence TR (Scanner OFF) (Scanner ON) (Scanner ON) Methods: fMRI Each task performed at 2 levels - at threshold (“difficult”) - above threshold (“moderate”) Also performed visual temporal discrimination (2 circles) Visual Temporal discrimination Sparse sampling (10 s pause), block design

19. Positive BOLD Positive BOLD Schmuel et al, Nature Neuroscience, 2006

20. 2.78 7.0 -2.78 -7.0 Analysis • Activity in task (ON) blocks contrasted against resting baseline (OFF) Baseline: eyes open on fixation, no task • ROI of visual-responsive occipital cortex Used to generate summary stats

21. Vis ROI +/-3.95 +/-8.0 Non-musicians: Moderate Deactivation of visual cortex

22. Vis ROI +/-3.95 +/-8.0 Non-musicians: Difficult Robust deactivation of visual cortex

23. Non-musicians: Difficult vs. Moderate Easier Difficult

24. Non-musicians: stats # significant voxels Post. Cing. V Cort. ROI A.Cort. A.Cort. V Cort. Post. Cing. ROI Mean signal Total Signal Magnitude

25. Vis ROI Vis ROI +/-3.95 +/-8.0 Conductors Moderate Difficult Only slight deactivation of visual cortex in both cases

26. Conductors: Difficult vs. Moderate # significant voxels A.Cort. V Cort. Post. Cing. ROI Difficult Easier Total Signal Magnitude

27. * n.s. Non-musicians vs. Conductors Difficult Task Difference seen when task is difficult * Non-Musicians Conductors

28. Conclusions • The degree and extent of cross-modal deactivation observed specifically depend of the task difficulty; when the same task is easier, cross-modal deactivations are attenuated. • Contrary to non-musicians, conductors show only minor cross-modal deactivation, even when the task is very difficult to perform. • This suggests that the role of functional deactivations is dynamic, and adapts to fit the needs of the individual and situation at hand.

29. Conductors • Conductors are highly-trained individuals with unique daily activties and specialization • This experience leads to not only high auditory acuity, but altered interactions when dealing with multiple senses • Two theories: - High auditory acuity negates the need for visual suppression. - Daily experience has led to familiarity with completing complex auditory tasks while also monitoring visual information.