Synaesthesia: an investigation using Stroop-like tasks and PET imaging

1. Synaesthesia: an investigation using Stroop-like tasks and PET imaging Tyson Baker Psychology 347.3 Monday, November 25

2. Presented articles • Paulesu, E., Harrison, J., Baron-Cohen, S., Watson, J.D.G., Goldstein, L., Heather, J. et al. (1995). The physiology of coloured hearing: A PET study of colour- word Synaesthesia. Brain, 118, 661- 676. • Odgaard, E.C., Flowers, J.H., & Brahman, H. L. (1999). An investigation of the cognitive and perceptual dynamics of a colour—digit synaesthete. Perception, 28, 651-664.

3. Paulesu et al. (1995)Introduction • Synaesthesia has been in scientific literature since John Locke (1690) • Over 300 years! • Many Synaesthetes ridiculed as children, try to repress it, but it stays • Remember having Synaesthesia all their lives; same inducer-concurrent connections earlier than 4 years old • Many musical composers theorized to have had ‘coloured hearing’ • Some (8) lexical-chromatic (sounds of words produces perception of colour) have colour mostly dependent on the dominant letter of word (Baron-Cohen et al., 1993) • 5 stated colour was not in visual field • 2 stated just above center of gaze • 6 stated it took the form of the word • 1 stated no particular shape • E.P. had a specific shade for each word (Baron-Cohen et al., 1987)

4. Paulesu et al. (1995)Introduction cont’d… • Cytowic (1989) suggested hypothesis that Synaesthesia was in limbic system • Xenon inhalation experiment did not test limbic system hypothesis, but did show widespread CBF decreases in the neocortex • Equipment failure? • Other information points to a cortical level (Sachs & Wasserman, 1987; Sachs et al 1988) • Synaesthete was in a car accident and became colour-blind; Synaesthesia disappeared as well.

5. Paulesu et al. (1995)Hypotheses • H1: Synaesthetes hearing words will show activation of brain areas responsible for colour perception under normal circumstances • E.g.. The fusiform gyrus (Lueck et al., 1989; Zeki et al., 1991) • H2: (Alternatively) Synaesthesia may be due to a connection between the auditory verbal cortex and some associative visual area responsible for representing colours and letters conjointly • Inferior temporal region used to identify objects based on colour and shape (Corbetta et al., 1991) • H3: Will the primary visual cortex (V1/V2) show activation? • Yes: support for feedback connections • No: support that brain is capable of generating perception of colour without V1 or V2

6. Paulesu et al. (1995)Methods • Subjects • Synaesthetes • 6 females (5 R-handed, 1 L-handed) • Synaesthesia ONLY responded to words • Mean age was 45 • MRI scans checked for large anatomical damage • Controls • 6 females (5 R-handed, 1 L-handed) • No record of Synaesthesia, neurological or psychiatric disease, or currently on psychoactive drugs • Mean age was 40 • Did not know the experiment was dealing with Synaesthesia

7. Paulesu et al. (1995)Methods cont’d… • Psychological assessment • National Adult Reading Test (Nelson, 1982) • Test of general intelligence • Genuineness for Synaesthesia test (Baron-Cohen et al., 1987) • Describe colours of more than 100 lexical items, tested 1-10 months later without warning (mean 6 months) • Test of phonological-based Synaesthesia or letter-based Synaesthesia • Photograph and Fish->phonological based • Kick and Knock->letter based

8. Paulesu et al. (1995)Methods cont’d… • Psychological stimulation • 12 consecutive rCBF (radioactive Cerebral Blood Flow) measurements, 6 in each of: • Control task • Pure tones within human voice range for 0.5s • Instructed to tap left index finger for every tone heard, intervals were random between 0.5s-1.5s • Experimental task • Similar as control task, but using single words • All words highly imaginable concrete words • All subjects to listen passively, but still tap finger

9. Paulesu et al. (1995)Methods cont’d… • PET data acquisition • All subjects also received an MRI scan coregistered with their PET scans to improve anatomical localization • Used 15O-labelled water (H215O) injected intravenously

10. Paulesu et al. (1995)Methods cont’d… • MRI data acquisition • Used to exclude anatomical lesions in volunteers • images aligned parallel to intercommissural line and interpolated to obtain a cubic voxel the size of 1 mm • Permitted coregistration with PET scans; both MRI and PET voxels were converted into cubes and combined into one image

11. Paulesu et al. (1995)Results • IQ estimates for Synaesthetes were in the superior range (NART range 115-128) • Consistency scores were 90% and above for all • 5 of 6 Synaesthetes reported numbers 1-9 had colour associations not consistent with the first letter of the spelled number • Synaesthetes experience colour during inner speech, but not during reading—unless they subvocalize what they are reading • Colour percept was dependent on letter, not phoneme • E.g.. “Nice”->Chocolate brown, “Knock”->Biscuit, “Kind”->Biscuit • 5 of 6 Synaesthetes’ colour percept was dependent on the first letter of the word, 1 of 6 percept was dependent on the first vowel

12. Paulesu et al. (1995)Results cont’d… • As expected, Both groups showed increased rCBF while hearing words compared to tones in areas involving language • Bilateral superior and middle temporal gyri • LH inferior frontal gyrus

13. Paulesu et al. (1995)Results cont’d… • Synaesthetes also showed increased rCBF when hearing words compared to tones in • RH middle frontal gyrus • RH insula • LH posterior inferior temporal (PIT) cortex • Synaesthetes showed increased rCBF when hearing words compared to controls hearing words in • RH middle and *inferior frontal gyrus • RH insula • *RH superior temporal gyrus • LH posterior inferior temporal (PIT) cortex • *LH superior occipital gyrus • *Bilateral superior parietal lobule junction

14. (top) Synaesthetes word vs. tone(bottom) Synaesthetes word vs. control word

15. Paulesu et al. (1995)Results cont’d… • Synaesthetes showed significant deactivations (word vs. tone) • *RH cingulate cortex • *Bilateral temporo-parietal junction • LH insula • LH Lingual gyrus • *LH Cerebellum • Synaesthetes showed significant deactivations (compared to controls) • LH insula • LH Lingual gyrus

16. (top) Deactivation in Synaesthetes (tone vs. word(bottom) Deactivation of Synaesthetes vs. controls (tone only)

17. Paulesu et al. (1995)Results cont’d… • No activation of primary visual cortex

18. Paulesu et al. (1995)Discussion • Self-proclaimed first neurophysiological account of brain activity related to colour-word Synaesthesia • Regarded as a “centrally synthesized percept rather than an a form of mental imagery”: perception, not imagination • Still to be explained why Synaesthesia is one-way • Any visual areas activated or inhibited by experiment cannot be due to real stimuli (blindfolds) • Some numbers also have colours associated that are independent of the letters that make up the written form of that number • Support for a lexical component • Similar to normals when they associate the written form of a word with the sound…(Stroop tasks?) • Strange; visual presentation of words automatically activates the visual orthographic lexicon (Patterson and Morton, 1985), yet the Synaesthetes do not perceive colours while reading silently

19. Paulesu et al. (1995)Discussion cont’d… • The ambiguous location of colour perception indicates that Synaesthesia is located in areas of visual processing with a much looser representation of the visual field than the primary visual cortex or V4 • Some suggestive evidence that some Synaesthetes in current study showed use of V4 in their Synaesthesia • Cerebral colour-blindness linked to lingual-fusiform gyri lesions • LH Lingual gyrus inhibited during Synaesthesia • Suggestive of being necessary for colour perception, with other areas (dorsal occipital, temporal and parietal cortex) used in higher levels of visual processing involving colour

20. Paulesu et al. (1995)Discussion cont’d…(PIT cortex) • Evidence from this study, and previous studies (e.g.. Corbetta et al., 1991) suggest that the posterior inferior temporal cortex is involved in complex aspects of colour perception, particularly linking colour to shape • Area not shown to be activated in normals hearing words, but is used in semantic judgments based on matching preselected types of nouns and adjectives • Explains much of “coloured hearing” if PIT cortex is used in colour processing, visual and language feature integration

21. Paulesu et al. (1995)Discussion cont’d…(RH frontal lobe) -Shown to be active in visual tasks of divided attention involving colour (i.e. looking for a red square in a variety of other coloured shapes) -Due to divided attention between words associated meaning and the colour perceived? • E.g. “grass” causes perception of blue

22. Paulesu et al. (1995)Discussion cont’d…(inhibition in LH lingual gyrus and insula) • Related to colour perception, converting written letters to sounds • Could be due to inhibitory feedback from other active visual areas (PIT)? • Could be indicative of where audio-visual pathways cross in Synaesthetes. • Related to reading • Damage can cause inability to read while other language abilities stay intact • Uncertain to relevance • ‘shutting off’ due to attentional requirements? • Not needed at the moment? • Subjects were blindfolded

23. Paulesu et al. (1995)Discussion cont’d… • Can conclude that any increased activation of visual system is due to Synaesthesia because of blindfolds

24. Odgaard, Flowers & Bradman (1999) Introduction -Case study on 46y.o. female Psychology graduate regarded as “L” -Naïve to her colour-digit synasthesia -studied when she stated “two is yellow, of course” -Experiment 1 to test if L’s synasthesia was discrete, memorable, durable, and involuntary but elicited -Experiment 2 to test lexical processing of synasthesia -Experiment 3 to investigate colour-digit patterns in Euclidian space

25. Experiment 1Introduction • Cytowic (1989) gave criteria for synasthesia, tested in this experiment by memory tests and various Stroop-like tasks. • Theorized Synaesthesia to be in the limbic system • Neuronal modulation theory states Synaesthesia to be a result of crossed neural connections during brain maturation • The memory tests are to remember colour-digit associations, which a synaesthete would obviously have an advantage • The Stroop-like tasks were used to measure the involuntary divisions in attention as a result of similar meaningfulness of the task distracter • Used to assure that the results on the memory tests are not just extremely superior memory or over learned non-synasthesia colour-digit associations

26. Experiment 1Methods: Participants • Controls • 7 males, 4 females • 19 – 39 years old (mean = 27.6) • Graduate or undergraduate students at University of Nebraska • L • 46 years old • Female • Psychology Graduate • Relatively naïve to perceptual and cognitive factors involved in Synaesthesia • Raised in Choctaw and Anglo heritage

27. Experiment 1Methods: Stimuli • Using a colour wheel, L gave a complete listing of 110 one-digit and two-digit numbers • Some unidentifiable on colour wheel • Experiment 3 will explain -identifiable digits on colour wheel were used for Stroop-type tasks • 5 types of Stroop-like tests • 8 lists each type (40 lists total) • 13 items each list • Read aloud, stated when “Done” • Baseline: xxxx, xxxx, xxxx • Incongruent colour-word associates: Sky, Grass • Incongruent colour names: blue, red, purple, green • Synaesthetically incongruent coloured numbers: • L associates such that 72, 9, 8, 7 then it would be presented in colours such as 72, 9, 8, 7 • Synaesthestically incongruent coloured numbers with negative priming: • L associates such that 72, 9, 8, 7 then it would be presented in colours such as 72, 9, 8, 7

28. Experiment 1Methods: Stimuli cont’d… • Memory tests • 1st: List of 20 single and double digit numbers • 2nd: List of same numbers as 1st, but in a new random order • 3rd: Complete list of 110 single and double digit numbers • Given only to L

29. Experiment 1Methods: Procedure • 8 weeks prior to experiment 1, L gave complete list of 110 colour-digit associations and first 2 memory tests • 1st memory test, 10 minute discussion on the characteristics of ligand-gated ion channels, 2nd memory test, 6 week interval, 3rd memory test • Controls were given the 1st memory test, then 1 random Stroop-like task from each of the 5 types (approx. 10 min.), then 2nd memory test • Controls given 1 practice trial on all 5 types of Stroop-like tasks, then 1 timed trial • L given all 40 Stroop-like tasks over 2 sessions

30. Experiment 1Results • L passed the durability and memory requirement • 95% on 2nd test (10 min interval) • 93.6% on 3rd test (6 week interval) • Controls mean accuracy was 28.4% (SD=11.4%) on the 2nd test (10 min interval)

31. Experiment 1Results cont’d… • Controls • Baseline = Synaesthesthetically incongruent coloured numbers = Synaesthesthetically incongruent coloured numbers with negative priming • Baseline < Incongruent colour-word associates • Baseline < Incongruent colour names • L • Overall slower than controls • Baseline < Synaesthesthetically incongruent coloured numbers < Synaesthesthetically incongruent coloured numbers with negative priming < Incongruent colour names • No differences between L and Controls on: • Baseline • Incongruent colour-word associates • Incongruent colour names • Controls were faster on: • Synaesthesthetically incongruent coloured numbers • Synaesthesthetically incongruent coloured numbers with negative priming

32. Experiment 1Results rephrased… • Controls • xxxx, xxxx, xxxx = 72, 9, 8, 7 = 72, 9, 8, 7 • xxxx, xxxx, xxxx < Sky, Grass • xxxx, xxxx, xxxx < blue, red, purple, green • L • Overall slower than controls • xxxx, xxxx, xxxx < 72, 9, 8, 7 < 72, 9, 8, 7 < blue, red, purple, green • No differences between L and Controls on: • xxxx, xxxx, xxxx • Sky, Grass • blue, red, purple, green • Controls were faster on: • 72, 9, 8, 7 • 72, 9, 8, 7

33. Experiment 1Discussion • Functionally, L’s scores on the incongruent coloured digits and incongruent coloured digits with negative priming are similar to the scores on incongruent colour-associate words and incongruent colour-words, respectively • Indicative of lexical component • Cytowic’s (1989) theory of Synaesthesia processed in the limbic system is discredited • Neuronal modulation theory should be altered to allow onset of lexical ability • L marginally faster at incongruent colour-word (“Classic”) Stroop task • 1)As an experimenter, L was more attentive during testing • 2)Simply fast for her age (L = 46, controls mean = 27.6) • 3)L’s colour-digit Synaesthesia helps her ‘practice’ by being constantly exposed to coloured stimulus while processing

34. Experiment 2Method • Participants • L • Controls • 6 male, 4 female • Mean age = 19.0 years, range 18-20 • Enrolled in Introductory Psychology course • Partial course requirements

35. Experiment 2Method cont’d… • Stimuli • Stroop-like tasks (80 items each test, 8 types, 5 versions each type) • Instructed to draw line through items of target colour(s) • Nonwords, numbers, colour-words • 1 or 3 target colours (24 targets per list) • Spectrally adjacent, spectrally non-adjacent • xxx, xxxx, xxxxx, 777, 9999, 88888, 777777 , blue, red, purple, green • xxx, xxxx, xxxxx, 777, 9999, 88888, 777777 , blue, red, purple, green • 8 types: X1, N1, X3A, N3A, W3A, X3non, N3non, W3non

36. Experiment 2Hypotheses H1: controls will score similar to past research, specifically: 1 target colour < 3 target colours • Nonword < colour words • Spectrally adjacent < spectrally nonadjacent H2: L will show classic Stroop • No specific hypothesis about her incongruent number performance • Exploratory nature of experiment

37. Experiment 2Results • Controls • (X1 = N1) < (X3A = N3A) < (X3non = N3non) • single targets were faster than spectrally adjacent nonwords, which were faster than spectrally nonadjacent nonwords, irrespective of stimuli being X’s or #’s • (X3A, N3A) < W3A < W3non • (X3non, N3non) < W3non • Colour word lists took longer than nonword lists, with spectrally incongruent colour words taking longer than any other stimuli

38. Experiment 2Results cont’d… • L • X1 = N1 • X3A < W3A < W3non • X1 < X3A < X3non< N3A = N3non • Synaesthetically incongruent numbers are enough to produce delay regardless of spectral adjacency

39. Experiment 2Results cont’d… • Between-group differences • L faster on X3A

40. Experiment 2Discussion • Controls performed as expected • Numbers produce no more interference than X’s • Single targets faster than 3 targets • Nonwords faster than incongruently coloured colour words • Spectrally adjacent faster than spectrally nonadjacent

41. Experiment 2Discussion cont’d… • L’s performance • Predicted • Matched control group on nonnumber Stroop lists • Unforeseen • Tripling the number of target colours almost tripled the time required (despite there still being 24 targets in the 80 item lists) • 10.30s to 28s(A), 30s(non) • 12.07s difference between X3A and N3A • 9.3s difference between X3non and N3non • Overall, spectrally adjacent and spectrally nonadjacent times are equal • Suggests due to memory overload from 3 target colours combined with the lexical component of her Synaesthesia

42. Experiment 2Discussion cont’d… • Is the apparent irrelevance of spectrally adjacent target colours (N3A vs. N3non) due to simple overloading of attentional capacity (as Odgaard and colleagues suggest), or is it somehow related specifically to an overload of attentional capacity combined with numbers, possibly only removing or reducing the advantage given by spectrally adjacent colour targets to the other lists? • N3A vs.. N3non is N.S; avg(A) vs.. avg(non) is N.S., but… • X3A < X3non; W3A < W3non • Qualitatively, what kind of colour-digit associations does L experience? • Experiment 3

43. Experiment 3Introduction • Because of the data obtained by L for Experiments 1 and 2, it seemed only logical to use the data of her reports as an attempt to find out more • Basically making her classify colour-digit associations that are impossible in Euclidian space, or group numbers she might otherwise identify as dissimilar

44. Experiment 3Introduction cont’d… • “Impossible in Euclidean space”?

45. Experiment 3Method • Stimuli • 20 single and double-digit numbers printed on individual white cards • 14 were numbers she could associate with the colour wheel • 6 were numbers she could not associate with the colour wheel • 57, 85, 86 (Red- greenish); 61, 64 (‘active’ combinations of multiple colours); 98 (orange-purple)

46. Experiment 3Method cont’d… • Procedure • L was presented with same cards on 14 occasions separated by 24 hours or a variety of distracter tasks • L instructed to sort cards into 3-6 piles, each with at least 2 cards each • L would otherwise sort cards into at least 8 piles based on colour-digit associations • Resulting piles were recorded, entered into a dissimilarity matrix

47. Experiment 3Results • Within-groups hierarchical cluster analysis was performed on the dissimilarity matrix • Four cluster solution had a coefficient of zero, so it was used (1-3 ranged from 53.25 to 8.47 respectively)

48. Experiment 3Results cont’d… • The four-cluster solution leads to a three-concept division for the non-synaethesiasts • “reddish”, “not red, but typically chromatic”, “other” • “Reddish” are digits identifiable on the colour wheel as having a red hue, sometimes with 57 • “Not red, but typically chromatic” are mostly digits identifiable on the colour wheel, but sometimes containing 57, 61, 64 • 85 and 86 in “other” as being “green-red” • 98 was by itself as being “orange-purple”

49. Experiment 3Discussion • Suggest distinction between two ‘typical’ groupings could be due to a “…metaphorical/emotional dichotomy between ‘warm’ and ‘cool’ colours.” • No further evidence • Conclude that Euclidean space is insufficient means to explain the colour-digit associations with Synaesthesia • 98 was described as a “hollow sphere of orange and greyish purple that is as real as red or green” but L stated she had “never seen anywhere in the world”

50. Odgaard et al. General Discussion • Stroop-style tasks can be used to better fulfill the diagnostic criteria of Synaesthesia • Lexical component discredits idea that Synaesthesia occurs in the limbic system • More likely due to the cortex being unable to modulate sensory information; development after or during lexical development • Synaesthesia could be the result of atypical neural development • Improbable due to onset of lexical ability • Or, could be result of experiences • Associations are learned, making child ‘prepared for Synaesthesia,’ then Synaesthesia develops with lexical ability • Unclear how it works; no research