I. Words in the Brain: Functional Webs (cont’d) II. Right Hemisphere in Language Processing

1. Ling 411 – 14 I. Words in the Brain: Functional Webs (cont’d)II. Right Hemisphere in Language Processing

2. Sequence • In language, sequence is very important • Word order • Order of phonological elements in syllables • Etc. • Also important in many non-linguistic areas • Dancing • Eating a meal • Can cortical columns handle sequences?

3. Simplified model of minicolumn I:Activation of neurons in a column Other cortical locations II III IV V VI Cell Types Pyramidal Spiny Stellate Inhibitory Thalamus Connections to neighboring columns not shown Subcortical locations

4. Lasting activation in minicolumn Cell Types Pyramidal Spiny Stellate Inhibitory Recurrent axon branches keep activation alive in the column – Until is is turned off by inhibitory cell Connections to neighboring columns not shown Subcortical locations

5. Notation for lasting activation > Thick border for a node that stays active for a relatively long time > Thin border for a node that stays active for a relatively short time

6. Recognizing items in sequence This node recognizes the sequence ab c This link stays active a b Node c is satisfied by activation from both a and b If satisfied it sends activation to output connections Node a keeps itself active for a while Suppose that node b is activated afternode a Then c will recognize the sequenceab

7. Recognizing a syllable and its demisyllables Cardinal node for dim Functional subweb for dim dim Just labels di- -im Auditory features of [di-] Auditory features of [-im]

8. Local and distal connections excitatory inhibitory

9. Cardinal nodes vis-à-vis “grandmother nodes” • ‘Grandmother node’: a node that responds to grandmother • i.e., a local representation for grandmother • The term ‘grandmother node’ usually refers to the naïve grandmother node • (To people who use the term) • Naïve grandmother node: a hypothetical node that would recognize grandmother all by itself • Alternative conception: the sophisticated grandmother node • The cardinal node of a functional web

10. The sophisticated grandmother node • GRANDMOTHER has a distributed representation • That distributed representation includes a cardinal node • This cardinal node is a local representation • Nodal specificity • It represents a specific value: GRANDMOTHER • Its receptive field is “grandmother” • Its operation is supported by an entire functional web • Other nodes in the web handle • The details • A range of diverse perceptual properties • Variety

11. Arguments against ‘grandmother nodes’ • They are directed against the naïve grandmother node • They usually assume that the local representation is representing a concept (like ‘grandmother’) all by itself • i.e., Local representation without distributed representation • i.e., without a supporting web

12. Arguments against (naïve version of ) local representation • Recognizing new things and producing motor responses to new things are problematic on the local-coding theory • The patterns recognized visually by a human in a lifetime vastly outstrip the number of sensory processing neurons in the entire human nervous system Churchland & Sejnowski The Computational Brain MIT Press, 1992, p. 163

13. Arguments against (naïve version of ) local representation • Recognizing new things and producing motor responses to new things are problematic on the local-coding theory This argument assumes that such a node recognizing grandmother all by itself. But it is the whole functional web that recognizes grandmother. Each part of this web naturally responds to a wide range of values, including novel values.

14. Arguments against (naïve version of ) local representation • The patterns recognized visually by a human in a lifetime vastly outstrip the number of sensory processing neurons in the entire human nervous system Churchland & Sejnowski 1992:163 On the contrary, the web can accommodate recognition of multiple new exemplars without the need for recruiting additional nodes. Not a problem after all. New nodes are needed only for new learning.

15. Support for the cardinal node hypothesis It follows from the properties of nodal specificity and hierarchy A hierarchy must have a highest level The node at this level must have a specific function It is needed for ignition of the whole web from activation of part of it For example, to activate the phonological representation from the visual It is automatically recruited in learning anyway, according to the Hebbian learning hypothesis Cardinal concept and phonology nodes are needed for the arbitrariness of the linguistic sign

16. Support for cardinal nodes - 2 The distributed network as a whole represents the concept (e.g. FORK) The whole can evidently be ignited by any part of the functional web From seeing a fork From eating with a fork Etc. The cardinal node provides the coordinated organization that makes such reactivation possible

17. Reactivating the functional web When the cardinal node (the integrating node) is activated, it can activate the whole (distributed) functional web Without it, how would that be possible? E.g., activating conceptual and perceptual properties of cat upon hearing the word cat From phonological recognition to concepts From visual image to phonological representation

18. Ignition of a functional word web from speech input (showing only major nodes) T M C PP PR PA V

19. Ignition from visual input T 3 M 3 C 4 2 PP 3 4 PR PA V 1 1

20. Ignition from tactile input 1 T 3 M C 4 2 PP 3 4 PR 3 PA V 1

21. Ignition from conceptual input T 2 M 2 1 C 3 PP 2 3 PR 2 PA V 1

22. Question Also, I had a question about how functional webs activate when a node activates. Would it be possible to memorize or learn information that would be activated (remembered) later by repeating a word or other stimuli that is part of the neural web? This is something they do in spy movies, but I am unsure if it actually is possible.

23. Support for the cardinal node hypothesis – 3 • It is automatically recruited in learning according to the Hebbian learning principle • Even if it weren’t there it would soon be recruited as a result of co-activation of its linked properties • This is the operating principle for building a functional web from bottom up • At each level, co-occurring properties will activate a node at next higher level • That newly activated node represents the combination of those properties • This process continues up to top of hierarchy

24. Support for cardinal nodes – 4: The linguistic sign Connection of conceptual to phonological representation Consider two possibilities A cardinal node for the concept connected to a cardinal node for the phonological image No cardinal nodes: multiple connections between concept representation and phonological image supported by Pulvermüller (2002)

25. Pulvermüller’s hypothesis:No cardinal nodes Phonological representation: a distributed representation in the perisylvian area Meaning of a visual object Meaning of a verb Friedemann Pulvermüller, The Neuroscience of Language, 2002

26. Implications of possibility 2 • No cardinal nodes: multiple connections between concept representation and phonological image • I.e., different parts of meaning connected to different parts of phonological image • Consider fork • Maybe /f-/ connects to the shape? • Maybe /-or-/ connects to the feeling of holding a fork in the hand? • Maybe /-k/ connects to the knowledge that fork is related to knife? • Conclusion: Possibility 2 must be rejected

27. Pulvermüller’s hypothesis:No cardinal nodes Phonological representation: a distributed representation in the perisylvian area Meaning of a visual object Meaning of a verb Friedemann Pulvermüller, The Neuroscience of Language, 2002

28. Functional Webs acc. to Pulvermüller • Distributed representation of form and of meaning • This part is correct • Multiple connections between form and meaning • Runs counter to the linguistic evidence • Implication: parts of the phonological representation connect to parts of the meaning • Example: walk - WALK • [w-] or [-k] for action with legs?

29. Properties of Cortical StructureApplied to Functional Webs • Property I: Intra-column uniformity of function • The nodes of functional webs are (implemented as) cortical columns • Property II: Cortical topography • Every functional web is a two-dimensional array of columns • Property III: Nodal specificity • Every node of a functional web has a specific function • Property IV: Adjacency • Adjacent nodes for related functions • More closely related function, more closely adjacent

30. Property IV(b): A deduction from the adjacency property The nodes in each area of a functional web Constitute a subweb Their function fits the portion of cortex in which they are located For example, Phonological recognition in Wernicke’s area Visual subweb in occipital and lower temporal lobe Tactile subweb in parietal lobe Nodal specificity: Each node of a subweb also has a specific function within that of the subweb

31. Properties of Cortical StructureApplied to Functional Webs (cont’d) • Property V: Competition • Neighboring nodes are likely to be in competition • Typically they will be in different functional webs • Property VI: Extension of II-V to larger columns • Properties II-V apply also to maxicolumns and hypercolumns • Property VII: Hierarchy in functional webs • A functional web is hierarchically organized • Property VIII: Cardinal nodes • Property IX: Reverberation

32. Property IX: Reverberation in functional webs • Reverberation among connections in an established web strengthens activation • Experimental verification: • The monkey experiment • Temporary lesion in part of the short-term memory web reduces activity in other part • (considered last time) • Comparing words and pseudo-words • Pseudo-words: phonologically OK but no meaning

33. Property IX: Reverberation in functional websWords and pseudo-words (Pulverműller 2002) • Reverberation among connections in an established web strengthens activation • Experimental verification: • Compare words and pseudo-words • Real words show greater activation • “About one-half second after the onset of spoken one-syllable words, high-frequency brain responses were significantly stronger compared to the same interval following pseudo-words.” (Pulverműller: 53)

34. Another word : pseudo-word experiment(Pulverműller 2002: 54-56) • Finnish • pakko ‘compulsion’ • takko : a pseudo word • Same 2nd syllable • Measurement was done on response to -ko • Technique used for measuring: EEG, MEG • Measurements used: MMN and MMNm • MMN : mismatch negativity • MMNm: magnetic flux from mismatch negativity (from MEG) • Subjects were watching a silent movie • I.e., not paying attention

35. Another word : pseudo-word experiment: Results(Pulverműller 2002: 54-56) • Measurements used: MMN and MMNm • (MMN : mismatch negativity) • Larger for –ko of real word • Strongest difference at 100-200 ms after word-recognition point • Word-recognition point: “the earliest point in time when the information present in the acoustic input allows the subject to identify the word with some confidence” • MEG showed that the activation was in left superior temporal lobe

36. Finnish ‘pakko’ experiment: discussion • [-ko] produces activation in either context, since it is a syllable occurring in Finnish • And in the area for phonological recognition • Stronger activation in pakko • pakko is an established word in Finnish • That means it has established connections to meaning • Established connections provide stronger activation • Indicates reverberation – strengthening of the activation from other parts of the web • For the pseudo-word, there are no other parts of a functional web – only phonological information

37. Right Hemisphere in Language Processing(What we are really talking about is non-dominant hemisphere – so it’s LH for RH-dominant people)

38. Major RH Linguistic Functions • Inference, Metaphor • Coarse coding • Music

39. Some findings w.r.t. RH speech perception • Vowel qualities • Intonation • Tones in tone languages

40. Possible bases for RH/LH difference • Higher ratio of white to gray matter in RH • Therefore, higher degree of connectivity in RH • Difference in dendritic branching • Different density of interneurons • Evoked potentials (EEG) are more diffuse over the RH than over LH Beeman 1998: 257

41. Anatomical differences between LH and RH • Geschwind & Levitsky (1968) • 100 brain specimens examined • Planum temporale • Larger in LH: 65% • Larger in RH: 11% • About the same, both sides: 24% • Correlates with shape of Sylvian fissure • Shorter horizontal extent in RH Goodglass 1993:60

42. Some Experiments (described by Beeman 1998) • Words presented to rvf-LH or lvf-RH • RH more active than LH • Synonyms • Co-members of a category: table, bed • Polysemy: FOOT1 – FOOT2 • Metaphorically related connotations • Sustains multiple interpretations • LH about same as RH • Other associations: baby-cradle • LH more active than RH • Choose verb associated with noun

43. Patients with brain-damage • Some patients with LH damage • Can’t name fruits but can say that they are fruits • Patients with RH damage • Impaired comprehension of metaphorical statements • More difficulty producing words from a particular semantic category than producing words beginning with a particular letter (258)

44. Imaging studies • When listening to spoken discourse, cerebral blood flow increases in • Wernicke’s area • Broca’s area • RH homologues of Wernicke’s and Broca’s areas • More cerebral blood flow in RH when subjects read sentences containing metaphors than literal sentences

45. Experiments on speech perception • Dichotic listening – normal subjects • Right ear (i.e. LH) advantage for distinctions of • Voicing • Place of articulation • Left hear (RH) advantage for • Emotional tone of short sentences • Sentences presented in which only intonation could be heard • RH advantage for identifying sentence type – declarative, question , or command

46. Experiments on speech perception • Split brain patients • They hear a consonant • Then written representations are presented • ‘Point to the one you heard’ • rvf-LH exhibited strong advantage

47. Patients with right-brain damage • Posterior RH lesions result in deficits in interpreting emotional tone • Anterior RH lesions abolish the ability to control the production of speech intonation

48. Split-brain studies • Isolated RH has ability to read single words • But not as fast nor as accurate as LH • Ability declines with increasing word length • Lexical context does not assist letter identification • In Japanese subjects • RH is better at reading kanji than kana • Kanji: from Chinese characters • Kana: syllabic writing system • LH is better at reading kana

49. Musical abilities and the hemispheres • Pitch, melody, intensity, harmony, etc. in RH • Rhythm in LH • Absolute pitch (if present) in LH temporal plane • Musicians’ ability to analyze chord structures in LH • Appreciation of chord harmony in RH • Discrimination of local melody cues more in LH • Timbre discrimination in anterior right temporal lobe • Melody recognition in anterior right temporal lobe Evidence from results of brain lesions/surgery, from dichotic listening experiments, from Wada test experiments, and from imaging

50. end