Language, Linguistics and the Brain

1. Language, Linguistics and the Brain Sharon Armon-Lotem From Neuron to Mind Session 5: Language as a window to the mind June 17, 2004

2. How does the brain manipulate and process language? • Language is a system of pairing of forms (often sounds) and meanings which is used to (formulate) and transfer information. • Language manipulation appears to be • a rule-governed activity • involving symbol manipulation • involving the pairing of symbols and 'meaning‘ • How the brain manipulates language involves answering how the brain performs these three functions. • If we can understand how the brain manipulates language, we have a window on to how the hardware is connected to the higher cognitive functions = mind.

3. What is language? Language includes: • a phonological system- which determines which sounds are part of the language and how they are strung together in words. • a set of words assigned to categories –lexicon. • asyntactic system - which determines how words canbe strung together in utterances. • asemantics system – which determines how meaning is assigned to words and strings of words. • apragmatics module– whichdetermines how context affects the interpretation of utterances.

4. Language as a window to the mind There are at least three approaches to language as a window to the mind: • What are the properties of the language that the brain is processing? (Theoretical Linguistics) Susan Rothstein, Yael Greenberg, Gabi Danon, Galit Adam • What are the properties of language in use, which will tell us something about language as it is processed? (Psycholinguistics) Jonathan Fine, Joel Walters, Sharon Armon-Lotem, Elinor Saiegh-Haddad • How does the brain process language?(Psychology and Language) Miriam Faust, Michal Raveh, Elisheva Ben-Artzi, Gil Diesdendruk

5. Theoretical Linguistics • Susan Rothstein specializes in formal semantics and the question of how our use of language reflects basic cognitive capacities; e.g. how the use of count/mass expressions reflects the way in which we individuate and count entities. • Yael Greenberg specializes in formal semantics, and, among other things, how we unconsciously use minimally distinct constructions to express different meanings e.g. rina yafa / rina hi yafa. • Gabi Danon specializes in formal syntax: what is the relation between syntactic representation and the meaning of a linguistic expression. Why are there environments where indefinites but not definites are allowed e.g. ba li glida but not *ba li ha-glida ha-zot? (but: ba li al ha-glida ha-zot is OK) • Galit Adamspecializes in phonological theory and the acquistion of phonological competence. Why are there disparities between child and adult production? Why do children say babuk for bakbuk ‘bottle’, dropping the k at the middle but not in the end?

6. Psycholinguistics • Jonathan Fine works on how language is a window onto the mind by examining the language of speakers with psychiatric disorders such as schizophrenia, autism and hyperactivity. The language reflects and, in fact, partly defines the disorders. Changes in the disorder by medication can then be studied in the language. • Joel Walters studies phenomena representing cognitive processes by examining the special characteristics of bilinguals. Such bilingual speakers switch languages and acquire languages in ways that give a unique view onto processing of language. • Sharon Armon-Lotem specializes in first language acquisition of syntax and semantics in normal and special populations; e.g., how children learn to use the inflectional system, the definite marker, relative clauses and the mass count distinction; and what makes some aspect of the system more penetrable than others. • Elinor Saiegh-Haddad works on language and reading acquisition in bi-dialectal and bilingual contexts. She studies the effect of linguistic distance between the oral language of children and the dominant/written language on acquiring basic linguistic and literacy skills.

7. Psychology and Language • Miriam Faust studies the representation of language in the human brain using behavioral, electrophysiological (ERPs) and brain imaging (fMRI) techniques. A special focus is given to the processing of high level linguistic information by the left and right cerebral hemispheres, language organization in the brain in different clinical populations and word retrieval processes in typically developing and clinical populations • Elisheva Ben-Artzi works on Dyslexia (with Harvey Babkoff) focusing on the processes underlying dyslexia; on false memories for words, studying the nature of false memories for words; and on aging and speech comprehension (with Harvey Babkoff), focusing on the interrelationships between auditory temporal processes and speech comprehension in the elderly. • Gil Diesdendruk focuses on ways in which language and thought relate. He investigates to what extent children rely on their understanding of the mental states of speakers in order to infer the meanings of words, arguing that while children might rely on the species-specific capacity to "mind-read" when acquiring words, the capacity is not language-specific. • Michal Raveh examines word recognition abilities of children and adults, normal and dyslexic to understand how the structure of words affects their recognition. She conducts computerized simulations of neural net model of language processing to develop theories of reading.

8. Looking through the three windows • The semantics of the count-mass distinction: Theory and Acquisition • Language Use and Processing • Brain and Creativity: The Effects of Semantically Convergent and Semantically Divergent Priming on Lexical Processing by the Two Cerebral Hemispheres

9. The semantics of the count-mass distinction: Theory and Acquisition Susan Rothstein and Sharon Armon-Lotem The Department of English and The Gonda Multidisciplinary Brain Research Center Bar-Ilan University

10. Semantic definitions of count and mass (Link 1983) • Mass expressions are homogeneous. Count expressions are not. • Upward homogeneity (cumulativity): wine + wine = wine a dog + a dog = dogs/*a dog • Downward homogeneity: If I split wine into two parts each part is still wine. If I split a dog into two parts, each part is not a dog.

11. Homogeneity is not a linguistic distinction (Rothstein 2004( • Homogeneity is a real-world distinction. • The fact that half a dog is not a dog while half a quantity of wine is wine depends on what counts as a dog or as wine in the world. • So is the count mass distinction formal, or does it reflect the way things are in the world?

12. Count nouns can be homogeneous too (Rothstein 2004) • Fence, wall, hedgeare all homogeneous and count. • A fence can be broken into two fences: • Two fences can be put together into a single fence. • Rothstein 2004: homogeneous count nouns are a systematic part of the language and not odd exceptions (in contradistinction to Krifka 1992, Zucchi and White). • So [±homogeneous] is not at the basis of the count mass distinction.

13. What are children sensitive too? Are children sensitive to • [± homogeneous] = world distinction • [± count] = formal distinction • both

14. Methodology • Syntactic information: plural vs. singular • 20 children, age: 3 -5 + 10 Adults • Methodology: elicitation task with real words and objects. • Three semantic categories, which the two features yield. • Presentation: Mom went shopping. She put an apple in her basket, then another apple, and then another apple. What’s in the basket? What do we have here?

15. When is plural used? Figure 1 – The use of plural forms by different age groups (by percentage)

16. Findings • The Threes like the adults made a binary distinction based on world distinction. • Three of the Fives made a binary distinction based on the formal distinction. • Seven of the Fives followed the world distinction, but used other linguistic means to make the formal distinction, making a three-fold distinction.

17. How did the Five’s make the three-fold distinction? • Half of their responses included classifiers in order to make the formal distinction between ‘sugar’ and ‘fence’: Shalosh peamim xol three times sandshtey kosot mayimtwo cups water • Half of their responses included an adjective indicating the length of the row of objects in order to make the formal distinction between ‘sugar’ and ‘fence’: srox arox Shoelace long sharsheret aruka Chain long

18. Conclusion • The Threes go by the world distinction between things which are homogeneous and things which are not. • The Fives show awareness of the tension between the linguistic indicators and the non-linguistic ones. • The Fives form a syntactic distinction between the three semantic categories, in line with the semantic distinctions.

19. From mental representations to language use and processing • The work just described on the formal semantics of the mass/count distinction and studies investigating the acquisition of that distinction in early child language is an attempt to get at mental representation, one, if not THE, central concern of theoretical linguistics. • This too will be the concern in the presentation of Naama Friedmann on syntactic theory and Broca’s aphasia. • Another line of work in our linguistics group focuses on LANGUAGE USE AND PROCESSING. • By way of example, we tell a tale of two frameworks; one on the language of psychiatric conditions and one on the languages of bilingualism.

20. Using and Processing Language Jonathan Fine and Joel Walters The Department of English and The Gonda Multidisciplinary Brain Research Center Bar-Ilan University

21. Two approaches to language use and processing • Fine starts with the resources for making meaning (from culture and genre, to discourse and syntax) and analyses texts from different kinds of speakers. These techniques of discourse analysis are used by both Fine and Walters. They are illustrated briefly here and elaborated in greater detail in Obler’s talk on Alzheimers patients. • Walters’ information components (identity, context, genre, intentions, formulation, and articulation) bridge the sociopragmatic-psycholinguistic divide. His processing mechanisms - imitation, variation, integration, and control - are grounded in psychology, and as Cziko shows later in his talk on Perceptual Control Theory, they make moreexplicit claims than conventional constructs. • Both approaches aim to capture levels of language processing which characterize special populations

22. Linguistic characteristics in Psychiatric Disorders • Psychiatric disorders are largely assessed through clinical interviews that depend almost entirely on the language of the patient and clinician. In fact, many disorders are defined by how speakers use language. • Attention deficit hyperactivity disorder (ADHD) • butts into conversations • answers questions before they are finished • difficulty playing quietly • Schizophrenia • loose associations • tangential talk • talks too much • Pervasive developmental disorders (PDD/Autism) • pedantic speech • one-sided social interactions • lack of spontaneous seeking to share

23. Methodologies of discourse analysis (ADHD) 1) Overlapping C1: can I go to the washroom? T: um yes you may go if you've got those < puzzles pieces> C2: < Miss he's going to play with his yo-yo> 2) Topic structure on topic(pronouns, lexical repetition, conjunction, ellipsis) T: okay, there you go, so what you want to do is figure out maybe you put the first three on the bottom and then work up. Just put them in the first three <at the top>. C1: <all of them> off topic T: and Paula, you can whole punch that so we'll put this into the detective section, so it was our last one. There you go, so the green section’s okay and it's almost finished. C2: glue glue glue glue. Glue glue. Glue. Glue. Glue. T: so . C1: who likes Pokeman here?

24. Percent of initiations that are off-topic

25. Percent of initiations that are off-topic and also overlap

26. Methodologies of discourse analysis (Bilingualism) Codswitching (1) Do you have חבילות עוגיות שלימות? (whole cookie boxes) (2) Do you know toלעטוף ספרים? (to wrap books) (3) muzi [move + zuzi] (4) noce [no + roce] Codeswitching and pragmatic markers (5) Secretary: What are you doing with our lists. Yo!! ((very agitated)) (6) Student: No, they gave it to me. (7) Secretary: Who gave it to you? (8) Student: AA, eh (.) I was advised by her. I spoke to her. (9) Secretary: AA gave it to you? (10) Student: Of course. (11) Secretary: To take home? (12) Student: Sure. (13) Secretary: This is not-at-all meant for students. It is not allowed to be given to anyone.

27. Summary • Functional linguistic features define psychiatric disorders and describe cognitive processes in bilingualism • Precise linguistic formulations allow psychiatric disorders and different forms of bilingualism to be distinguished and track changes in the course of treatment or development

28. From language use and processing to imaging language in the brain • The understanding of functional linguistic categories in psychiatry and bilingualism broadens our insights about language in use ands offers different methodologies for studying language processing. • The next line of work described hereby takes us into experimental laboratories of psychology and language, in the study of language processing in the hemispheres.

29. Brain and Creativity: The Effects of Semantically Convergent and Semantically Divergent Priming on Lexical Processing by the Two Cerebral Hemispheres Miriam Faust Department of Psychology And The Gonda Multidisciplinary Brain Research Center Bar-Ilan University

30. Comprehension of semantic information by the LH and RH • The LH strongly activates a narrow semantic field, only including the interpretations that are dominant or most relevant to the immediate context. This process is very effective for most straightforward language processing • The RH weakly activates a broad semantic field, including many interpretations or meanings that may seem less relevant. This process is ineffective for rapid interpretation or selection but increases semantic overlap among multiple semantic fields. • This asymmetry can be described as fine semantic coding in the LH and coarse semantic coding in the RH (Beeman, 1994).

31. Right Hemisphere (RH) contribution to language processing Nonselective RH semantics seems particularly well-suited for: • Comprehending words/messages where several levels of meaning must be simultaneously considered • Revising an initial interpretation • Continued activation of discourse level meaning • Processing unusual, nonsalient semantic relations (poetry) • Insight problem solving

32. Insight problem solving and the RH The Mednick Test of Creative Cognition (1962) REPORT, CREDIT, PLAYING - CARD • Performance on insight problems is associated with creative thinking (Schooler & Melcher, 1997) • There are some striking parallels between the properties of insight problem solving and the cognitive characteristics of the RH (Fiore & Schooler, 1998) • The RH may have unique abilities to avoid the inhibitory processes that promote fixedness and that reduce access to nondominant interpretations (Fiore & Schooler, 1998)

33. The present research If the activation of word meanings in the RH is broader, especially sensitive to overlapping from distantly related words, and less differentiated, than the RH should take better advantage of semantically divergent primes Four divided visual field experiments examined the ability of the LH and the RH to process semantically convergent (either dominant or subordinate) versus semantically divergent (combinations of dominant + subordinate) primes for facilitating the recognition of ambiguous target words

34. Lexical decision and semantic judgment(Faust & Lavidor, 2003) Participants: Sixty-four right handed, native English speaking students Stimuli: Targets: 128 ambiguous words (“novel”) 128 nonwords (“nohel”) Primes: 256 words related to the dominant meaning 256 words related to the subordinate meaning 256 unrelated words

35. Four types of word pairs were constructed from the word primes: Target: Novel Unrelated: Apple, Fruit Dominant (convergent): New, Fresh Dominant + Subordinate/Subordinate+ Dominant (divergent): New, Story Subordinate (convergent): Story, Book

36. Procedure Focusing signal X Primes (500 ms) Focusing signal (300 ms + 150 ms) X Target word Target word (150 ms) (150 ms)

38. Findings - Lexical decision The overall pattern of priming indicates that the combinatorial influence of multiple related primes is different within each hemisphere RVF/LH • Large benefit from dominant primes • No benefit from subordinate primes • Facilitation from divergent primes is smaller than the facilitation from dominant primes • Facilitation from 2 dominant primes is significantly larger than from one dominant prime LVF/RH • No benefit from either dominant or subordinate convergent primes • Only divergent primes facilitate target word recognition

40. Findings - semantic judgment The overall pattern of priming indicates that the combinatorial influence of multiple related primes is different within each hemisphere RVF/LH • Large facilitation from both dominant and subordinate primes • No facilitation from divergent primes (underadditive effect, inhibition) LVF/RH • Facilitation from all types of primes (largest for divergent primes) • Absolute LVF/RH advantage for divergent primes (65 ms)

41. Conclusions • The findings point to an important implication of RH propensity to activate and maintain many potential word meanings: The ability to converge on a target word from different, including incompatible, word meanings without being disrupted by seemingly semantic conflicts • RH ability to utilize two different incompatible concepts may serve specific linguistic functions • Maintaining simultaneous activation for alternate word meanings could be essential in cases where the ability to recognize alternative interpretations of problem elements represents a critical component, e.g., in creative, insight problem solving

42. Concluding remarks • The three presentations focused on different windows to meaning construction in the mind: • Rothstein and Armon-Lotem show how semantic categories are built on the interaction between formal and 'real world' criteria. • Fine and Walters focus on special populations in an attempt to show how we construct meanings from rich context • Faust shows how our two hemispheres process and access meanings, highlighting the role of the RH in these processes.

43. These different approaches to language research complement each other and lead to an understanding of what language is and what the mind must be in order to use language. Thank you!