Linguistics 001, Spring 2010

1. Linguistics 001, Spring 2010 Neuroimaging: Syntax

2. Syntax in the brain • A standard view is that ‘syntax’ resides in Broca’s area • This finding connects in some ways with what is found in the aphasia literature, although the situation is quite complicated • The reading moves beyond this basic picture in many ways by looking at imaging of syntax • Today: Introduce functional Magnetic Resonance Imaging (fMRI), and look at a few experiments that involve syntax

3. Spatial and Temporal Dimensions Imaging techniques differ in spatial and temporal sensitivity. Technique Spatial Temporal ERP 10-20mm 1msec MEG 5-10mm 1msec PET <5mm 30sec fMRI <2mm ~1sec

4. Basics of fMRI • Uses extremely strong magnetic field • Measures changes in local blood flow/blood oxygenation • Excellent spatial resolution • Hemodynamic (= blood-related) response is slow in comparison with neuronal activation.

5. Procedure • Subjects lie within the bore of the magnet, with heads fixed • Functional images, showing changes in blood flow/oxygenation, show “activation” in different areas of the subject’s brain

6. Further Details • During changes in neuronal activity, there are local changes in the amount of oxygen in the tissue • The changes in blood oxygenation are detected by the scanner • The resulting changes are an indirect measure of neuronal activity (indirect because the hemodynamics are related to the firing)

7. Cortical Activity and Blood-Flow Cortical activation results in a local increase in oxy- genated blood, and blood flow, without an increase in oxygen consumption.

8. Illustration Areas in which neuronal activity occurs show patterns of activation which the scanner detects. The image is paired with a graph showing the blood flow in that region by experimental condition (moving vs. stationary dots in this case).

9. Plan • We’ll look at two early studies that use fMRI to study aspects of syntax • While these studies have been followed by many others, they give an idea of the kinds of questions that are addressed in the functional neuroimaging of language

10. Study 1 • Just, Carpenter, Keller, Eddy, and Thulborn (1996), Science 274 • Cortical activation and sentential complexity

11. Basics • Question posed: Do computationally more demanding sentences result in more brain activation than less complex sentences? • Focusing on areas known to be involved in language, i.e. Wernicke’s and Broca’s

12. Stimuli • Three types of sentences, with the same number of words: • Active:The reporter attacked the senator and admitted the error. • Subject relative:The reporter that t attacked the senator admitted the error. • Object Relative:The reporter that the senator attacked t admitted the error.

13. Control • Something to control for some of the visual components of the sentence-reading part • Consonant strings: • Pws ntkgqrfm zjkjrng kwtdc sbfght swn mrjbxq kgt mxbtq

14. Idea • The three different types of sentences have been studied extensively in the processing literature • They increase in difficulty • The reason for this appears to have to do with the position of the trace, when there is one

15. Design and Task • Sentences were arranged in sets of four to five of the same type • Following each sentence, subjects answered ‘True’ or ‘False’ to a comprehension probe: • “The reporter attacked the senator, True or False?”

16. Subject Performance • Reaction time and error rate data were collected for most of the subjects • Analysis of the behavioral data showed these indices of complexity to increase monotonically in the expected way with the increasingly complex sentence-types

17. Images The analysis looked at the number of activated voxels by Condition; more of these are present from left to right. NOTE: Left is Right and Right is Left in these images

18. Cortical Activation

19. Interpretation • It is suggested that Wernicke’s is involved in something other than purely lexical matters (since these are constant across conditions) • Broca’s area is involved in something as well (perhaps syntax; it’s hard to tell because it could be “complexity”) • Right homologues are implicated in processing as well • Overall: The brain’s response to increased processing demands is to recruit more tissue in each area in a network of cortical areas

20. Study 2 • Dapretto and Bookheimer (1999), Neuron 24 • fMRI study, attempting to dissociate syntax and semantics in sentence comprehension

21. Motivation • A number of prior studies showing activation in Broca’s area during tasks of different syntactic complexity • The idea that this does not establish anything about exclusivity– perhaps Broca’s area would also be modulated by increasing demands along another, non-syntactic dimension

22. Design Basics • Auditory presentation of pairs of sentences • Subjects had to decide whether or not the meaning of the two sentences differed • Sentence pairs were classified as semantic or syntactic; in each case, there were ‘same’ and ‘different’ types

23. Conditions • SEMANTIC: Pairs of identical sentences, in which one word was replaced either by a synonym or a different word. • SYNTACTIC: Sentences in the pair were in different voices (active vs. passive) or had word-order differences

24. Examples: Semantics • Same • The lawyer questioned the witness. • The attorney questioned the witness. • Different • The man was attacked by the doberman. • The man was attacked by the pitbull.

25. Examples: Syntax • Same • The policeman arrested the thief. • The thief was arrested by the policeman. • Different • The teacher was outsmarted by the student. • The teacher outsmarted the student.

26. Justification of Stimuli • In the semantic condition: Judgment is supposed to rely on single word meanings • In the syntactic condition: Judgment requires computation and comparison of two syntactic structures • Authors note that both kinds of processing are present in each task, and expect the two conditions to differ in relative ‘weight’ of syntax and semantics

27. Images Syntax vs. Rest Semantics vs. Rest

28. Results (General) • Many areas associated with language are found to be active in this experiment • What about the syntax vs. semantics comparison?

29. Syntax vs. Semantics Syntax (A): Task-specific activity centered in the (lower) pars opercularis (BA 44). Semantics (B): Task-specific activity in the pars orbitalis (BA 47).

30. Brodmann Areas

31. Interpretation • Authors • Syntax: BA 44 is particularly involved in syntactic processing • Semantics: BA 47 is selectively involved in the processing of lexico-semantic information • Spend some time thinking about how strong this kind of conclusion could be, given what happened in the experiment etc. • I.e., the conditions both involve syntax and semantics; moreover, the tasks are not thoroughly understood; moreover, the behavioral data are not reliable, etc.

32. Wrapping up • Broca’s area (and other language areas) can be modulated in different tasks • How this connects with the idea of specialization is complicated, as discussed in the reading • Functional neuroimaging is impressive technologically, but the connections with cognitive hypotheses are only starting to become clear

33. Space/Time/Brain • Prospects for future theories of linguistic computation in the brain • Integration of spatial and temporal information • Correlation between linguistic ontology and neurological ontology • Bringing the neurological theories “up to speed” with the categories from the representational analyses of language

34. Linguistics/Neurolinguistics • If you want to know about the analysis of sentences/sounds/words, etc., you consult linguistic theory • If you want to know about structures in the brain, cells relevant to brain activity, etc., you consult neurology. • What role is there for Neurolinguistics of the type that we have been studying? What are this area’s results, and prospects?

35. Results • Clinically, results showing that (parts of) language in the broad sense are correlated with certain brain areas are useful • The next step is to ask two related further questions, and see what has to be done to answer them: • What can looking at the brain tell us about the complex internal structure of language? • What can looking at language tell us about the complex internal structure of the brain?

36. Specialization Question • Specialization Question: What does it mean for a specific brain region to be specialized for some (linguistic) computation? • I.e., we have seen many descriptions of the different areas of cortex in the brain • Brodmann areas:

37. Specialization, Cont. • To answer the specialization question, we need to know • Why some areas compute some things, and not others • What this means neurobiologically • How ‘flexible’ the correlations are • Then we can ask if there is any explanatory relationship between claims like “syntax happens in region X because this region has certain properties etc..”

38. Granularity Question • Linguistic theory and Neurolinguistic theory operate at different levels of granularity • I.e. we had lots of detailed analyses in the discussion of phonology, morphology, etc. • For a variety of reasons, many neuroimaging studies ask questions about “syntax”, or “semantics”; extremely coarse by comparison (remember that our intro lectures subdivided “syntax” and “semantics”)

39. Granularity, Cont. • When we talk about e.g. syntax, we know computationally that many things are going on; this is not a single “task”: • Creation of tree structures • Linearization of tree structures • Movement of constituents • Etc. • In order to bring neurolinguistics into line with linguistic theory, the brain has to be investigated in terms of the linguistic categories that seem to be important

40. Conclusion • The future of neurolinguistics depends on the development of unifying hypotheses that crosses these domains • The way we’ve been looking at this involves integrating linguistics and neurolinguistics by using linguistic categories to explore the brain • There’s not much “back and forth” at present