Language in context: Emergent features of word, sentence, and narrative comprehension

1. Language in context: Emergent features of word, sentence, and narrative comprehension Xu, J., Kemeny, S., Park, G., Frattali, C., & Braun, A. 2005 - NeuroImage

2. Research Gap to be Filled… • Context (broadly construed) has widespread effects on neural activity; context in language processing are a natural area of interest • Most previous research has focused only on one level of text processing (e.g., word, sentence, discourse-level) • Work focusing on discourse-level has not examined the role of context within the establishment of a discourse representation

3. Background on discourse-level • “mental models” or “situation models” - mental representation of a described state of affairs. • Situation model NOT equal to a schema • Schemas represent stereotypical situation (e.g., visiting a restaurant) • Situation model is mental representation of specific situation (e.g., a specific visit to a restaurant on a certain day in your memory) • Situation model draws upon prior knowledge, constrained by attentional & memory capacities, and utilization of abstraction, imagery, and inference

4. In the present study… • fMRI methodology • Text presented at word, sentence, and discourse levels • Drawn from Aesop’s fables* • No metalinguistic tasks used (to reduce potential for brain activity from cognitive strategy)

5. Hypothesized brain regions • Across all text levels, LH Perisylvian areas should be active: • Sentence-level: more widespread perisylvian, esp. frontal operculum (covering the insula, aka Island of Reil) • Discourse-level: extra-sylvian activation

6. Hemispheric predictions • RH activity increase w/ increasing context • Both LH & RH activity early in discourse-level processing, RH activity stronger at end, when situation model construction becomes coherent and well-developed

7. Methods • 22 right-handed male english speakers (21-65 years old) with no history of neurological impairment or psychiatric disease* • BOLD contrast fMRIs using a whole-body 3T GE scanner

8. Methods cont… • Aesop’s fables were used (with some edits to make the English more modern) • The Birds, the Beasts, and the Bat THE BIRDS waged war with the Beasts, and each were by turns the conquerors. A Bat, fearing the uncertain issues of the fight, always fought on the side which he felt was the strongest. When peace was proclaimed, his deceitful conduct was apparent to both combatants. Therefore being condemned by each for his treachery, he was driven forth from the light of day, and henceforth concealed himself in dark hiding-places, flying always alone and at night. (He winds up friendless who plays both sides against the middle)

9. Methods cont… • Words in each fable were coded for (1) part of speech, (2) frequency, (3) imageability, (4) concreteness, (5) grammatical category* • Words in each sentence coded for 15 syntactic attributes • Sentences coded for (1) # of words, (2) propositions, (3) verbs, (4) thematic roles

10. Methods cont… • 9 fables selected that were matched on ALL aforementioned features* • Words from 9 fables were presented in three conditions: • Random word • Individual, unconnected sentences • Coherent narrative (full fable) • Comparison baseline controls were random consonant letter strings*

11. Comparisons… • Word/Sentence/Narrative minus Random letter string • Across levels: • Sentence-Word, Narrative-Word, Narrative-Sentence • Within narrative: grammar codes • Setting • Initiating Event • Goal/Attempt • Outcome • Reaction • Combined first two & last two… “to provide sufficient # of volumes per block”*

12. Design • Each of the 4 text levels presented in separate blocks (46s duration) with 3 repeated blocks • 350 ms presentation rate, 50 ms pause • Total of 12 blocks over 9 min, 12 s • Block order randomized

13. Results: Region Activation

14. Word vs. Letters Baseline • LH Perisylvian activation: • Middle Temporal Gyri (MTG, BA 21) • Frontal Operculum (BA 45) • Lateral Premotor cortext & anterior portion of supplementary motor area (pre-SMA, BA 6)

15. Sentence vs. Letters Baseline • Similar LH perisylvian activation, but more widespread than single words: • Higher MTG • Higher frontal operculum • Some ventral operculum activity (perhaps reflective of semantic features?) • Some subcortical activation • Right posterior cerebellum (syntax & semantics?) • Components of motor circuit (putamen & ventral thalamus)

16. Narrative vs. Letters Baseline • Robust perisylvian activation and subcortical regions • Medial prefrontal cortex (BA 8/9/10) • Implicated in ability to intuit others’ thoughts, metaphors • Dorsal TPO junction • Precuneus • Implicated in linking text w/ prior knowledge, accessing of episodic memory • hippocampus, temporal sulcus, amygdala, etc.

17. Results: Contrasts Across Levels Left frontal operculum, Left TPO cortex MTG, TPO cortex Medial prefrontal cortices, precuneus, etc. (extrasylvian areas)

18. Summary of Across-Level Contrasts • Increasing parasylvian activity with increasing language complexity • Increasing bilateral activation with increasing language complexity • Selective perisylvian & extrasylvian activity only with narrative-level

19. Results: Within-Narrative • Strong shift in lateralization (L to R) • Establishment of setting & plot associated with mostly LH • Resolution of narrative associated with RH increases

20. Within-Narrative explanations • LH activation early in narrative likely reflects construction of textbase-level features • Bilateral activation late in narrative likely reflects RH contribution to “global synthesis” of story as a whole (e.g., extraction of a moral from the story)

21. Unusual findings… • Word-level: some activity in supplementary motor cortex areas (despite lack of motor response in task) • Perhaps subarticulation? • Perhaps stronger ties between language & motor function than previously believed • Sentence-level: left putamen/ventral thalamus activity (part of motor circuit) • Perhaps motor circuit plays role in serially organizing syntactic structure?

22. Questions… • If narrative processing leads to combinations of perisylvian & extrasylvian brain areas, would more frequent readers (e.g., those more familiar with certain genres which promote situation models) show different brain activation patterns? • might familiarity (previous knowledge) of Aesop’s fables have contributed in part to the processing load of narratives over time? • what might happen if we asked the participants to create a story in the word and sentence conditions (in which they might think of Aesop’s fables right away as well.)

23. Questions • Any reason for using consonant strings as a baseline condition instead of letting the participants rest? • Why use random consonants as a base? Why not pseudowords with vowels? Does it matter? • Although 350ms is still slower than normal reading speed, it seems to be much faster than most imaging studies run on fMRI. (is it true?) Do they need to compromise anything because of the fast display of stimuli? (e.g. noisier data or more assumption in the data analyses?) • First of all, what are sham blocks and sham contrasts. What do they have to do with vectors?

24. Questions • And is it really possible for 9 different fables to be matched on so many variables? • Why white font on a black background? Does it matter? • For ERPs, I understand that we want to stay with younger participants (20s range). Why the variety in ages?? 21 to 65 yrs old? Can they do that?

25. Questions • Perhaps if subjects were shown a series of pictures that told a story, simulating the narrative condition but without language, it would become clear if the additional activity was caused by linguistic context or something else (like working memory load)? • don't sentence level computations take some of extra-linguistic operatoin as well since people can use semantics in the sentence level too? • I'm just wondering how much does this has to do with the different speed of activation pattern across hemispheres. (LH is faster, and RH is slower). Is it possible that since the RH is slower, the activation get 'accumulated over time and appears to be more prominent at the story's conclusion?