PSY 369: Psycholinguistics

PSY 369: Psycholinguistics Language Comprehension: Compositional meaning

Embodiment in language • Embodied Representations • Much of this work argues that language is embodied (e.g., Barsalou, 2008; Glenberg, 2008; Zwaan & Taylor, 2006) • Perceptual and motor systems play a central role in language production and comprehension (and meaning/concepts) • Words and sentences are usually grounded to perceptual, motoric, and emotional experiences. • In absence of immediate sensory-motor referents, words and sentences refer to mental models or simulations of experience

Embodiment in language • Embodied Representations • Simulation hypothesis (Gallese, 2008) • Simulation exploits some of the same neural structures activated during performance, perception, imagining, memory… • Language gives us enough information to simulate • Processing (producing or comprehending) walk involves the use of representations involved in the act of walking producing or comprehending “walk”

Embodiment in language • Evidence for Embodied Representations • Stanfied & Zwaan (2001) • Presented participants with sentences • John put the pencil in the cup. • John put the pencil in the drawer • See a picture and ask “does this describe what you read about?” • Results: faster at saying horizontal pencil with drawer and vertical pencil with cup

Embodiment in language • Evidence for Embodied Representations • Zwaan et al (2004) • Presented participants with a sentence • A: The pitcher hurled the softball at you. • B: You hurled the softball at the pitcher. • See two pictures and ask “are these pictures the same object” B A • Results: faster at saying ‘Yes’ when sentence matched the pictures (e.g., sentence A and pictures in A, if the ball is small and then gets big, it is coming towards you)

Embodiment in language • Evidence for Embodied Representations • Hauk et al (2004) • Do action words activate the motor cortex? fMRI study • 50 words from 3 semantic subcategories • (words matched for freq, length, imageability, etc.) • Rated for whether words reminded them of face, arm, or leg • Movement Comparison: moved their foot, finger, or tongue

Embodiment in language • Evidence for Embodied Representations • Hauk et al (2004) • Do action words activate the motor cortex? fMRI study • Action words did activate some of the same areas as the movements

Traditional Cognition = Computation Representation by propositions Propositions are abstract relations Embodiment of Meaning Cognition is serving perception and actions Representation = Patterns of possible bodily interactions with the world (lawfully related to the world) What an object, event, sentence means for you, is what you can do with the object, event, sentence. Summing up

Meaning beyond the word • Move to compositional semantics • Not all meaning resides at the level of the individual words. • Conceptual combinations • Meaning of Sentences • Meaning across multiple sentences (texts & discourse)

Discourse Psycholinguistics • Traditional Psycholinguistics • Determining what happens when we understand sentences • Broader View • How we resolve/understand sentences against the current discourse representation • Sentence comprehension is a process that anchors the interpretation of the sentence to the representation of the prior text

Processing Discourse • What is discourse? • The ways that we process (i.e., comprehend and remember) units of language larger than a sentence • Lectures, personal narratives, expository discourse • Units of analysis larger than a sentence • Applies to both spoken and written forms Discourse processing is sort of like syntactic processing – a way of organizing/connecting the different pieces in to larger chunks. Here the chunks are larger than sentences.

Bill and Ted traveled through time and space. Bill asked, “Dude, did you find him?” “Yeah, Bach was in the music store, but I haven’t found Abe yet. Dude, you should hear him play, he rocks!” “Excellent! Man, we’ve got to get these dudes back to school before we get there.”

Bill and Ted traveled through time and space. Bill asked, “Dude, did you find him?” “Yeah, Bach was in the music store, but I haven’t found Abe yet. Dude, you should hear him play, he rocks!” “Excellent! Man, we’ve got to get these dudes back to school before we get there.” • To whom does “him” refer?

Bill and Ted traveled through time and space. Bill asked, “Dude, did you find him?” “Yeah, Bach was in the music store, but I haven’t found Abe yet. Dude, you should hear him play, he rocks!” “Excellent! Man, we’ve got to get these dudes back to school before we get there.” • To whom does “him” refer? Bach

Bill and Ted traveled through time and space. Bill asked, “Dude, did you find him?” “Yeah, Bach was in the music store, but I haven’t found Abe yet. Dude, you should hear him play, he rocks!” “Excellent! Man, we’ve got to get these dudes back to school before we get there.” • To whom does this “him” refer?

Bill and Ted traveled through time and space. Bill asked, “Dude, did you find him?” “Yeah, Bach was in the music store, but I haven’t found Abe yet. Dude, you should hear him play, he rocks!” “Excellent! Man, we’ve got to get these dudes back to school before we get there.” • To whom does this “him” refer? Bach again

Bill and Ted traveled through time and space. Bill asked, “Dude, did you find him?” “Yeah, Bach was in the music store, but I haven’t found Abe yet. Dude, you should hear him play, he rocks!” “Excellent! Man, we’ve got to get these dudes back to school before we get there.” • To whom does this “him” refer? Bachagain Why not Abe?

Bill and Ted traveled through time and space. Bill asked, “Dude, did you find him?” “Yeah, Bach was in the music store, but I haven’t found Abe yet. Dude, you should hear him play, he rocks!” “Excellent! Man, we’ve got to get these dudes back to school before we get there.” • Huh!?

Bill and Ted traveled through time and space. Bill asked, “Dude, did you find him?” “Yeah, Bach was in the music store, but I haven’t found Abe yet. Dude, you should hear him play, he rocks!” “Excellent! Man, we’ve got to get these dudes back to school before we get there.” • Huh!? Oh yeah, they’re time travelers.

Characteristics of Discourse • Local Structure (microstructure): • The relationship between individual sentences • Cohesion • Coherence • Global Structure (macrostructure): • The relationship between the sentences and our knowledge of the world

Characteristics of Discourse • Local Structure (microstructure): • The relationship between individual sentences • Cohesion • Does the discourse “stick together”? • Interpretation of one sentence depends on other sentences? • Coherence • Does the passage make sense? • Logical consistency and semantic continuity?

Characteristics of Discourse • Cohesion: Interpretation of one sentence depends on other sentences • Referential Cohesion • “Dude, you should hear him play…” • Substitution Cohesion • “We’ve got to get these dudes back to …” • And many more • Ellipsis, conjunction, lexical cohesion (See pg 362 of textbook for examples) • The relationship between the referring expression and the antecedentcreate referential cohesion of discourse

Types of Referential Cohesion • Anaphoric Reference • Using an expression to refer back to something previously mentioned in discourse “…Bach was in the music store …” “Dude, you should hear him play, he rocks.” • Cataphoric Reference • Using an expression to refer forward to something that is coming up in discourse Dude, did you find him?” “Yeah, Bach was in the music store...”

Comprehending Anaphoric References Daneman and Carpenter (1980) • Task: Reading a passage and answer questions about the referents of pronouns Sitting with Richie, Archie, Walter and the rest of my gang in the Grill yesterday, I began to feel uneasy. Robbie had put a dime in the juke box. It was blaring one of the latest “Rock and Roll” favorites. I was studying, in horror, the reactions of my friends to the music. I was especially perturbed by the expression on my best friend’s face. Wayne looked intense and was pounding the table furiously to the beat. Now, I like most of the things other teenage boys like. I like girls with soft blonde hair, girls with dark curly hair, in fact all girls. I like milkshakes, football games and beach parties. I like denim jeans, fancy T-shirts and sneakers. It is not that I dislike rock music but I think it is supposed to be fun and not taken too seriously. And here he was, “all shook up” and serious over the crazy music. Question: Who was “all shook up” and serious over the music?

Comprehending Anaphoric References Daneman and Carpenter (1980) • Reading Span Test • Smaller reading spans = smaller working memory capacity • Manipulated how many sentences intervened between the pronoun ‘he’ and the antecedent ‘Wayne’ • Task: Reading a passage and answer questions about the referents of pronouns Sitting with Richie, Archie, Walter and the rest of my gang in the Grill yesterday, I began to feel uneasy. Robbie had put a dime in the juke box. It was blaring one of the latest “Rock and Roll” favorites. I was studying, in horror, the reactions of my friends to the music. I was especially perturbed by the expression on my best friend’s face. Wayne looked intense and was pounding the table furiously to the beat. Now, I like most of the things other teenage boys like. I like girls with soft blonde hair, girls with dark curly hair, in fact all girls. I like milkshakes, football games and beach parties. I like denim jeans, fancy T-shirts and sneakers. It is not that I dislike rock music but I think it is supposed to be fun and not taken too seriously. And here he was, “all shook up” and serious over the crazy music. Question: Who was “all shook up” and serious over the music?

Comprehending Anaphoric References Daneman and Carpenter (1980) • Results • Conclusions: The number of intervening sentences don’t matter for high span people, but does for low span

Comprehending Anaphoric References Arnold, Eisenband, Brown-Schmidt, and Trueswell (2000) • Measured where/what participants looked at while listening to the sentence. • Compared two types of cues: • Gender • Accessibility (first mentioned things highly accessible) • Results • Both Gender and accessibility were used to resolve the referential ambiguity “Donald is bringing some mail to Mickey (or Minnie) while a violent storm is beginning. He’s (or She’s) carrying an umbrella, and it looks like they’re both going to need it.”

Characteristics of Discourse • Local Structure (microstructure): • The relationship between individual sentences • Cohesion • Does the discourse “stick together”? • Interpretation of one sentence depends on other sentences? • Coherence • Does the passage make sense? • Logical consistency and semantic continuity?

Characteristics of Discourse • Coherence: • Given/new distinction • Readers expect speakers to provide cues as to what information is old (already known by the listener) and what is new (not known) • Making Inferences • Filling in missing pieces of information to maintain coherence • Haviland and Clark (1974) • Singer, Halldorson, Lear, & Andrusiak (1992)

Developing coherence Haviland and Clark (1974) • Process of understanding a sentence in discourse context involves 3 stages: • Identify the given and new info in the current sentence • Find an antecedent in memory for the given information • Attach the new information to this spot in memory

Developing coherence Haviland and Clark (1974) • Task: Press a button when you understand the sentence. • We got some beer out of the trunk. The beer was warm.

Developing coherence Haviland and Clark (1974) • Task: Press a button when you understand the sentence. • We got some beer out of the trunk. The beer was warm. Definite article “The” signals that “The beer” is given information

Developing coherence Haviland and Clark (1974) • Task: Press a button when you understand the sentence. • We got some beer out of the trunk. The beer was warm. Definite article “The” signals that “The beer” is given information Connect the new information “was warm” to the appropriate discourse concept “some beer” This process is called Direct Matching

Developing coherence Haviland and Clark (1974) World knowledge • Task: Press a button when you understand the sentence. • We checked the picnic supplies. The beer was warm. Definite article “The” signals that “The beer” is given information Connect the new information “was warm” to the appropriate discourse concept “??”

Developing coherence Haviland and Clark (1974) World knowledge • Task: Press a button when you understand the sentence. • We checked the picnic supplies. The beer was warm. Definite article “The” signals that “The beer” is given information Connect the new information “was warm” to the appropriate discourse concept “picnic supplies” Need a bridging inference to connect “the warm beer” to “picnic supplies”

Developing coherence Haviland and Clark (1974) • Task: Press a button when you understand the sentence. • Conclusion: If you don’t know the old information and need to make an inference, this may slow down comprehension. Typical results • Direct Matching • We got some beer out of the trunk. The beer was warm. Comprehended faster • Bridging Inference • We checked the picnic supplies. The beer was warm. Takes more time World knowledge

Developing coherence “Murray poured water on the fire.” “The fire went out.” Singer, Halldorson, Lear, & Andrusiak (1992) • Task: Press a button when you understand the sentence, if given a question, answer Yes or No. • Conclusions: • Suggests that the bridging inference was made • More time consuming to make coherence of temporal than causal relations Results Causal condition Requires bridging inference Faster reading time T/F“water extinguishes fire” Faster “T” Temporal condition No required inference “Murray drank a glass of water.” “The fire went out.” T/F“Does water extinguish fire?”

Brief summary • Local Structure (microstructure): • Discourse is coherent if its elements are easily related. • Coherence is achieved with cohesive ties between sentences. • Comprehension is impeded when • There are no antecedents, forcing a bridging inference • The antecedent was not recent, forcing a reinstatement of the antecedent.

Characteristics of Discourse • Local Structure (microstructure): • The relationship between individual sentences • Coherence • Cohesion • Global Structure (macrostructure): • The relationship between the sentences and our knowledge of the world

Characteristics of Discourse • Global Structure (macrostructure): • The relationship between the sentences and our knowledge of the world Jill bought a new sweater. Sweaters are sometimes made of wool. Wool production gives some farmers a good livelihood. Farming is a high-risk business. On the news last night, I saw a group of business executives discussing recent trends in the stock market. • Okay local structure, but each sentence isn’t relevant to an overall topic of discourse

Characteristics of Discourse • Global Structure (macrostructure): • Schemas (Scripts) • General knowledge structures for common social situations • Genres • Narrative structure • Story grammars - extension of idea of grammatical rules, specify the organization of a story • Expository structure • Different structures

Effects of world knowledge If the balloons popped, the sound would not be able to carry since everything would be too far away from the correct floor. A closed window would also prevent the sound from carrying since most buildings tend to be well insulated. Since the whole operation depends on a steady flow of electricity, a break in the middle of the wire would also cause problems. Of course the fellow could shout, but the human voice is not loud enough to carry that far. An additional problem is that a string could break on the instrument. Then there could be no accompaniment to the message. It is clear that the best situation would involve less distance. Then there would be fewer potential problems. With face to face contact, the least number of things could go wrong. • Bransford & Johnson (1972)

Effects of world knowledge Rocky slowly got up from the mat, planning his escape. He hesitated a moment and thought. Things were not going well. What bothered him most was being held, especially since the charges against him had been weak. He considered his present situation. The lock that held him was strong but he thought he could break it. He knew, however, that his timing would have to be perfect. Prison escape OR Wrestling match • Anderson et al (1977)

Effects of world knowledge • Schemas (Scripts) • Mental structures of how the world works, acquired through experience • A whole package of information about what we know about the world and events • Generic story of situations • A framework with causal information • Used to facilitate comprehension of discourse, as well as to guide recall (and reconstruction)

Restaurant Script Scene 4: Pay Scene 1: Enter Scene 2: Order Scene 3: Eat Go inside Go to table Sit down Get menu Read menu Choose food Give order Get food Eat food Ask for check Received check Tip waiter Pay check Exit Effects of world knowledge • Schemas (Scripts) • Generic story of situations

Effects of world knowledge Bartlett (1932) • Task: • Read native American folk tale • Write down everything that you can remember from that story that I read earlier • Bartlett had them recall after a longer periods of time (between 15 mins. Up to 10 years later)

Effects of world knowledge Bartlett (1932) • Results: • Participants’ memories changed to fit their existing beliefs (reconstructive memories) • Added new details • Changed details • Deleted details • Conclusions: We use our Schema to facilitate comprehension of discourse, as well as to guide recall (and reconstruction)

PSY 369: Psycholinguistics