Language Use and Understanding

Language Use and Understanding BCS 261 LIN 241 PSY 261 CLASS 6: EFFECTS OF DISFLUENCY ON REFERENCE COMPREHENSION

What are the cognitive processes of reference comprehension? …the apple… …thee uh apple…

Everyone is disfluent ok, give the gray squirrel a blue umbrella • About 6% of speech is disfluent • Um, uh • Repeats, repairs • “thee” for “the” • Pauses • Elongations • Pitch contours ok, take, ok, give, give the bl- the gray squirrel a blue umbrella

Most comprehension research ignores disfluency • Laboratory speech • Sanitized • Scripted • Fluent ok, give the gray squirrel a blue umbrella • Disfluency not “core” language • Discussion Question: Why not? (MR) • At best: irrelevant • At worst: a source of processing difficulty

Why study disfluency? • Disfluency is ubiquitous • Disfluency is systematic • It therefore provides a unique window onto language processing • Does expectancy affect reference comprehension?

How does disfluency affect language comprehension?

How does disfluency affect language comprehension? • Me: … the battery’s really low. Can you hand me thee um - • Dana: yeah

Language is ambiguous • Point to that animal. • Point to it. • Point to the camel… with a red bow. • Point to the camel.

Can listeners use expectancy to facilitate language comprehension, given temporary indeterminacies? • How do listeners integrate information from various sources? • Lexical information • Discourse context

Many information sources can facilitate reference resolution • Each piece of information can be • PARTIAL • PROBABILISTIC • How might this make the comprehension system more efficient?

Disfluency is systematic • Disfluency indicates production difficulty • “thee” likely to be followed by: • Pause • Repeat • Um / Uh Theee, um, ... (Clark & Wasow, 1998, Fox Tree & Clark, 1997)

Given/New fundamental for language • GIVEN: “…. the ball. Give me the ball.” • NEW: “…. the ball. Give me the bat.” • Language structure codifies information structure • Given before new in production(e.g., Arnold, Wasow, Losongco, & Ginstrom, 2000) • Given more accessible in comprehension • Pronouns (e.g., he, she, it) • Definite noun phrases (e.g., the ball, the bat) (e.g., Chafe, 1976; Clark & Sengul, 1979; Dahan et al., 2002)

Disfluency more likely for NPs with new referents • GIVEN: “…. the ball. Give me the ball.” • NEW: “…. the ball. Give me thee uh bat.” (Arnold & Tanenhaus, to appear; data from Arnold, Wasow, Losongco & Ginstrom, 2000) Does disfluent speech make new objects more expected?

Discussion Q • Is there a possibility that, as adults, we are naturally predisposed toward disfluent speech in order to allow our children the opportunity to realize that we are providing new information (prime them, such as was mentioned in the article), and to give them a small amount of extra time to construct a memory for the newly acquired information? (Jessee Blake)

EXPERIMENT 1Does disfluency lead comprehenders to expect reference to a new object? “Put the grapes below the camel. Now put the-” Off-line task: What is the speaker likely to mention next?

Context: Put the grapes below the camel. Short instruction Long instruction FluentNow put -- Now put the-- DisfluentNow put -- Now put thee uh --

Exp. 1 Results • Disfluency makes reference to new objects more expected • Does disfluency also affect the on-line processes of reference comprehension?

EXPERIMENT 2Does disfluency create an on-line bias toward new objects? Put the grapes below the camel.Now put theee, uh candle above the salt shaker.

Trials 1 2 3 4 5 Time 200 ms Percentage of looks Time Percentage of Looks over Time Target = camel Competitor = candle Unrelated = salt shaker grapes Click on the camel.

GIVEN target NEW target Given vs. New cohorts • Put the grapes above the camel. • Now put the CA…. (Dahan, et al. 2002)

Disfluent- new bias? • Put the grapes above the camel. • Now put thee, uh CA…. Fluent- given bias • Put the grapes above the camel. • Now put the CA…. (Dahan, et al., 2002)

Comprehension Study Methods • Instructions recorded by experimenter • Subjects told they were recorded by another subject • Disfluencies sounded natural • Lexical retrieval was a plausible source of disfluency

Experiment 2 design & predictions FLUENT / GIVEN Put the grapes above the camel. Now put the CAMEL . . . FLUENT / NEW Put the grapes above the candle. Now put the CAMEL . . . DISFLUENT / GIVEN Put the grapes above the camel. Now put thee uh CAMEL . . . DISFLUENT / NEW Put the grapes above the candle. Now put thee uh CAMEL . . .

camel… candle… camel… candle… Context: “Put the grapes above the camel.” New cohort Given cohort unrelated

Context: “Put the grapes above the camel.” New cohort Given cohort unrelated

Discussion Question • Is it not possible that the disfluencies gave the participants more time to think and therefore let their eyes wander to the new objects whereas in the fluent case they had no time for such meandering? (Nicole Dobrowolski) • Is the difference in difference between discourse-old and discourse-new stimuli in this experiment basically related to the idea of frequency in lexical items? (i.e. candle 'appears' more in common English than camel) (Anthony Shook)

Exp. 2 conclusions • Disfluency affects reference comprehension • 200 msec after onset of target • Combines with lexical information • Disfluency eliminates/reverses given bias

Why does disfluency create a bias toward new objects? • New objects are harder to refer to Theee, um, ... Aha! This speaker is having trouble!

Does disfluency create biases to other types of referents? Known objects Novel objects

EXPERIMENT 3Does disfluency cause a novel bias on-line? Click on the red … Click on thee uh red…

EXPERIMENT 3 DESIGN • FLUENT: Click on the red … • KNOWN … ice cream cone. • NOVEL … funny squiggly shape… • DISFLUENT: Click on thee uh red … • KNOWN … ice cream cone. • NOVEL … funny squiggly shape…

unrelated looks known looks novel looks

Exp. 3: Competitor looks 200-600 msec after onset of color word

Exps. 2-3 summary • Fluency information affects on-line reference comprehension • Disfluency introduces biases • NEW objects • NOVEL objects

Discussion Questions • Do these findings possibly happen because of our ability to use top-down processing and our previous experience with everyday speech through speaking and listening? (Jessica DeSisto)

What would happen if the disfluency occurred within a different part of speech, such as the noun or a verb in a sentence? Are these effects of disfluency on language comprehension and the expectancy hypothesis only observed when a definite article is disfluent? (Beth Riiina) • What would be the effect on resolution of longer disfluencies, according the the results of this paper? (e.g. my friend uses "whats its face" whenever searching for a word). Would the paper even have anything to say about it, or does the sheer length of certain phrases influence the resolution too much? (Anthony Shook)

I realize that the article summarized the finding that disfluent speech leads us to new information, rather than given-is it that the disfluent speech allows for a moment to search your memory for the information that is being presented, and if you do not have a memory for it, you then build one? Or does the subject, when presented with disfluent speech, automatically look to the new information without hesitation? (Jessee Blake)

Does anyone feel that these findings could be useful in programming a computer to comprehend everyday speech? Would these findings aide in our own understanding of language comprehension, increasing our ability to program a computer to understand human speech errors? (Jessica DeSisto)

Language Use and Understanding