550 likes | 669 Views
How Speakers' Eye Movements Reflect Spoken Language Generation. Zenzi M. Griffin Department of Psychology. Intro to Cognitive Science University of Texas at Austin October 14, 2011. ?. M E S S A G E. TIME. “ blah blah blah… ”. What the eyes tell us about speaking:.
E N D
How Speakers' Eye Movements Reflect Spoken Language Generation Zenzi M. Griffin Department of Psychology Intro to Cognitive Science University of Texas at Austin October 14, 2011
? M E S S A G E TIME “blah blah blah…”
What the eyes tell us about speaking: • when speakers prepare names for objects • which object they intend to name • possibly when they commit to an order of mention or syntactic structure
green current fixation blue earlier fixation size fixation duration Movie Legend
Speakers gazed at referents in the second before naming them The girl is kicking the boy Griffin & Bock (2000) Psych Science
Long gazes followed order of mention • Not left-right • or salient-less salient • or agent-patient • or big-small • or human-nonhuman
Difficulty in selecting a name varies with number & strength of competing candidates Word selection "TV" or "television" "baby"
FrequencyLow High Codability Medium High
SE Isolated object naming • Latencies from a large norming study (Griffin & Huitema, 1999) • Huge, additive effects of codability & frequency.
SE Object categorization task • Is it bigger than a piece of paper? • Codability effect ns • Frequency effect ns • Interaction ns NO YES
“The clock and the TV are above the needle” Griffin (2001) Cognition
“The|clockand theTVare above theneedle.” Griffin (2001) Cognition
Gaze duration correlated with name difficulty Griffin (2001) Cognition
Gaze duration correlated with name difficulty Griffin (2001) Cognition
1st noun Gaze time on B Freq: Codability: MEDIUM HIGH Effects of preparing 2nd name only during speech, not before See also work by Antje Meyer & colleagues on effects of word frequency, phonological priming, image degradation, etc. Griffin (2001) Cognition
Prepare then speak Until ready to describe The crib & the limo are above the needle time
Gaze reflects difficulty when preparing words in advance Disfluent, Multiple names Fluent, Multiple names Fluent, One name Griffin (in prep)
Gaze time on B Freq: Codability: MEDIUM HIGH MEDIUM HIGH Extemporaneous vs. Prepared speech 1st noun Griffin (in prep)
Not much looking at locations while articulating prepared speech “The A and the B are above the C”
Gazes Reflect Word Preparation • When describing scenes, speakers gaze at objects just before naming them. • Time spent gazing at an object reflects how difficult it is to prepare a name. • Gazes reflect when words are prepared, not articulated.
Gazing at intended object could facilitate name retrieval leaf
Gazing could prevent interference from other objects uh leaf
No eye movements Fixation point
"finger" for "thumb" (half speed then real time)
Timing of gaze: before error = before correct Griffin (2004) Psych Science
Speakers gazed longer at agents before calling them by inaccurate, similar names even when only fluent trials considered Griffin & Oppenheimer (2006) JEPLMC
Timing of gaze offsets was similar Griffin & Oppenheimer (2006) JEPLMC
Speakers even gazed at referent when a better match to name was visible (real time) Griffin & Oppenheimer (2006) JEPLMC
Speakers also gazed at referent when using a novel word "blick" (half speed then real time) Griffin & Oppenheimer (2006) JEPLMC
Name-related eye movements aren't necessary & do not seem to facilitate name preparation • Speakers can describe simple scenes without moving their eyes. • Gazes before speech errors are similar to those before correct names. • Speakers gaze at referents before lying about them.
How early must speakers commit to order-of-mention or syntactic structure?
“The|womanis throwing adogabone.” Referent for
Order of mention/ structure choice a bone to a dog." "A woman throws theme-recipient a doga bone." recipient-theme
Director/Confederate Matcher/Participant Is a man drinking coffee? No match. 1st presentation
Director/Confederate Matcher/Participant Is a tall woman throwing a bone to a dog? Yes, a tall woman is throwing a bone to a dog. 2nd presentation: Biasing Question
Biasing question like having pre-planned order: no uncertainty about order of mention "Yes, a woman is throwing a bone to a dog"
Director/Confederate Matcher/Participant Is a tall woman swimming? No, a tall woman is throwing a bone to a dog. 2nd presentation: Neutral Question
Emergent argument ordering: "Yes, a woman is throwing a bone to a dog"
Dependent measure: gaze shifts "Yes, a woman is throwing a bone to a dog"
More shifts between theme & recipient during articulation of 1st noun phrase shifts + SE Match, a woman is throwing a bone to a dog Griffin, Garton, & Mouzon (in prep)