1 / 37

Shari R. Speer Ohio State University

Intonationally marked contrast in instructed visual search: Intersective color vs. subsective scalar adjectives. Shari R. Speer Ohio State University. Acknowledgements. Co-I, Kiwako Ito. NSF BCS-018464 NSF BCS-0617609 NIDCD R01-DC007090 Ping Bai Allison Blodgett Laurie Maynell

lareina-strong
Download Presentation

Shari R. Speer Ohio State University

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Intonationally marked contrast in instructed visual search: Intersective color vs. subsective scalar adjectives Shari R. Speer Ohio State University

  2. Acknowledgements Co-I, Kiwako Ito NSF BCS-018464 NSF BCS-0617609 NIDCD R01-DC007090 Ping Bai Allison Blodgett Laurie Maynell Mary Beckman

  3. Contrast • Similarities and differences  comparisons Bolinger (1961) “…cases where one or more individual items are singled out from a larger (but limited) set as being true as regards some relationship whereas others in the same set are untrue…” Zeevat (2004) Contrastors (alternatives)… “must be obtainable from the actual utterance by substituting something else for the intonationally prominent constituent.”

  4. Contrast • Discourse coherence prerequisite for anaphora resolution • Must be in focus domain • Often accompanied by structural parallelism • Matter of degree?

  5. Accentuation for expressing contrast • B (vs. A) accent (Bolinger, 1961; Jackendoff, 1972) • L+H* (vs. H*) (Pierrehumbert, 1980; ToBI) Pierrehumbert & Hirschberg (1990) “the accented item - and not some alternative related item - should be mutually believed (p 296.)” • L+H*: +AGREED, theme accent (Steedman, 2003)

  6. Intonationally marked contrast in sentence/discourse processing • Faster comprehension of short contrastive discourse when prominent accents were appropriately located (vs. inappropriate) (Bock & Mazella, 1983) • Faster comprehension and higher acceptance of appropriately accented Q-A pairs (English pitch accent: Birch & Clifton, 1995; Japanese, Basque pitch range expansion/peak alignment: Ito, 2002). • Faster phoneme monitoring when the contrastive entity was negated with prominent accent (vs. no accent). Intonationally marked narrow focus in negation generates contrast set (Davidson, 2001).

  7. Accentuation shows immediate effects on instructed visual search (Dahan, Tanenhaus & Chambers, 2002) • Prominent accent interpreted non-anaphorically, lack of prominent accent interpreted anaphorically. 1. “Put the candle/candy below the triangle.” 2. “NOW, put the CANDLE above the square.” “NOW, put the candle ABOVE THE SQUARE.” fixation to candy candle  “CANDLE” candle  “candy” fixation to candle

  8. L+H* on a modifier evokes a contrast set (Ito & Speer, 2008) Manipulated pitch accent type on a modifier adjective or modified noun L+H*: Evokes a contrast set that contains alternative related item(s) (Pierrehumbert & Hirschberg, 1990) blue ball GREEN GREEN ball ball

  9. Holiday Tree task(Ito & Speer, 2008) Eyemovements monitored as naïve participants followed recorded audio instruction to decorate Christmas trees. • (ASL e5000) at 60Hz.

  10. Visual Stimuli(Ito & Speer, 2008) • 44 ornaments displayed by type on a grid with 11 cells (8 target + 3 filler)

  11. Sound Stimuli(Ito & Speer, 2008) • Accentual patterns of stimuli were selected based on the analysis of spontaneous speech in a similar task (Ito, Speer & Beckman, 2003). • A trained female phonetician produced stimuli with the intended pitch accent patterns. Adjective-noun pairs were presented in discourse context. • Recorded 44.1KHz, 16bits • All stimuli were ToBI-annotated by two native speakers of American English. Items on which they did not agree were re-recorded.

  12. Experiment 1: Stimuli • Felicitous L+H* Patterns green onion ORANGE onion L+H* no accent brown ball brown ANGEL H* L+H* • Infelicitous L+H* Patterns gray stocking brown STOCKING H* L+H* orange candy ORANGE onion L+H* no accent

  13. Experiment 1 Results: Felicitous vs. Infelicitous L+H* (Ito & Speer, 2008)

  14. Experiment 1: Summary • Felicitous L+H* on the color term in contrastive environments (green drum, BLUE drum) facilitated visual search compared to infelicitous L+H* on non-contrastive noun (green drum, blue DRUM). • Felicitous L+H* on the object noun in contrastive environments (blue egg, blue DRUM) showed a smaller advantage over infelicitous L+H* on the adjective (blue egg, BLUE drum). • Listeners ‘tune’ their sensitivity to contrastive accent on the basis of the visual task?

  15. Experiment 2: stimuli • Does L+H* on the contrastive adjective lead to anticipatory eye-movement compared to H*? e.g. brown drum --> RED/red drum • Does infelicitous L+H* on the color adjective lead to an expectation strong enough to lead a “garden-path” in eye-movement? e.g. red onion --> GREEN drum

  16. Experiment 2 Results: Felicitous/Infelicitous L+H* vs. H*

  17. Experiment 2 results: “Garden-path” due to L+H*

  18. Experiment 2 results: Refixation of previous cell w/out L+H*?

  19. The effect of infelicitous L+H*

  20. Ito & Speer 2008: Summary • For cases with repetition of the immediately preceding noun, L+H* showed a processing advantage as compared to H* on the adjective (brown drum --> RED/red drum) • For cases without repetition of the noun, infelicitous L+H* led to incorrect anticipatory fixations to the most-recently mentioned ornament type. significant delay in fixations to the real target • NO incorrect anticipatory fixations were observed when non-repeated targets were felicitously presented with [H* !H*]

  21. Discussion • Dahan et al results showed that L+H* accent led to the expectation of contrast. Contrastive accent on a (potentially) repeated word led listeners to fixate the object type that had *not* just been mentioned. • Results here show contrast effects from L+H* accent on the modifier that preceded the noun. L+H* led listeners to restrict the set of expected referents to those of the object type that *had* been most recently mentioned.

  22. Discussion • Evidence suggests that L+H* evokes generation of a contrast set in the listener’s discourse representation. The set is based on the accented word interpreted within the structure of the utterance. Here, contrastive accent on the adjective restricted expectations about the coming head noun to the set of objects specified by the immediately preceding noun. • Facilitative anticipatory fixations and prosodic ‘garden-path’ effects due to prominence and have also been shown for German (Weber et al, 2006) and Japanese (Ito et al, under review).

  23. Remaining questions • Were facilitation effects of L+H* dependent on the visual layout, where contrastive trials required re-fixation of a ‘just-visited’ ornament cell? In Ito&Speer 2008, rapid integration of contrastive L+H* involved refixation of the immediately preceding ornament cell. • Was predictive use of L+H* in the ‘garden-pathing’ trials dependent on the organization of the ornament array by objects (nouns)? • Are primary color adjectives, which describe a context- independent distinction between modified objects, more likely to require intonational contrast than other types of adjectives (e.g. scalars)?

  24. Followup studies: We compared 2 adjective types: • Color (context-independent, intersective) • Size (scalar,subsective) Because relative size terms are inherently contrastive and context-dependent, prominent pitch accent may be redundant as a cue to contrast. We used adjective-sorted arrays, where contrastive sequences (e.g., red ball  green ball, small ball  medium ball) require fixations to a new cell (instead of returning fixations).

  25. Exp3 Visual stimuli - Colors

  26. Exp4 Visual stimuli - Sizes

  27. Sound stimuli • Articles rather than determiners, e.g. ‘Hang a green ball,’ ‘Hang a large ball,’ as there were multiple like ornaments. • Same native English ToBI-trained speaker • Same procedures for annotation, presentation and eyemovement monitoring • Four conditions: Contrastive vs. non-contrastive H* !H* vs. L+H* no accent

  28. Anticipatory effect w/out ‘refixation’ Exp3, color adjectives: “Hang a yellow/YELLOW star” Exp4, scalar adjectives: “Hang a large/LARGE tree”

  29. Garden-path effects Color: fixations to green tree hearing ‘GREEN ball’ Scalar: fixations to large tree hearing “LARGE ball”

  30. Summary: Fixation proportions • L+H* facilitates eyemovements to the targets for bothcolor and size adjectives in contrastive sequences. • Visually more complex size-sorted boards led to slower eye movements to the target than color-sorted boards. • L+H* led to more frequent fixations to the incorrect targets for bothcolor and size adjectives in non-contrastive sequences. • Non-prominent size adjective (with H*) did not lead to looks to contrastive cells, i.e., size adjectives are not automatically interpreted contrastively.

  31. Summary • Prominent pitch accent (L+H*) on a pre-nominal modifier evokes a contrast set and facilitates reference resolution with both intersective color adjectives and subsective scalar adjectives. • L+H* evokes the anticipation of contrast, and thus may lead to intonation-based “garden-pathing” regardless of the adjective type used. • The interpretation of pre-nominal adjectives depends on their efficacies within each referential environment rather than on their inherent semantics. Intonational prominence is used immediately and predictively for reference resolution regardless of the modifier type.

  32. Current directions • L+H* vs. H* controversy: • Phonetic analyses of stimuli from 4 comprehension experiments underway • Correlation of first fixation latency and acoustic measures • Parallel analysis of spontaneous speech from production version of holiday tree task

  33. Issue on categorical distinction:Is L+H* a kind of H*? • Color adj with H*  weaker interpretation of contrast? • Categorical interpretation but non-categorical perception (Ladd & Morton, 1997) • Great overlap between H* and L+H* in pitch scale, shape & alignment (Tayler, 2000) • Frequent uncertainty between H* and L+H* in expert-ToBI labeling (Brugos et al. 2008). • More frequent use of H* than L+H* to mention contrastive discourse entities in story continuation (Metusalem & Ito, 2008). • How can we define categories of prosodic prominence? • What factors contribute to recognition & processing of contrastiveness?

  34. Step-wise Multiple Linear Regressions:Predicting First fixation latency: L+H* EXP1: Color-sorted EXP2: Size-sorted Total R2: .051 F(2,256) = 7.96, p<.001 Total R2: .245 F(2,296) = 25.38, p<.0001

  35. Step-wise Multiple Regressions:Predicting First fixation latency: H* EXP1: Color-sorted EXP2: Size-sorted Total R2: .108 F(3,257) = 11.59, p<.0001 Total R2: .088 F(3,293) = 10.57, p<.0001

  36. Further exploration needed: • Different dependent variable? • Additional phonetic variables: • Normalized F0 scaling • Peak, ‘low elbow’ alignment from vowel onset • Word/syllable intensity • Vowel quality (F1, F2, breathiness, etc.) • Following noun’s phonetic status • F0 prominence • intensity • vowel quality, etc.

More Related