Stress as Attention Modulation and the Roots of its Formal Properties

1. Stress as Attention Modulation and the Roots of its Formal Properties Kenneth de Jong Indiana University Jones.ling.indiana.edu/~kdejong & www.indiana.edu/~lsl (slides at jones.ling.indiana.edu/~kdejong/Buffalo_stress.ppt)

2. Stress as Attention Modulation and the Roots of its Formal Properties Work reported here in collaboration with Byung-jin Lim, Kyoko Nagao, Eric Oglesbee, Noah Silbert, and Bushra Zawaydeh, and supported by NSF: BCS-9910701 & BCS-04406540, and NIDCD: R03 DC04095

3. Overall Thesis • The formal properties of linguistic stress can be explained terms of ... • the definition of linguistic stress • Linguistic stress = conventionalized attention modulation at the level of the syllable. • how stress begins historically • Attention modulation 'grows from' seeds of syllable-level prominence • the function of linguistic stress • Attention modulation is one strategy for governing focus on informationally important units

4. Types of Explanations (deJong, 2007) • Physiological facts and constraints on control Hardware • Motoric propensities which encourage certain articulatory coordinations Activeware OR … • Requirements of communication with a listener Shareware OR … • It’s the way mom & dad did it Ancestorware

5. Types of Explanations (deJong, 2007) • Need to consider interaction of activeware & shareware factors • As they are encoded in the ongoing aggregation of a language … ancestorware

6. Overall Thesis • The formal properties of linguistic stress can be explained terms of ... • the definition of linguistic stress • Linguistic stress = conventionalized attention modulation at the level of the syllable. • how stress begins historically • Attention modulation 'grows from' seeds of syllable-level prominence

7. Overall Thesis • The formal properties of linguistic stress can be explained terms of ... • the definition of linguistic stress Shareware • Linguistic stress = conventionalized attention modulation at the level of the syllable. • how stress begins historically • Attention modulation 'grows from' seeds of syllable-level prominence

8. Lindblom’s Hr + Ho Model (Lindblom 1990) • Hyperarticulation (Hr) - Hypoarticulation (Ho) • Hr = output oriented constraints (Shareware, since these constraints are those imposed by the function of matching speaker and listener) • Ho = production-oriented factors (Hardware & Activeware, since these factors are aspects of speakers’ motor control systems) • Not OR …; but AND

9. Lindblom (1990) “ For the ideal speaker, H+H claims that … adaptations reflect his tacit awareness of the listener’s access to sources of information independent of the signal and his judgement of the short-term demands for explicit signal information.”

10. Evidence behind Hr + Ho • Ho: motor optimization • Lindblom’s work over a long period of time shows economization of articulatory actions (e.g. tip & jaw movement), and ‘soft’ temporal effects on undershoot. • Economization, if not omnipotent, appears omnipresent • Hr: intelligibility boost • The best speech intelligibility enhancement device is the speaker • Speakers know how get themselves heard

11. Extension of Hr+Ho(de Jong, 1991; 1995; 1998) • Phonetic variation due to linguistic stress looks like Lindblom’s Hyperarticulation. • Specified not in terms of specific features, but in terms of a functional increase in intelligibility • At that time: This was Uhglee. • Essentially stress is open-ended adaptation, introducing the unanswered question of how to characterize intelligibility

12. Stress functionality • Productions of stressed items are given more attention (de Jong, 1991; 1995;1998) • Perceptions of stressed items are more important for intelligibility (Cole, et al, 1978, Cole and Jakimik, 1980; Bond and Garnes, 1980; Terken and Nooteboom, 1987) A stress system = a convention in which both speakers and listeners dynamically pay more attention to certain syllables than to others

13. Shareware Explanation - Attentional Dynamics • Auditory attention work by Mari Jones & others (Jones, 1976; Jones & Boltz, 1989; Jones & Yee, 1993; Large & Jones, 1997) • attentional selectivity • attentional capture • attentional integration • temporal expectancy

14. Stress variation - Attentional dynamics • Part 1.attentional selectivity: some parts of a stimulus are more readily acted upon than others • Stress => some syllables are attentionally selected

15. Stress variation - Attentional dynamics • Part 2. attentional selectivity arises from • ---> attentional capture: parts which suddenly change in salient dimensions tend to garner such selective advantages • Attention is attracted to acoustic events where sudden changes take place --> syllables are typically characterized by a sudden onset of acoustic energy • Attention is attracted to acoustic events where sudden changes take place, e.g. pitch accents --> stressed syllables often act as sites for pitch accent docking

16. Stress variation - Attentional dynamics • Part 3. attentional selectivity also • --->may exhibit attentional integration: aspects which get attended to as a unit are those which work together to define an object or event • works on portions of speech which cohere in audition, such as syllabic groupings • Hence, again, stress is a property of the syllable • (MORE on this later on, though)

17. Stress variation - Attentional dynamics • Part 4. Attentional modulation is generally governed by temporal expectancy. • High attention areas may, under appropriate conditions, come at temporally predictable intervals. • Hence stress is sometimes governed by rhythmic, metrical patterning.

18. Stress variation - Attentional dynamics What we have is a Shareware explanation for stress-related phonetic variability.

19. Focus & Stress (de Jong & Zawaydeh, 2002; de Jong, 2004) • To firm up link between stress & hyperarticulation … • have speakers explicitly do hyperarticulation, contrasting a lexical target with another one. • - two types: • 'broad' lexically contrastive focus (e.g. 'bad', not 'good') • 'narrow' phonologically contrastive focus (e.g. 'bad', not 'bat') (c.f. van Heuven, 1994)

20. Vowel Duration & Voicing Focus by voicing Stress by voicing

21. Type of Focus - example results • Stress & focus have similar effects • Focus interacts with stress • Explicit phonological focus is less consistent across speakers

22. English vs. Arabic • Voicing-stress interactioneffects are different

23. Arabic vs. English quantity • Stress degree is different in the two • English: long & short vowel differences get larger with focus and stress in combination • Arabic: no interaction between stress & focus

24. Interim Summary • Contrastive dimensions are expanded by stress and focus • Dimensions that are not used are not expanded, e.g. voicing -/-> vowel duration in Arabic • English has more of a stress system than Arabic; focus effects are strongly localized in stressed positions in English

25. Next • Survey what effects are found for stress and focus -> tells us more about relationship between stress and syllables

26. Focus & Stress Effects • Connect contrastive dimensions to focus modulations: onset voicing and VOT & F0 in voicing contrasts (Oglesbee, 2008) • VOWELS: vowel quality (de Jong & Zawaydeh, 2002; de Jong, 2004) • CONSONANTS: fricative timbre (Silbert & de Jong, 2008) - More CONSONANTS: coda voicing and vowel duration (de Jong & Zawaydeh, 2002; de Jong, 2004)

27. Focus & Stress Effects • Connect contrastive dimensions to focus modulations: onset voicing and VOT & F0 in voicing contrasts (Oglesbee 2008) • VOWELS: vowel quality (de Jong & Zawaydeh, 2002; de Jong, 2004) • CONSONANTS: fricative timbre (Silbert & de Jong, 2008) - More CONSONANTS: coda voicing and vowel duration (de Jong & Zawaydeh, 2002; de Jong, 2004)

28. Oglesbee (2008) • Measured perceptual sensitivities in a goodness estimation task to VOT, F0 contour, and intensity dynamics for /p/ and /b/ • Compared productions of /p/ and /b/ in focus and non-focus positions • Subjects created larger differences with focus in dimensions with perceptual sensitivity • Dimensions: • All: VOT • S1: vowel amplitude ramp (/b/ sharper) • S2: F0 contour (/p/ higher)

29. Focus & Stress Effects • VOT, F0, and onset amplitude cues to voicing become more extreme with focus, in parallel with perceptual sensitivity in goodness tasks • VOWELS: vowel quality (de Jong & Zawaydeh, 2002; de Jong, 2004) • CONSONANTS: fricative timbre (Silbert & de Jong, 2008) - More CONSONANTS: coda voicing and vowel duration (de Jong & Zawaydeh, 2002; de Jong, 2004)

30. Focus & Stress Effects • VOT, F0, and onset amplitude cues to voicing become more extreme with focus, in parallel with perceptual sensitivity in goodness tasks • VOWELS: vowel quality (de Jong & Zawaydeh, 2002; de Jong, 2004) • CONSONANTS: fricative timbre (Silbert & de Jong, 2008) - More CONSONANTS: coda voicing and vowel duration (de Jong & Zawaydeh, 2002; de Jong, 2004)

31. Non-high front vowels -> Vowel quality Stress by voicing Dashed arrows indicate voicing effects Solid arrows indicate effects going from: Unstressed -> secondary stressed Secondary -> primary stressed - F1 (Hz) - F2 (Hz)

32. Focus & Stress Effects • VOT, F0, and onset amplitude cues to voicing become more extreme with focus, in parallel with perceptual sensitivity in goodness tasks • VOWELS: vowel quality becomes more extreme with focus and stress -> vowel space is a gradient contrast space with weak motor constraints • CONSONANTS: fricative timbre (Silbert & de Jong, 2008) - More CONSONANTS: coda voicing and vowel duration (de Jong & Zawaydeh, 2002; de Jong, 2004)

33. Focus & Stress Effects • VOT, F0, and onset amplitude cues to voicing become more extreme with focus, in parallel with perceptual sensitivity in goodness tasks • VOWELS: vowel quality becomes more extreme with focus and stress -> vowel space is a gradient contrast space with weak motor constraints • CONSONANTS: fricative timbre (Silbert & de Jong, 2008) - More CONSONANTS: coda voicing and vowel duration (de Jong & Zawaydeh, 2002; de Jong, 2004)

34. Silbert & deJong (JASA, 2008) • Examined English /f/ /v/ /s/ & /z/ • Prevocalic, post-vocalic & focus, non-focus • Measured intensity, duration, spectral quality and dynamics of quality • Focus affects only duration and intensity -> NO spectral effects

35. Focus & Stress Effects • VOT, F0, and onset amplitude cues to voicing become more extreme with focus, in parallel with perceptual sensitivity in goodness tasks • VOWELS: vowel quality becomes more extreme with focus and stress -> vowel space is a gradient contrast space with weak motor constraints • CONSONANTS: fricative timbre is not affected by focus -> rigid motor constraints - More CONSONANTS: coda voicing and vowel duration (de Jong & Zawaydeh, 2002; de Jong, 2004)

36. Focus & Stress Effects • VOT, F0, and onset amplitude cues to voicing become more extreme with focus, in parallel with perceptual sensitivity in goodness tasks • VOWELS: vowel quality becomes more extreme with focus and stress -> vowel space is a gradient contrast space with weak motor constraints • CONSONANTS: fricative timbre is not affected by focus -> rigid motor constraints - More CONSONANTS: coda voicing and vowel duration (de Jong & Zawaydeh, 2002; de Jong, 2004)

37. Voicing -> Vowel duration Focus by voicing Stress by voicing

38. Focus & Stress Effects • VOT, F0, and onset amplitude cues to voicing become more extreme with focus, in parallel with perceptual sensitivity in goodness tasks • VOWELS: vowel quality becomes more extreme with focus and stress -> vowel space is a gradient contrast space with weak motor constraints • CONSONANTS: fricative timbre is not affected by focus -> rigid motor constraints - More CONSONANTS: other consonantal cues are heavily affected

39. Why Syllables? (2) • Stress effects strongly localized in vocalic nucleii • Consonant effects similarly most readily apparent in nucleii • Consonant acoustics restricted to effects in terms of durational and intensity properties • All of which suggests a model of syllable nucleii as motorically fluid repositories of variation • Consonantal margins are motorically constrained

40. Summary 2 • Stress - Hyperarticulation link is generally supported, BUT … • Stress looks to be a better exemplar of Hyperarticulation than does focus • Um, huh? Why? • Stress is part of English conventions: it’s actually Ancestorware

41. Summary 3 Suggests a model with two components • Shareware: Hyperarticulation happens according to speakers’ perceptions of listeners’ ability to perceive - This hyperarticulation in general is unevenly distributed in the signal • Ancestorware: Hyperarticulation effects get contributed to the pool of experiences that the listener has with particular lexical items - Shareware factors get encoded in Ancestorware

42. Overall Thesis • The formal properties of linguistic stress can be explained terms of ... • the definition of linguistic stress Shareware • Linguistic stress = conventionalized attention modulation at the level of the syllable. • how stress begins historically • Attention modulation 'grows from' seeds of syllable-level prominence

43. Overall Thesis • The formal properties of linguistic stress can be explained terms of ... • the definition of linguistic stress Shareware • Linguistic stress = conventionalized attention modulation at the level of the syllable. • how stress begins historically Ancestorware • Attention modulation 'grows from' seeds of syllable-level prominence

44. Ancestorware component • Some formal properties may be explained by historical language dynamics • Quantity sensitivity in stress systems

45. Quantity sensitivity • Stress location often is preferential as to what sort of syllable it resides on • Formal property used to determine preference is weight (Newman, 1972) • Weight is determined with respect to an implicational hierarchy of the following form: Long V > Coda son > Coda obstruents > Short vowels [..V:C] > [..V:] > [..VR] > [..VO] > [..V] • Languages differ as to where boundary between light and heavy syllables falls

46. Quantity sensitivity problems • Gordon (1999) • Surveyed 388 stress rules • Languages have different weight criteria for different phonological phenomena • Stress rules virtually always treat all consonants the same when determining weight • Ahn (2000) • Surveyed 136 stress rules • Found huge differences in the frequency of different stress systems

47. Ahn (2000) • In the 23 unbounded quantity sensitive system, consonants never count as heavy • Only vowels get stress, if stress position is not tied to a word edge

48. Ahn (2000) • Further, foot type (iamb vs. trochee) is strongly linked to which word edge is important

49. Ahn (2000) • Further effects • 81% of languages: Heavy syllable stresses are one syllable closer to a word edge than light syllable stresses are • The effect of consonants in stress rules is almost entirely restricted to that of pushing the target vowels away from word edge

50. Korean Stress • Utterance final or word initial (Polianov, 1936) • Word initial (Huh, 1985; JS Lee, 1992) • Unbounded, but if no long vowels then word initial quantity sensitive (HB Lee, 1974) • Unbounded system, closest to initial edge (J Yu, 1989) • Word initial, quantity sensitive (HY Lee, 1990; JK Kim, 1998) • Phonemic (Zong, 1965; Cho, 1967) • None of the above (SA Jun, 1993; me, 1994)