Intonation: buildings and bricks

1. Intonation:buildings and bricks Francesco Cangemi Universität Zürich & Universitätzu Köln fcangemi@uni-koeln.de

4. Prosody and arbitrariness Both highly universal and language-specific Physio- and psychologically motivated Acquisition Learning Pathology Phylogenies Ontogenesis First Last

5. PROSODY Last to disappear Learning Pathology First to appear The workmen from Boston were leaving M F • hier soir, avant de s'endormir, François fumait une dernière cigarette, en relisant le cours d'allemand qu'il avait préparé pour ses élèves de terminale. Puis, il écrasa sa gauloise dans un cendrier, et éteignit la lumière. Un moment plus tard, une odeur de brûlé le réveilla. La pièce était envahie de fumée, et François s'aperçut avec effroi que les rideaux de la fenêtre avaient pris feu.  • [Louis et alii, 2001]

6. Prosody and function PROSODY Disambiguation of (surface) syntactic structures Message encoding and decoding (I’ll move on) (Saturday) (I’ll move) (on Saturday)

7. Message encoding and decoding PROSODY Disambiguation of (surface) syntactic structures Lexical access [Christophe et alii 2004] Le livreracontaitl’histoire… [d’un grand chat grincheux] [qui avaitmordu un facteur] CHAGRIN [d’un grand chat drogué] [qui dormait tout le temps] *CHAD

8. Disambiguation of (surface) syntactic structures Message encoding and decoding Lexical access PROSODY Management of interaction e.g. backchannels [Benus et alii 2007] Information structure Modality E.g., it.: Michele viene con me Micheal comes with me Contrastivity Givenness

10. Architectures amplitude voice quality duration f0 SUBSTANCE FORM FUNCTION Syntax Information structure Lexicon Interaction

11. Architectures SUBSTANCE FORM /kæt/ FUNCTION

12. Architectures (intonation) SUBSTANCE • Intonation refers to • the use of suprasegmental phonetic features • to convey post-lexical or sentence-level pragmatic meanings • in a linguistically structured way. • [Ladd , 1996] FUNCTION FORM

13. Architectures (prosody) (Prosodyis a) branch of linguisticsdevoted to the factual description (phonetic aspects) and the formalanalysis (phonological aspects) of the systematicelementsin the phonic expression which are not coextensive to phonemes, such as accents, tones, intonation and quantity. whoseactual manifestations in speech production are associatedwith variations in the physicalparameters of f0, duration and intensity, whichrepresentprosody’s objective parameters. These variations are perceived by the listener as changes in pitch, length and loudness, which are prosody’s subjective parameters. The prosodicelementsplayat (lexical prosody) or above (post-lexicalprosody) the wordlevel a bundle of grammatical, para-grammatical and extra-grammatical functions, related to ‘whatissaid’, ‘how itissaid’ and to speaker identity. Thesefunctionsprove crucial in signalling the structure of utterances and of discourse, and in guidingtheirsemantic and pragmaticinterpretation. [Di Cristo2004] FORM SUBSTANCE FUNCTION

15. An alternative architecture …these models introduce a phonological level of description that is intermediate between (abstract) function and (concrete) phonetic form …it is our experience that one always get better results if one can do without such an intermediate level, i.e., if one can establish a direct link between (syntactic/semantic) function and phonetic form • “the unfortunate notion of pitch accent” [Batliner and Möbius2005]

16. An alternative architecture exemplar FORM substance function auditory properties f0,F1,F2,F3,dur category labels word,sex,speaker [Johnson 1997] [K. Schweitzer 2012]

17. Categorization Monothetic approach (classical view) Singly necessary and jointly sufficient features bug /bʌg/ bun /bʌn/

18. Polythetic (Familienähnlichkeit) elements of a class share more or lessfeatures Prototypic (probabilisticview) Categories center around members sharing manyfeatures

19. A Episodic(exemplarview) Online categories, comparing probe to storedexemplars B C

21. Phonetic detail Phoneticdetail: systematicallyproduced and perceived phonetic information whichis not included in abstract phonologicalrepresentations Exemplar-basedmodels: Traces for words are stored withoutreducingphonetic information to an abstract phonologicalrepresentation

23. Exemplar prosody • Exemplar-basedmodels: • Traces for words are stored • withoutreducingphonetic information • to an abstract phonologicalrepresentation • + • Functional approaches to prosody: • No intermediate phonological level, but only a direct link between function and phonetics • = • Exemplar-basedprosody: • Words are storedalongwith • their f0 contours exemplar set of category labels set of auditory properties [K. Schweitzer 2012]

24. Featureanalysis Memory requirements Restrictive view of phoneticdetail: systematicallyproduced and perceivedphonetic information whichisnot YET included in abstract phonologicalrepresentations Words are stored alongwiththeir f0 contour

25. Prosodic detail Abstract forms (e.g. pitch accents) in AM theory are phoneticallyveryunderspecified Ifsystematicallyproduced and perceived phonetic informationisfound tocuefunctionalcontrasts, phonologicalrepresentations mightbeenriched either in inventory orgrammar PROSODIC CUES amplitude spectral features f0 duration interpolations events

27. PROSODIC CUES amplitude spectral features f0 duration interpolations events Partial Topic constructions seem to have distinctive riseshape • Interpolation ratherthanevents?!

28. Partial Topics in Neapolitan Italian Narrowing down the Discourse Topic Non exhaustive answer [Büring 1997] A: How do your friends like their coffee? B: Milena drinks it black (as for the others, I wouldn’t know)

29. Reading task with contextualizing paragraph 3 sentences 2 contexts 7 speakers 5 repetitions 210 items Two-sample two-tailed T-tests Alignment Scaling Curve index: [Dombrowski & Niebuhr 2005; Cangemi 2009] Both with prosodic break after N L is acceleration peak (d”)

30. Events Interpolations p<0.001

31. Other contrasts… Petrone and D’Imperio (2008): PRENUCLEAR instead of nuclear FALLS instead of rises MODALITY instead of S-Topic (narrow focus question/statement)

32. Other languages… Appointment scheduling task Turn-yielding (left) “Ostermontag” vs turn-holding (right) “AnfangDezember” Dombrowski and Niebuhr (2005): Utterance FINAL instead of initial CONVERSATIONAL functions

34. Enrichingforms… Ifsystematicallyproduced and perceived phonetic informationisfound tocuefunctionalcontrasts, phonologicalrepresentations mightbeenriched either in inventory orgrammar Petrone and D’Imperio (2008) Modalitycontrast via a novel tonal contrast for a new prosodicdomain: Accentual phrase Dombrowski and Niebuhr (2005) Turn-takingcontrast via a split of earlyvalleys according to concavity/convexity Rise shapes

35. …but whataboutfunction? • This functional point of view has givenway to more formalcriteriasuch as • economy of description • [Batliner and Möbius 2005] In the segmentaldomain, linguisticcategoriesare expected to relate both to differences in sounds and articulationsand to differences in semanticinterpretation. For example, wesaythat [p] isdifferentfrom [b] becausethey are pronounceddifferently, and because [pit] meanssomethingdifferentthan [bit] does. [Pierrehumbert and Hirschberg 1990] SUBSTANCE FORM /kæt/ FUNCTION

36. Bricksandbuildings • Constitutionaldifferencebetween the primitive-architecture relationship • for segmental and suprasegmental phonology! SUBSTANCE SUBSTANCE labial closure norounding nasality … ...labial clos. norounding velar closure /bʌn/ FORM /bʌg/ – animate + edible + organic + animate + edible + organic FUNCTION FUNCTION

37. Bricksandbuildings • Constitutionaldifferencebetween the primitive-architecture relationship • for segmental and suprasegmental phonology! SUBSTANCE SUBSTANCE labial closure norounding nasality … ...labial clos. norounding velar closure /bʌn/ FORM /bʌg/ – animate + edible + organic + animate + edible + organic • NO • DIRECT • LINKS ! FUNCTION FUNCTION

38. Thenwhysuggest a direct linkbetween substance and function for prosody? • Batliner, Möbius • Even intonational phonologistsworkingwith gestalt-likewholes! • Dombrowski, Niebuhr SUBSTANCE SUBSTANCE valley early concave valley early convex FORM – turn yielding … + turn yielding … FUNCTION FUNCTION

39. Intonational meaning • Segments and tones: epilinguisticallysalientlexical contrasts • Intonation: theory-dependentpost-lexicalcontrasts • Evenworse, • theories of post-lexicalcontrasts • oftenimplicitly use • unanalyzed introspection or • one of many intonation theories • to frame and validatetheirproposals • [Jackendoff 1972] • SEMIOTIC PRIMITIVES: • There is no consensus on an • « atomiclevel » for intonation… • not even as controversial as the phoneme! • Extra tones in more complexgrammar • vs Shapes as part of holisticforms, • Sentence Partial Topicvs • Question Narrow Focus !

41. Challenging intonational primitives • Some slow steps in rethinking the (unfortunate) notion of pitch accent, • addressingtwo issues: • Individualvariability • in production • (soon perception) • Time-evolving and multiparametric • phoneticrepresentations • D’Imperio • Niebuhr • GiliFivela • Lou • Boves • Michele • Gubian

43. F0 time Manipulating f0 contours • So, interpolations betweentargetsmightbe relevant • Still, whenresynthesizing stimuli, weoftenmakelinearstylizations (Praat & Psola) • How canweretain • (or evenbetter, explore) • phonetic information reduction • in resynthesizing stimuli? Gubian, Cangemi, Boves (2010), Automatic and Data Driven Pitch Contour Manipulationwith Functional Data Analysis, Speech Prosody, Chicago

44. A datadrivenapproach Functional Data Analysis x

45. Test dataset • 2 male speakers, read speech • 3 sentences, 5 repetitions • 2 modalities • 57 utterances • Data driven resynthesis procedure • Provideonlycategorylabels • Test on smalldataset • Timing of stressedsyllables • Equalsyllabictemplate

46. Data Preparation • Sampled f0 curves have to be turned into functions • A basis of functions (B-splines) expresses each original curve • Decide how much detail to retain (smoothing)

47. Data Preparation (2) • Landmark registration • Align points in time that are deemed as having • the same meaning across the dataset

48. PC1 x PC2 x x x x x x x x x x x x x x x x x x salary x x x x x x x x x x x x x x x x x 25 65 age

49. Analysis (functional PCA)

50. Resynthesis (procedure) + 1.65 x - 0.46 x mean(t) PC1(t) PC2(t)