The prosodic marking of the contrast between restrictive and appositive clause in Dutch

1. The prosodic marking of the contrast between restrictive and appositive clause in Dutch Vincent J. van Heuven With the help of: Crit Cremers, Hanna Gauvin, Constantijn Kaland, Eddin Najetovic, Marjoleine Sloos, Hanna de Vries Linguistics Program, Universiteit Leiden

2. Towards the phonetics of the relative clause in Dutch Vincent J. van Heuven With the help of: Crit Cremers, Hanna Gauvin, Constantijn Kaland, Eddin Najetovic, Marjoleine Sloos, Hanna de Vries Linguistics Program, Universiteit Leiden

3. Introduction • Framework: improving prosody of text-to-speech (TTS) systems • Our parser/generator Delilah computes rich syntactic and semantic structures • Can be interfaced to standard TTS systems for Dutch • Opportunities for phonetic optimalisation of output van Heuven, NVFW/MPI

4. Introduction • Grammatical distinction in relative clause • (1) Restrictive clause • Within scope of antecedent • Rel. pronoun is not subject of relative clause • Not preceded by comma in text • Suggests relatively shallow prosodic boundary • E.g. No journalist [who signed the petition], was arrested. van Heuven, NVFW/MPI

5. Introduction • Grammatical distinction in relative clause • (2) Appositive clause • Not within scope of antecedent • Rel. pronoun is subject of relative clause • Preceded by comma in text input • Suggests relatively deep prosodic boundary, • E.g. Michael, [who beat up his girl friend], was arrested. van Heuven, NVFW/MPI

6. Introduction • Questions • How is the contrast between restrictive and appositive clause coded in the prosody? • Does the listener associate a specific prosodic structure with each type of clause? • Does correct prosodic marking yield better evaluation scores of text-to-speech system? van Heuven, NVFW/MPI

7. Prosodic boundaries • The deeper a prosodic boundary, ... • ... the longer the physical silence (pause) immediately before the boundary • ... the stronger the domain-final lengthening • ... the more disruptive the intonation pattern van Heuven, NVFW/MPI

8. Approach (1) • Perceptual evaluation of prosodic marking of • Synthetised speech with restrictive vs. appositive clauses • With / without pause • With / without domain-final lengthening • Four different melodies van Heuven, NVFW/MPI

9. Approach (2) • Four melodies at utterance-internal boundary • Cohesion marker H*L • No/neutral marker H*+L • Moderate break H* %L • Strong break H*L H% %L van Heuven, NVFW/MPI

10. Approach (3) • Two sentence structures that force just one interpretation onto relative clause • Restrictive (with downward quantifiers) • No journalist who signed the petition, was arrested • Few students who are are fraternity members, dislike a beer • Appositive (with proper names and/or evidentials) • Michael, who beat up is girl friend, was arrested • The woman, who – by the way – is wearing a pink dress, is from Germany van Heuven, NVFW/MPI

11. Approach (4) • Judgment/evaluation task for listeners • How do you rate the way the speaker reads the following sentence? • Scale from 0 to 10 • 0: very poor/inappropriate • 10: perfect van Heuven, NVFW/MPI

12. Hypotheses • Appositive clause correlates with • Physical silence • Pre-final lengthening • Maximally disruptive melodic pattern • Strong break > Moderate break > Neutral > Cohesion • Three factors are additive • i.e. better scores as more factors reinforce one another van Heuven, NVFW/MPI

13. Hypotheses • Restrictive clause correlates with • Absence of physical silence • Absence of domain-final lengthening • Minimally disruptive melody • Cohesion > Neutral > Moderate break > Strong break • Factors are additive van Heuven, NVFW/MPI

14. Methods (1) • Factors • Appositive vs. restrictive clause • Two lexically different instantiations • With / without pause (of 200 ms) • With / without domain-final lengthening (+40%) • Four different melodies • Total 2 x 2 x 2 x 2 x 4 = 64 types • Presented twice = 128 tokens per listener van Heuven, NVFW/MPI

15. Methods (2) • Materials spoken by native speaker of Dutch • Digital tape splicing for removal/insertion of lengthening and pause • PSOLA for duration manipulation and import of computer-generated melodies (imported from Fluency text-to-speech) van Heuven, NVFW/MPI

16. Methods (2) • Listeners • 20 native speakers of Dutch • Presentation through loudspeakers • 11-point quality judgment scale • Demonstration of sound files: van Heuven, NVFW/MPI

17. Disaster: the red condition (strong break) was inadvertently generated as the blue one (neutral). It will therefore be absent from the results section. My student Constantijn Kaland is now re-running the complete experiment for his BA thesis. No pause, +pause, No length +length GEEN journalist die de verklaring onderTEkend had, werd OPgepakt ‘No journalist who the statement signed had, was arrested’

18. Results • First: main effects • Yes / no physical pause • Yes / no domain-final lengthening • Type of melodic boundary marking • Second: interactions • i.e. breaking factors down van Heuven, NVFW/MPI

19. Effect of physical pause • Absence of pause is preferred for restrictive clause • Presence of pause is disfavored for restrictive clause • Difference is significant • Both presence and absence of pause are OK for appositive clause

20. Effect of domain-final lengthening • Tiny effects in predicted direction • No lengthening preferred for restrictive clause • Lengthening preferred for appositive clause • Effects are statistically insignificant

21. Effect of melodic configuration • Order of conditions reasonably in line with prediction • Indeed: green > yellow > red for restrictive clause • But should be red > yellow > green for appositive clause • Effects are statistically insignificant

22. Summary of results so far • Only one effect • Listeners prefer restrictive clauses without a physical pause separating clause from antecedent • All other prosodic markings are OK for both types of clause • Now let us look at interactions van Heuven, NVFW/MPI

23. Results: interactions • Effects of (i) lengthening and (ii) melody evaluated separately for presence vs. absence of physical pause van Heuven, NVFW/MPI

24. Interaction of pause and lengthening • Effect of lengthening as predicted (though not always significant) • More clearly when pause is absent than when present

25. Interaction of pause and melody Physical pause absent Physical pause present • Preference for cohesive H*L with restrictive clause • But only when pause is absent

26. Summary of results • Effects of (i) lengthening and (ii) melodic configuration are as predicted • But only if pause is absent • Indicates hierarchical structure • physical pause is the primary cue • lengthening and melody are subsidiary cues van Heuven, NVFW/MPI

27. Conclusions • Appositive and restrictive relative clauses in Dutch • are marked by different prosodic boundaries, • which are implemented by different phonetic cues • More specifically, … van Heuven, NVFW/MPI

28. Conclusions • … the evaluation study shows that the syntax-prosody interface should assign different prosodic boundaries to restrictive and appositive clauses: • deeper boundary to appositive clause • shallower boundary to restrictive clause • Therefore, … van Heuven, NVFW/MPI

29. Discussion • … quality of (Dutch) text-to-speech systems can be improved • Never insert physical pause before restrictive clause • Optionally accompany by appropriate secondary cues • Do not lengthen syllable before restrictive clause • Use cohesive melody before restrictive clause van Heuven, NVFW/MPI

30. Discussion • But • Implementation of secondary cues is not very effective • Fast and dirty implementation requires proper use of pause only • Rerun needed with strong prosodic break • Challenge left for computational linguists: • How to compute status of relative clause from text input? van Heuven, NVFW/MPI

31. The end • Thank you for your attention van Heuven, NVFW/MPI