1 / 14

Automatic and Data Driven Pitch Contour Manipulation with Functional Data Analysis

Automatic and Data Driven Pitch Contour Manipulation with Functional Data Analysis. Michele Gubian, Lou Boves Radboud University Nijmegen Nijmegen, The Netherlands Francesco Cangemi Laboratoire Parole et Langage University of Provence, Aix-en-Provence, France. Outline.

kacia
Download Presentation

Automatic and Data Driven Pitch Contour Manipulation with Functional Data Analysis

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Automatic and Data DrivenPitch Contour Manipulationwith Functional Data Analysis Michele Gubian, Lou Boves Radboud University Nijmegen Nijmegen, The Netherlands Francesco Cangemi Laboratoire Parole et Langage University of Provence, Aix-en-Provence, France

  2. Outline • Pitch Contour Manipulation • Context and problem • Sketch of proposed approach • Use of Functional Data Analysis (FDA) • Case study • Data preparation • Functional PCA • Functional synthesis and listening • Conclusions

  3. Context • Languages can express oppositions using intonation • Question/Statement opposition in Neapolitan Italian QUESTION STATEMENT “Milena lo vuole amaro (?)” = Milena drinks it (her coffee) bitter (?) • What are the intonation cues that listeners use? • Perceptual experiments where listeners judge stimuli whose pitch (F0) contour has been manipulated • STEP 1: extract pitch contours from speech data • STEP 2: modify pitch contours • STEP 3: re-synthesize speech

  4. Pitch Contour Manipulation F0 time • Use of an intonation model • Stylization • Manual changes POSSIBLE IMPROVEMENTS • Handle dynamic detail • Locally (e.g. concavity/convexity) • Long range correlation • Derive useful variation modes directly and automatically from data

  5. A data driven approach Functional Data Analysis x

  6. Question/Statement opposition in Neapolitan Italian DATA • 2 male speakers • 3 carrier sentences (read speech) • “Milena lo vuole amaro (?)” = Milena drinks it (her coffee) bitter (?) • “Valeria viene alle nove (?)” = Valeria arrives at 9 (?) • “Amelia dorme da nonna (?)” = Amelia sleeps at grandma’s (?) • 2 modalities = Q / S • 5 repetitions • 2 x 3 x 2 x 5 - 3 discarded = 57 utterances

  7. Data Preparation • Sampled F0 curves have to be turned into functions • A basis of functions (B-splines) expresses each original curve • Decide how much detail to retain (smoothing)

  8. Data Preparation (2) • Landmark registration • Align points in time that are deemed as having the same meaning across the dataset

  9. PC1 x PC2 x x x x x x x x x x x x x x x x x x salary x x x x x x x x x x x x x x x x x 25 65 age ClassicPrincipal Component Analysis (PCA)

  10. Functional PCA

  11. PC-based signal reconstruction + 1.65 x - 0.46 x mean(t) PC1(t) PC2(t)

  12. Manipulated stimuli

  13. Conclusions • A data driven approach is possible in the exploration of intonation phenomena • FDA provides automatic tools to describe variation in a set of pitch contours extracted from real utterances • provided that the relevant landmarks are annotated • The same tools allow to construct artificial contours with desired perceptual characteristics • Smooth and global variation are applied • Variations come from a statistical analysis of data • The process is automatic

More Related