140 likes | 330 Views
Automatic and Data Driven Pitch Contour Manipulation with Functional Data Analysis. Michele Gubian, Lou Boves Radboud University Nijmegen Nijmegen, The Netherlands Francesco Cangemi Laboratoire Parole et Langage University of Provence, Aix-en-Provence, France. Outline.
E N D
Automatic and Data DrivenPitch Contour Manipulationwith Functional Data Analysis Michele Gubian, Lou Boves Radboud University Nijmegen Nijmegen, The Netherlands Francesco Cangemi Laboratoire Parole et Langage University of Provence, Aix-en-Provence, France
Outline • Pitch Contour Manipulation • Context and problem • Sketch of proposed approach • Use of Functional Data Analysis (FDA) • Case study • Data preparation • Functional PCA • Functional synthesis and listening • Conclusions
Context • Languages can express oppositions using intonation • Question/Statement opposition in Neapolitan Italian QUESTION STATEMENT “Milena lo vuole amaro (?)” = Milena drinks it (her coffee) bitter (?) • What are the intonation cues that listeners use? • Perceptual experiments where listeners judge stimuli whose pitch (F0) contour has been manipulated • STEP 1: extract pitch contours from speech data • STEP 2: modify pitch contours • STEP 3: re-synthesize speech
Pitch Contour Manipulation F0 time • Use of an intonation model • Stylization • Manual changes POSSIBLE IMPROVEMENTS • Handle dynamic detail • Locally (e.g. concavity/convexity) • Long range correlation • Derive useful variation modes directly and automatically from data
A data driven approach Functional Data Analysis x
Question/Statement opposition in Neapolitan Italian DATA • 2 male speakers • 3 carrier sentences (read speech) • “Milena lo vuole amaro (?)” = Milena drinks it (her coffee) bitter (?) • “Valeria viene alle nove (?)” = Valeria arrives at 9 (?) • “Amelia dorme da nonna (?)” = Amelia sleeps at grandma’s (?) • 2 modalities = Q / S • 5 repetitions • 2 x 3 x 2 x 5 - 3 discarded = 57 utterances
Data Preparation • Sampled F0 curves have to be turned into functions • A basis of functions (B-splines) expresses each original curve • Decide how much detail to retain (smoothing)
Data Preparation (2) • Landmark registration • Align points in time that are deemed as having the same meaning across the dataset
PC1 x PC2 x x x x x x x x x x x x x x x x x x salary x x x x x x x x x x x x x x x x x 25 65 age ClassicPrincipal Component Analysis (PCA)
PC-based signal reconstruction + 1.65 x - 0.46 x mean(t) PC1(t) PC2(t)
Conclusions • A data driven approach is possible in the exploration of intonation phenomena • FDA provides automatic tools to describe variation in a set of pitch contours extracted from real utterances • provided that the relevant landmarks are annotated • The same tools allow to construct artificial contours with desired perceptual characteristics • Smooth and global variation are applied • Variations come from a statistical analysis of data • The process is automatic