1 / 34

Functional Data Analysis for Speech Research

Functional Data Analysis for Speech Research. Michele Gubian Radboud University Nijmegen The Netherlands London, March 24 th 2010 Cambridge, March 26 th 2010. Content. What and why Functional Data Analysis (FDA) Motivation Case study 1 Case study 2 – pitch re-synthesis

shawna
Download Presentation

Functional Data Analysis for Speech Research

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Functional Data Analysis for Speech Research Michele Gubian Radboud University Nijmegen The Netherlands London, March 24th 2010 Cambridge, March 26th 2010

  2. Content • What and why Functional Data Analysis (FDA) • Motivation • Case study 1 • Case study 2 – pitch re-synthesis • How to use FDA • Using the R package ‘fda’

  3. Motivation

  4. dur ext 58 48 98 … 2.8 3.8 2.9 … x ext ? x x x dur Analyzing curves PCA ANOVA Linear models

  5. x ext ? x x x dur Problems • Decide what are the important features of a curve using • models • intuition / trial and error However • Those features may not capture all the relevant dynamic aspects • e.g. concavity/convexity • long range correlatioins

  6. x ext ? x x x dur Problems (2) • Identify those feature points • manually • (semi)automatically However • The identification may be hard, even ill-posed • time consuming • risk of subjective judgment

  7. x ext ? x x x dur Analyzing curves with FDA Functional Data Analysis

  8. Analyzing curves with FDA • All the information contained in the curve (dynamics) is used • No need to reduce a curve to a set of significant features • No need to introduce assumptions on what is relevant in a curve shape and what is not • FDA provides both VISUAL and QUANTITATIVE results • input is curves, output is also curves • plus classic statistical output like p-values, confidence intervals …

  9. Example x x x salary salary x x x x x age age Functional Data Analysis: an extension of (some) statistical techniques to the domain of functions CLASSIC FDA • Ask people: • How old are you? • How much do you earn? • Each data point is a point in 2D • Record people salary through the years • Each “data point” is a whole CURVE

  10. Case study

  11. Diphthong vs. hiatus in Spanish • /ja/ vs. /i.a/ contrast is unstable in European Spanish • Diachronically, in Romance languages /i.a/ becomes /ja/ • Diatopically, in Latin American Spanish the contrast seems to be lost • It is not present in orthography (“ia” in either case) • No strict minimal pairs • Investigate • Consistent realization of the contrast • Inter-speaker variation • Cues used in the realization

  12. Cues DIPHTHONG /ja/ HIATUS /i.a/ • Duration • Formants • Pitch short long f2 f2 f1 f1 f0 f0

  13. Example diphthong

  14. Example hiatus

  15. Dataset • Read speech • Diphthong ‘Emiliana no, …’ /e.mi.lja.na#no#.../ (‘Not Emiliana, …’) • Hiatus ‘Mi liana no, … ‘ /mi#li.a.na#no#.../ (‘Not my liana, …’) • 9 speakers (gender balanced) • 20 repetitions per speaker per type • In total 365 utterances

  16. Duration

  17. lja li a Pitch • Pitch was extracted from the beginning of /l/ to the end of the rising gesture • In Spanish the pitch rising peak falls beyond the accented syllable

  18. speaker /ja/ vs /i.a/ The raw data

  19. FDA data preparation • Each sampled curve has to be turned into a function • Decide how much detail to retain (smoothing)

  20. B-spline FDA data preparation (2) • All functions will be obtained by a combination of so-called basis functions, usually B-splines • All functions will be linearly stretched in time to become of equal duration Functional representation

  21. PC1 x PC2 x x x x x x x x x x x x x x x x x x salary x x x x x x x x x x x x x x x x x 25 65 age ClassicPrincipal Component Analysis (PCA)

  22. Functional PCA on pitch contours

  23. Functional PCA on pitch contours PCA does not know about labels !!

  24. Functional PCA on pitch contours PC1

  25. Functional PCA on pitch contours PC1

  26. Functional PCA on pitch contours PC2

  27. PC2 Functional PCA on pitch contours

  28. f1 f2 PC1 PC2 Functional PCA on formants

  29. Functional PCA on formants PC1 PC1

  30. Duration vs formants Duration vs pitch Cues coordination

  31. Summary • FDA provides tools to extract relevant dynamic characteristics of a set of curves • Traditional tools like PCA (and linear regression) are extended to curves • Functional PCA revealed the main dynamic cues used in the realization of a (weak) contrast in Spanish • Without using the labels information • Without extracting features from the curves (e.g. peaks) • Combining multi-dimensional curves (formants) without effort

  32. References • Functional Data Analysis website: www.functionaldata.org • Books: • Software: a bilingual (R and MATLAB) tool is freely available online

  33. Appendix

  34. Functional linear models y(t) = a(t) + b(t) x diphthong, x = 0 hiatus, x = 1 Confidence intervals for a(t) and b(t) R2(t) = percentage of explained variance

More Related