Martin Duckworth, Kirsty McDougall, Gea de Jong, Linda Shockey

1. The Reliability of Formant Measurements in High Quality Audio Data: The Effect of Agreeing Measurement Procedures Martin Duckworth, Kirsty McDougall, Gea de Jong, Linda Shockey DyViS

2. Introduction • Formant measurement implicitly required legally in the UK in speaker comparison cases • Measurements on analogue spectrograms had to be by hand and eye • Measurements on digital spectrograms can be assisted by formant trackers, LPC is common

3. Introduction • How replicable are measurements by eye on digital spectrograms?

4. Introduction • How replicable are measurement by eye on digital spectrograms? • If LPC tracking is used what can lead to variability?

5. Introduction • How replicable are measurement by eye on digital spectrograms? • If LPC tracking is used what can lead to variability? • Software settings

6. Introduction • How replicable are measurement by eye on digital spectrograms? • If LPC tracking is used what can lead to variability? • Software settings • Point at which data is extracted

7. Study Aims • What is required in order to make measurements more replicable?

8. Study Aims • What is required in order to make measurements more replicable? • If software (but not method) is held constant and data is high quality, can different laboratories make the same F1-3 measurements?

9. Study Aims • What is required in order to make measurements more replicable? • If software (but not method) is held constant and data is high quality, can different laboratories make the same F1-3 measurements? • If method of analysis is the same does this lead to statistically improved reliability between laboratories?

10. Aims continued • We are aiming to find a reliable means of obtaining formant values • We are examining reliability, not validity

11. Data • read speech from Cambridge DyViS database • male • Standard Southern British English • aged 18-25 • 40 speakers: Set 1 (20 speakers) Set 2 (20 speakers)

12. Data • 6 monophthongs: / iː, æ, ɑː, ɔː, ʊ, uː / • 6 repetitions per vowel per speaker • elicited in hVd contexts in sentences: It’s a warning we’d better HEED today. It’s only one loaf, but it’s all Peter HAD today. We worked rather HARD today. We built up quite a HOARD today. He insisted on wearing a HOOD today. He hates contracting words, but he said a WHO’D today.

13. Measurements • Analysts from 3 labs – Cambridge, Plymouth, Reading • Task: to measure F1, F2, F3 for each vowel token using Praat • Set 1 – using individual – but constrained- methods • Set 2 – after a meeting at which a single method is agreed

14. Set 1 Methods • Measure the formants at a relatively early point in the vowel

15. Set 1 Methods • Measure the formants at a relatively early point in the vowel • Measure formants over no more than 5 glottal pulses

16. Set 1 Methods • Measure the formants at a relatively early point in the vowel • Measure formants over no more than 5 glottal pulses • Use either: • LPC tracking checked against the spectrogram or

17. Set 1 Methods • Measure the formants at a relatively early point in the vowel • Measure formants over no more than 5 glottal pulses • Use either: • LPC tracking checked against the spectrogram or • hand/eye measures

18. Set 2 Method • Measure towards the start of the vowel

19. Set 2 Method • Measure towards the start of the vowel • Measure in a relatively steady early part of the vowel

20. Set 2 Method • Measure towards the start of the vowel • Measure in a relatively steady early part of the vowel • Measure around the vowel's maximum intensity

21. Set 2 Method • Measure towards the start of the vowel • Measure in a relatively steady early part of the vowel • Measure around the vowel's maximum intensity • Use a single time slice

22. Set 2 Method (continued) • Use the LPC formant tracker adjusted for best visual fit

23. Set 2 Method (continued) • Use the LPC formant tracker adjusted for best visual fit • When values generated by Praat are judged by visual inspection to be incorrect, replace them by correct values from a time-slice immediately preceding or following the slice being measured.

24. Results: HAD, F1 Set 1 Lab1 Lab2 Lab3

25. Results: HAD, F1 Set 1 Lab1 Lab2 Lab3

26. Results: HAD, F1 Set 1 Set 2 Lab1 Lab2 Lab3 Lab1 Lab2 Lab3

27. Results: HAD, F1 Set 1 Set 2 Lab1 Lab2 Lab3 Lab1 Lab2 Lab3

28. Statistical Analysis • 3 formants  6 vowels  2 datasets = 36 tests • Two-way ANOVA - repeated measures on the factor Lab (3) - between-groups factor Speaker (20) • If Lab signficant at p < 0.05:Pairwise comparisons with Sidak correction

29. Results: HAD, F1 Set 1 Set 2 Lab1 Lab2 Lab3 Lab1 Lab2 Lab3

30. Results: HAD, F1 Set 1 Set 2 Lab1 Lab2 Lab3 Lab1 Lab2 Lab3 Lab: significant

31. Results: HAD, F1 Set 1 Set 2 0.001 0.000 0.000 Lab1 Lab2 Lab3 Lab1 Lab2 Lab3 Lab: significant

32. Results: HAD, F1 Set 1 Set 2 0.001 0.000 0.000 Lab1 Lab2 Lab3 Lab1 Lab2 Lab3 Lab: significant but pairwise comparisons NS Lab: significant

33. Results: HAD, F1 Set 1 Set 2 0.001 0.000 NS 0.000 NS NS Lab1 Lab2 Lab3 Lab1 Lab2 Lab3 Lab: significant but pairwise comparisons NS Lab: significant

34. Results: HAD, F2

35. Results: HAD, F2 Set 1 Set 2 NS NS NS NS NS NS Lab1 Lab2 Lab3 Lab1 Lab2 Lab3 Lab: not significant Lab: not significant

36. Results: HAD, F3

37. Results: HAD, F3 Set 1 Set 2 NS 0.000 NS 0.000 NS NS Lab1 Lab2 Lab3 Lab1 Lab2 Lab3 Lab: not significant Lab: significant

38. Summary - HAD Set 1 Set 2

39. Summary - HAD main effect Set 1 Set 2

40. Summary - HAD Set 1 Set 2 pairwise comparisons

41. Summary - HAD Set 1 Set 2

42. Summary - HAD Set 1 Set 2 improvement

43. Summary - HAD Set 1 Set 2

44. Summary - HAD Set 1 Set 2

45. Summary - HAD Set 1 Set 2 improvement

46. Summary - HAD Set 1 Set 2

47. Summary - HAD Set 1 Set 2 Set 2: good news

48. Effect of Lab - 6 vowels Set 1

49. Effect of Lab - 6 vowels Set 1 Set 2

50. Influence of Speaker • Interaction Lab x Speaker significant (p < 0.05) for F1-F3 of all 6 vowels for both Set 1 and Set 2 • certain speakers lead to measurement differences among labs for example…