SpeakGreek: Computer-Assisted Teaching of Vowel Production in Greek L2 Learning and Speech Disorders

1. speakgreek.web.auth.gr Computer assisted teaching of vowel production to learners of Greek as an L2 and individuals with speech disorders Nicolaidis, K., Papanikolaou, G., Sfakianaki, A., Kainada, E., Avdelidis, K., Konstantoudakis, K. Phonetics Laboratory & Laboratory of Electroacoustics & TV systems School of English & Dept. of Electrical and Computer Engineering Aristotle University of Thessaloniki ΑRISTEIA ΙΙ

2. Aims of the presentation • Basic structure of “SpeakGreek” • Background • Speech databases • Analysis of vowels • Application for the training of vowels • Work in progress

3. SpeakGreek online biofeedback speech training tool • applications in education and in clinical intervention • training in the perception and production of segmental and suprasegmental aspects of Greek • Assessment: real time feedback, ASR • addressed to • learners & teachers of Greek as a foreign or second language (L2) • individuals with articulation and phonation problems and speech and language therapists working with these clinical populations • speech scientists • interested users

4. Development of SpeakGreek multidisciplinary framework which draws from • theoretical and empirical research in phonetics/phonology • current methodological approaches for pronunciation training and speech intervention • current advances in speech technology applications including automatic speech recognition

5. Bio-feedback utilization of speech technologies for extra or alternative training in the L2 classroom and in speech therapy • Biofeedback: the real-time moment-to-moment information about a physiological event • Real time spectrograms • Real time pitch contours • Real time displays of vowel production on an F1xF2 vowel space • Real time lingual movement or contact with palate (UTI, EPG) Applications • L2 pronunciation teaching Chun, 2007 • remediation of numerous speech disorders, e.g., hearing impairment, articulation, fluency and voice disorders Brooks et al., 1981, Maryn et al., 2006

6. SpeakGreek: δομή Speech training tool Phonetic Library Applications

7. Applications \ Basic training pitch, loudness, phonation duration, voicing The sounds of my language The melody of my language Perception Production

8. SpeakGreek: speech databases Database Α:audio visual data of Greek consonants, vowels, stress, intonation • Video recording of 3 Greek speakers 550 items • EPG and UTI data for a subset of speakers 208 items Use: Phonetic Library Database Β:acoustic data from 60 Greek speakers • Consonants, vowels, stress, intonation Use: • Study of target productions and variability • Selection of parameters for biofeedback • Data for ASR 132.560 items Database C:acoustic data from 4 speakers • Syllables, words with target sounds, minimal pairs (isolated, contextualised), sentences 27.000 items Use: applications for perception and production training

9. Vowel training Greek vowel system • simple 5 vowel system /i, e, a, o, u/ Cross-linguistic errors • Substitution, e.g. Arabic: /e/[i], /o/ [u] • Centralisation of unstressed vowels, e.g. English Speech disorders • Substitution, e.g. /e/  [a] • Centralisation, neutralisation of contrasts

10. // // // // // // Greek vowels Nicolaidis & Sfakianaki (submitted) Baltazani (2007), Botinis (1981, 1989), Dauer (1980a, b), Fourakis (1986), Fourakis, Botinis, and Kastaiti (1999), Jongman, Fourakis, and Sereno (1989), Lengeris (2012), Nicolaidis (2003), Sfakianaki (2002); review by Arvaniti (2007) There is paucity of studies on gender differences in child speech in languages other than English.

11. Methodology • Acoustic data from • 20 male adults (8 speakers for this study) • 20 female adults (11 speakers for this study) • 20 children aged 8-10 (10 male, 10 female) (6 speakers) • Recordings: sound-treated booth, Phonetics Laboratory, English Department • Sennheiser microphone ME66/K6 • 5 repetitions of all material from adult speakers • 3 repetitions from children

12. Speech material • Real CVCV words • carrier /'lee …… pa'du/ “ say ____ everywhere” • C1= /p, t, k, b, d, g, f, , , z, x, , m, n, , l, , / • C2= coronal consonant • V= /i, e, a, o, u/ • Stress on first syllable • mostly symmetrical, when not possible, V2 selected to have similar features to V1 • e.g. pata, pini, pede, pono, pula

13. Vowel analysis • Automatic segmentation • Manual check of segmental boundaries • Automatic formant tracking using PRAAT • F1, F2 at the vowel midpoint of stressed vowel • Statistical analysis using R Tokens analysed: • Males: 1058 • Females: 3412 • Children: 1666 (830 male and 836 female) • Total: 6136

14. Vowel spaces: adults Statistically significant differences in the F1 of /i/, /e/ and /a/, and F2 of /i/, /e/, /o/ and /u/ for male and female speakers. Larger vowel space for female than male adults Nicolaidis & Sfakianaki submitted, Simpson, 2009, Weinrich & Simpson, 2014 Overlap between back vowels /u/ and /o/

15. Vowel spaces: children Child vowel system pattern similar with the adult one Chung et al., 2012 Higher acoustic variability for children than adults Eguchi & Hirsh, 1969; Vorperian & Kent, 2007; Chung et al., 2012 Statistically significant differences, as in adults, in the F1 of /i/, /e/ and /a/, and F2 of /i/, /e/, /o/ and /u/ for male and female children cf., Bennett, 1981; Busby and Plant,1995; Whiteside and Hodgson, 2000; Vorperian & Kent, 2007 for English Larger vowel space for female than male children Vorperian & Kent, 2007 Small degree of overlap between several vowels

16. Vowel duration Longer duration by female than male speakers for both age groups Hillenbrand et al., 1995; Simpson, 2009; but see also Jacewicz et al., 2007 Longer duration by children than adults Lee et al., 1999 for English speaking children 5-6 yrs but not >7 yrs Expected intrinsic duration differences: /i/ shortest for all groups, /a/ longest for all groups except male children Fourakis et al., 1999

17. Application: design • F1/F2 plot • Real time feedback • Formant frequencies in Hz • Tongue height and position • User can select a target vowel • Background information on vowel articulation in the phonetic library • Customized vowel spaces for adult/children, male/female • User selection of graphics: plain vs. animated graphics editions

18. Application: design • Real-time analysis using moving windows • Pre-emphasis filtering to tone down low frequencies • Formant frequencies derived from peaks of LPC spectrum • Burg’s algorithm • 16 poles • Peak stability enhanced by exponential moving average • Peaks that remain at constant frequencies over several instances are given more weight • Extremely short-lived peaks are discarded

19. Challenges • Phonetic • vowel target overlap • vowel variability • Technological • LPC spectrum peaks at non-formant frequencies • F3 of /u/ mistaken for F2 of/i/ • Solutions: • Appropriate selection of number of poles for LPC • Exponential moving average for peak detection • Robustness in noise

20. SpeakGreek: innovations • Based on research findings of a large database • development of novel approaches to speech processing and analyses • Free online availability of the tool • Independent use by the client or together with the teacher/clinician • User-friendly interface • User selection of graphics: plain vs. animated graphics editions • Assessment of the tool for future improvements

21. Acknowlegments Thanks to • Eirini Kelmali, Dionysia Saratsli, Melina Charta • Julia Starchenco, Lazaros Vrysis • Tasos Pasxalis • All subjects that were recorded for the database

22. References • Arvaniti, A. (2007). Greek Phonetics: The State of the Art. Journal of Greek Linguistics,8, 97-208. • Baltazani, M. (2007). Prosodic rhythm and the status of vowel reduction in Greek. Selected Papers on Theoretical and Applied Linguistics from the 17th International Symposium on Theoretical and Applied Linguistics, 1, 31-43. Department of Theoretical & Applied Linguistics, Aristotle University of Thessaloniki. • Bennett, S. (1981). Vowel formant frequency characteristics of preadolescent males and females. Journal of the Acoustical Society of America, 69, 231-238. • Botinis, A. (1981). The Greek Vowels. Working Papers Linguistics-Phonetics Lund University 21, 29–36. • Botinis, A. (1989). Stress and Prosodic Structure in Greek: A Phonological, Acoustic, Physiological and Perceptual Study. Lund: Lund University Press. • Brooks, S., Fallside, F., Gulian, E., & Hinds, P. (1981). Teaching vowel articulation with the Computer Vowel Trainer: Methodology and results. British Journal of Audiology,15, 151-163. • Busby, P. A., & Plant, G. L. (1995). Formant frequency values of vowels produced by preadolescent boys and girls. Journal of the Acoustical Society of America, 97, 2603- 2606.

23. Chun, D.M. (2007). Technological advances in researching and teaching phonology. In M.C. Pennington (Ed.), Phonology in context. 135-158. Basingstoke: Palgrave Macmillan. • Chung, H., Jong Kong, E., Edwards, J., Weismer, G., Fourakis, M., & Hwang, Y. (2012). Crosslinguistic studies of children’s and adults’ vowel spaces. Journal of the Acoustical Society of America, 131, 442-454. • Dauer, R. M. (1980a). Stress and Rhythm in Modern Greek. Ph.D. dissertation, Universityof Edinburgh. • Dauer, R. M. (1980b). The Reduction of Unstressed High Vowels in Modern Greek. Journalof the International Phonetic Association, 10, 17–27. • Eguchi, S. & Hirsh, I. J. (1969). Development of speech sounds children, Acta OtoLaryngol.Supplement 257, 1-51. • Fourakis, M. (1986). An Acoustic Study of the Effects of Tempo and Stress on SegmentalIntervals in Modern Greek. Phonetica, 43,172–188. • Fourakis, M., Botinis, Α., & Katsaiti, Μ. (1999). Acoustic characteristics of Greekvowels. Phonetica, 56,28–43.

24. Hillenbrand, J., Getty, L. A., Clark, M. J. & Wheeler, K. (1995). Acoustic characteristics of American English vowels. Journal of the Acoustical Society of America, 97(5), 3099-3111. • Jacewicz, E., Fox, R. A., & Salmons, J. (2007). Vowel Duration in Three American English Dialects. American Speech, 82(4), 367–385. • Jongman, A., Fourakis, M., & Sereno, J. A. (1989). The Acoustic Vowel Space of Modern Greek and German. Language and Speech, 32, 221–248. • Lee, S., Potanianos, A., & Narayanan, S. (1999). Acoustics of children’s speech: Developmental changes of temporal and spectral parameters. Journal of the Acoustical Society of America, 105, 1455-1468. • Lengeris, A. (2012). Phonetic vowel reduction in Standard Modern Greek. In Z. Gavriilidou, A. Efthymiou, E. Thomadaki & P. Kambakis-Vougiouklis (eds), Selected papers of the 10th International Conference on Greek Linguistics, Komotini/Greece: Democritus University of Thrace, 401-407. • Maryn, Y., de Bodt, M. & van Cauwenberge, P. (2006). Effects of biofeedback in phonatory disorders and phonatory performance: A dystematic literature review. Applied Psychophysiology and Biofeedback, 31(1), 65-83.

25. Nicolaidis, K. (2003). Acoustic variability of vowels in Greek spontaneous speech. Proceedings of the 15th International Congress of Phonetic Sciences, 3221-3224. Universidad Autónoma de Barcelona. • Nicolaidis, K. , & Sfakianaki, A. Acoustic characteristics of vowels produced by Greek intelligible speakers with profound hearing impairment I: Examination of vowel space. Submitted to the International Journal of Speech-Language Pathology. • Sfakianaki, A. (2002). Acoustics Characteristics of Greek Vowels Produced by Adults and Children. In M. Makri-Tsilipakou (ed) Selected Papers on Theoretical and Applied Linguistics from the 14th International Symposium, 383–394. Thessaloniki: Department of Theoretical and Applied Linguistics, School of English, Aristotle University of Thessaloniki. • Simpson, Adrian P. (2009). Phonetic differences between male and female speech. Language and Linguistics Compass 3(2), 621-640. • Vorperian, H.K. & Kent, R.D. (2007). Vowel acoustic space development in children: a synthesis of acoustic and anatomic data. Journal of Speech, Language & Hearing Research, 50(6),1510-1545. • Weirich, M. & Simpson, A.P. (2014). Differences in acoustic vowel space and the perception of speech tempo. Journal of Phonetics, 43, 1-10. • Whiteside, S.P. & Hodgson, C. (1998). The development of fundamental frequency in 6 to 10 year old children: A brief study. Journal of the International Phonetic Association, 28, 55-62.