The SPACE project: Speech Algorithms for Clinical and Educational Applications

1. The SPACE project:Speech Algorithms for Clinical and Educational Applications Hugo Van hamme SPACE symposium Antwerp

2. Outline • partners • What is it about ? • Why does it make sense ? • The educational component • The clinical component • Challenges – examples • The technologies: foreground and background • The first 6 months The SPACE project

3. Partners • K.U.Leuven - ESAT: coordinator - speech recognitionProf. Hugo Van hamme • R.U.Gent – ELIS: speech recognitionProf. Jean-Pierre Martens • V.U.Brussel – ETRO: text-to-speechProf. Werner Verhelst • K.U.Leuven – ORTHO: disability, special needs education and child careProf. Pol Ghesquière • U.Antwerpen: communication disordersProf. Marc De Bodt The SPACE project

4. In touch with the field … • user group • Technology providers • ScanSoft • Technology users • Technology & Integratie • Artec • eXplio • Interest groups • Stichting Integratie Gehandicapten (SIG) • Modem • this symposium The SPACE project

5. What ? • Speech technology • automatic speech recognition (ASR) • speech synthesis (TTS) • Clinical and educational • Speech therapy related. • Speech assessment • Adapt technology • To suit requirements of the applications • Demonstrate usefulness of technology • Automation of existing methods • New methods enabled by the technology • Interdisciplinary The SPACE project

6. Why ? • spoken interaction with the computer comes naturally • Unlike many other applications of ASR/TTS • Similar characteristics language learning • pre-assessment in 2003 • social relevance • role of universities • large group of beneficiaries • persons with dyslexia • reading skill development of all primary school pupils • deaf, communication disorders The SPACE project

7. Why ? (2) • other applications possible • language learning and language proficiency assessment • training of professional speakers • pronunciation training and stutter therapy • E-learning • technology improvements applicable in other areas: • HMI with voice mode • entertainment The SPACE project

8. Some background • Project sponsor: IWT • Instituut voor de aanmoediging van innovatie door Wetenschap & Technologie in Vlaanderen • SBO: Strategisch BasisOnderzoek • 4 years: March 1, 2005 – February 28, 2009 • 28 person-years total effort • This symposium is co-sponsored by the Nederlandse Taalunie The SPACE project

9. Domain of interest 1 Automated reading assessment and remedial practice • reading tutor • replace human supervision in current diagnostic practice and in therapy • make assessment objective and repeatable • explore new strategies for diagnosis and remedy, enabled by speech technology • use: • automate diagnosis of dyslexia => early detection • a program that helps you develop your reading skill • increase intensity (and effectiveness) of therapy • AVI reading tests in primary schools The SPACE project

10. Domain of interest 2 Clinical applications for speech assessment • clinical practice • perceptual evaluation • subjective tests of articulation • interrater and intrarater disagreements • use articulatory speech analysis • compare to human judgement • reference database • determine type and degree of error The SPACE project

11. The challenge - examples • reading tutor: • mis-pronunciation • Immediate auditive feedback (cues) • assessment: mis-articulation The SPACE project

12. Hestiations, unwanted speech Joep rijdt op zijn fiets door de straat. Het is een mooie gele fiets.Die heeft hij voor zijn verjaardag gekregen. Er zit een grote glimmende bel op. The SPACE project

13. The technology • background: • large vocabulary speech recognizer (ESAT) • voice assessment, pronunciation modelling (ELIS) • text-to-speech and voice modification (ETRO) • requirements • accurate assessment of utterance • acceptance/rejection • Fine-grained analysis/feedback • speech representations that give articulatory insight • modelling of imperfect speech: • mis-articulations • mis-pronunciations • at phoneme, word or sentence level • feedback and guidance through TTS The SPACE project

14. Approaches: acoustics • optimize acoustic models for children • model the disfluencies • non-phonemes • articulatory analysis of speech • voicing, high/low, lip rounding, … • estimated from wave form • relevant for articulation assessment • accurate phonetic classification • phonetic hypotheses generated in phoneme lattice • phoneme-specific features and tests added The SPACE project

15. Approaches: miscues • lexical mispronunciation models • exploit prior knowledge on reading mistakes • orthography • frequency: rare words substituted by common • semantics: read-by-guessing strategy • data driven: at word level or by transformation rules • sentence level misreading models • hestitations, restarts … The SPACE project

16. Approaches: TTS • TTS for • providing pronunciation examples • providing reading cues • synchronised reading • special reading mode speech synthesis • spelling mode (letter/phoneme) • syllable mode (isolated/lengthened) • extremely slow speech • special stress patterns The SPACE project

17. Where are we ? • articulatory speech analysis • data collection: • dyslalia, dysarthria, hearing loss • reading exercises: content, tools • TTS: public domain software analysis • reading tutor prototype • children’s acoustic model • track reading progress • model for word skips and restarts • model for unintended speech • model for lexical errors: swap of letters, phoneme substitution … The SPACE project

18. conclusion • the SPACE project • has challenging objectives • interdisciplinary • will deepen insights in new speech modelling approaches • will develop prototypes in both application areas • has mainly a social relevance, also • economic spin-off activities possible • improvement in accuracy and robustness of ASR • additional speaking modes and synchronisation in TTS The SPACE project