1 / 20

A bilingual corpus-based TTS system as a foreign language learning tool

Lehlohonolo Mohasi Supervisors: Prof. Mqhele E. Dlodlo Prof. Mbulungeni Madiba Co-Supervisor: Prof. Daniel Mashao. A bilingual corpus-based TTS system as a foreign language learning tool. Outline. Current Situation Problem Solution Evaluation Conclusion Future Work. 2.

gwendolyn-winters
Download Presentation

A bilingual corpus-based TTS system as a foreign language learning tool

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Lehlohonolo Mohasi Supervisors: Prof. Mqhele E. Dlodlo Prof. Mbulungeni Madiba Co-Supervisor: Prof. Daniel Mashao A bilingual corpus-based TTS system as a foreign language learning tool

  2. Outline Current Situation Problem Solution Evaluation Conclusion Future Work 2

  3. Current Situation I don’t think so. Andiphilile neze. Ana u ea sekolong tsatsing lee? 3

  4. Current Situation Language Acquisition How babies learn to talk & understand language Exposure to spoken language Reading & writing learnt through instruction Second/Foreign Language Learning Not easily acquired Need voice exposure as well + instruction Different instructional methods 4

  5. Problem Not enough voice exposure for L2 or foreign language learners. There might be exposure but there is also a need for ability to read and write for various reasons. Use of correct grammar & pronunciation in written and spoken languages. 5

  6. Problem – Research Question How can technology be used as an effective second/foreign language learning tool?

  7. Solution Human Language Technologies/Information Communication Technology Proposed technology -> Text-to-speech (TTS) as a second/foreign language learning tool What is TTS? Automatic conversion of written text into speech. 7

  8. Phonemes Prosody Solution - TTS NLP Text Analysis LTS Rules Prosody Generation DSP Concatenation-based Synthesis Waveform Generation Speech Text 8

  9. Solution - TTS • TTS Applications • Telecommunications • Visually-impaired • Deaf and Vocally Handicapped • Educational Applications • What value is TTS? • Easy voice access mode to all • Useful for the blind and illiterate • Easy way of improving language listening & speaking skills • Helps with correct pronunciation 9

  10. Solution – Language & TTS So how do we make sense of text-to-speech technology implementation in foreign language learning? 10

  11. Solution – Language & TTS

  12. Solution - Methodology • Scope • Use of 3 SA official languages – English, Sesotho & isiXhosa. • Multiconcord – Mconsal • Festival Speech Synthesis Engine • Parallel corpora selection from 1994 to date • 1 million words

  13. Solution - Methodology • Phase 1 – Creating corpora • Creating a parallel corpora database • Tools • BITS (Bilingual Internet Text Search) • Strand • Scanner • Multiconcord • MinMark

  14. Solution - Methodology • Phase 2 – TTS implementation • TTS database taken from parallel corpora • Limited domain synthesis • Tools • Festival Speech Synthesis Engine • Phase 3 – Web interface • Integration of Phases 1 & 2 • Tools • PHP • mySQL

  15. Evaluation • Acoustic measurements – voice talent selection • Fundamental frequency, F0 • RMS energy for voiced and unvoiced speech • Long-term spectra • Vowel formants and bandwidths • Target costs • Concatenation costs • Functional testing - TTS-system assessment on performance for communicative purpose (intelligibility measure) 15

  16. Evaluation • MOS (Mean Opinion Score) - general speech quality evaluation • Pleasantness * Pronunciation • Articulation * Understandability • Sound quality * Intelligibility • Listening effort

  17. Evaluation • Groups of interest • UCT staff (n=75) • UCT students (n=75) • Medical (Rehabilitation & Doctors) • Law • Other streams • Exposure to voice access & non-exposure • Monitor over time • Results obtained from assessment exams • Use age as one variable 17

  18. Conclusion Text-to-speech technology is a convenient and support tool - possible solution. With an increasing recognition for demographics - need for engaged learners to take full responsibility for their language learning. There is also a need for teachers who are not only language experts but who are also trained in the use of technology and who can facilitate foreign language learning. 18

  19. Future Work • Multilingual Speech to Speech • Speech to speech takes speech processing a step further towards a voice-only computer interface. • Incorporates ASR & TTS • Enables computing devices to first understand spoken language, then analyze the utterance through natural language processing, and finally formulate and utter a response through synthesized speech. 19

  20. Thank You QUESTIONS?? SUGGESTIONS?? 20

More Related