1 / 40

Proposals for Extending SSML 1.0 from the Point-of-View of Hungarian TTS Developers

Proposals for enhancing SSML 1.0 for Hungarian speech synthesis, enabling better text structure analysis, phoneme conversion, and prosody prediction.

annot
Download Presentation

Proposals for Extending SSML 1.0 from the Point-of-View of Hungarian TTS Developers

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Proposals for Extending SSML 1.0 from the Point-of-View of Hungarian TTS Developers Géza Németh, Géza Kiss, Bálint Tóth Laboratory of Speech Technology, Department of Telecommunications and Media Informatics Budapest University of Technology and Economics, Budapest, Hungary {nemeth,kgeza,toth.b}@tmit.bme.hu

  2. Budapest University of Technology & Economics (BME)Dept. of Telecommunications & Media Informatics (TMIT) • Speech activities: • Coordinator: Gordos Géza D.Sc. Speech Technology Lab(STL) Németh Géza andOlaszy Gábor PhD D.Sc. Telecommunications & Signal Processing Lab(TSP) Tatai PéterMSc Laboratory of Speech Acoustics Vicsi Klára (LSA) D.Sc. • In each lab • 4-6 PhD students • Graduate students • 306 in Speech Information Systems subject (2005)

  3. Basic research • Multi-lingual artificial speech generation (synthesis, STL) • limited vocabulary (e.g., numbers, date, address) • multi-lingual TTS (Hungarian, German, Polish, Spanish) • speech profiles (variability, individual features) • expression/emotion presentation (user’s manual <-> news) • Speech recognition (TSP, LSA) • noise handling (telephone, in-car, ..., TSP) • dictation (good quality, continouos, LSA) • audio indexing (e.g. radio archives, broadcast news, TSP) • speech segmentation (TSP, LSA) • emotion detection (TSP) • Speech understanding (TSP) • Speech databases (LSA, TSP)

  4. Applied Research • Fully proprietary components and solutions: • All parameters controlled, systems are tailor-made for the end-user, Integration of original research results, unique products • T-Mobile Hungary services: E-mail reader 1999-, name- and address reader in reverse directory, 2003 (Motto: Why is the human operator speaking, not the machine?!), Symbian SMS-reader 2002- (STL) • Others: SMS reader 2001-, bookreader 2002-, (STL) • Voice portals (Generali Hungary name dial-in 2004, Hungarian VoiceXML browser, 2003, TSP+STL) • Industrial information systems (STL, TSP) • Unified Messaging (STL) • Call Center (STL, TSP) • Audio user interfaces (especially portable/mobile devices, car information systems, wearable devices, STL, TSP) • Disability (1986-, speech, vision, Hungarian version of Jaws for Windows, notetaker for blind people, STL, TSP, LSA)

  5. Contact information Tel: (+36 1) 463-38-83 Fax: (+36 1) 463-31-07 http://speechlab.tmit.bme.hu email: nemeth@tmit.bme.hu

  6. Overview Text structure Text-to-phoneme conversion Text normalization Prosody prediction Prosody prescription Summary

  7. Overview Text structure Text-to-phoneme conversion Text normalization Prosody prediction Prosody prescription Summary

  8. Overview Text structure Text-to-phoneme conversion Text normalization Prosody prediction Prosody prescription Summary

  9. Overview Text structure Text-to-phoneme conversion Text normalization Prosody prediction Prosody prescription Summary Text structure elements already contained in SSML 1.0: • paragraph • sentence Suggested further structuring: • word • syllables

  10. Overview Text structure Text-to-phoneme conversion Text normalization Prosody prediction Prosody prescription Summary This can be used • to help • text-to-phoneme conversion • prosody prediction and prescription • … by giving higher level information, namely • syllable structure • part-of-speech information (Examples given later) • to indicate words in languages that do not use space to separate words

  11. Overview Text structure Text-to-phoneme conversion Text normalization Prosody prediction Prosody prescription Summary Reasons to use text structure elements instead of e.g. phoneme, prosody, break, emphasis • Easier for human editor to add • Replacing synthesis processor may necessitate rewriting • phoneme specification • prosody prescription

  12. Overview Text structure Text-to-phoneme conversion Text normalization Prosody prediction Prosody prescription Summary Suggested word element <w [syllables=“…-…”][POS=“…” [number=“…” …]]> … </w> E.g. <w syllables="hosz-szú"> hosszú </w> <w POS="noun" number="plural" case="accusative"> halászsasokat </w>

  13. Overview Text structure Text-to-phoneme conversion Text normalization Prosody prediction Prosody prescription Summary Suggestion extended from other proposals <w [syllables=“…-…”][POS=“…” [number=“…” gender=“…” case=“…” …][morph=“…+…”][tone=“h+l+…”]]> … </w> When not a word, but an expression is labeled: <e [POS=“…” [number=“…” …]> … </e> E.g. three kilos <e POS=“cardinal” number=“plural” gender=“neuter” case=“genitive”]> 3 k. </e>

  14. Overview Text structure Text-to-phoneme conversion Text normalization Prosody prediction Prosody prescription Summary

  15. Overview Text structure Text-to-phoneme conversion Text normalization Prosody prediction Prosody prescription Summary When pronunciation cannot be determined, you can • Add a lexicon elementBUT hard to add all • Specify using phoneme:BUT hard to write & read for human • Add a textual replacement using sub • Provide higher level information Currently this is only say-as

  16. Overview Text structure Text-to-phoneme conversion Text normalization Prosody prediction Prosody prescription Summary Other types of higher level information (easier, more natural) • Syllable structure • Part-of-speech information • Language of included foreign text We are going to give you some examples.

  17. Overview Text structure Text-to-phoneme conversion Text normalization Prosody prediction Prosody prescription Summary Syllable structure Hungarian: • highly agglutinative • pronunciation inference rules are used • rules can be tricked by some words E.g. “egészség” (“health”) Letter combinations might be “s+zs” [S]+[Z]→[Z] but they are in fact “sz+s” [s]+[S]→[S]

  18. Overview Text structure Text-to-phoneme conversion Text normalization Prosody prediction Prosody prescription Summary Syllable structure Enough to know syllable structure. Instead of <phoneme alphabet="ipa" ph="&#x25B;ge&#x2D0;#x283;#x283;e&#x2D0;g"> egészség </phoneme> you can write <w syllables="e-gész-ség"> egészség </w> (Note: here you could also write <sub alias="e-gész-ség"> egészség </sub>)

  19. Overview Text structure Text-to-phoneme conversion Text normalization Prosody prediction Prosody prescription Summary Part-of-speech • Word forms may have several meanings/pronunciations • Specifying part-of-speech may help E.g. • I will <w POS=“verb” tense=“present”> read </w> the book • I have <w POS=“participle”> read </w> the book

  20. Overview Text structure Text-to-phoneme conversion Text normalization Prosody prediction Prosody prescription Summary Language • Foreign parts often occur in texts • Using same voice, currently you can • Do nothing • Specify using phoneme • Another desirable approach • Specify lexicon for language and specify language of text

  21. Overview Text structure Text-to-phoneme conversion Text normalization Prosody prediction Prosody prescription Summary Language Instead of …<speak … xml:lang="en-US">The title of the movie is:<phoneme alphabet="ipa"ph="&#x2C8;l&#x251; &#x2C8;vi&#x2D0;&#x27E;&#x259; &#x2C8;&#x294;e&#x26A; &#x2C8;b&#x25B;l&#x259;">La vita è bella </phoneme> (Life is beautiful).

  22. Overview Text structure Text-to-phoneme conversion Text normalization Prosody prediction Prosody prescription Summary Language you could write …<speak … xml:lang="en-US">The title of the movie is:<phoneme lang="it"> La vita è bella </phoneme> (Life is beautiful).

  23. Overview Text structure Text-to-phoneme conversion Text normalization Prosody prediction Prosody prescription Summary Language Suggested language attribute <phoneme [lang=“…” | “x-unknown”][ph=“…” [alphabet=“…”]]> …</phoneme> If both lang and ph is given, langhas priority If language is “x-unknown”, LID (language identification) is used. We suggest that “x-unknown” can be used with xml:lang also.

  24. Overview Text structure Text-to-phoneme conversion Text normalization Prosody prediction Prosody prescription Summary

  25. Overview Text structure Text-to-phoneme conversion Text normalization Prosody prediction Prosody prescription Summary • Text normalization effectively assisted by say-as element. • The constructs we found appropriate in our practice include:date, time(including time intervals like opening hours), number, currency, name, address. • Additionally suggest as standard values: acronym/abbreviation, web, e-mail, phone, program-code, table, equation.

  26. Overview Text structure Text-to-phoneme conversion Text normalization Prosody prediction Prosody prescription Summary

  27. Overview Text structure Text-to-phoneme conversion Text normalization Prosody prediction Prosody prescription Summary • We speak differently in different situations(e.g. speaking with friends, giving a talk at a conference, reading news, reading stories to children) – speaking style • Differences in prosody can be quantified • Emotional speech also in the focus of research • Modern TTS systems are likely to be able to imitate these to some extent

  28. Overview Text structure Text-to-phoneme conversion Text normalization Prosody prediction Prosody prescription Summary Speaking style Suggested speaking-style attribute • Can be used where the xml:lang element, i.e. voice, speak, p, s, w • Synthesis processors can define their own set of supported speaking-styles • They should support: "spelling"– can be viewed a special reading style • They may support e.g. "syllabification", "causal", "news reading", "story telling"

  29. Overview Text structure Text-to-phoneme conversion Text normalization Prosody prediction Prosody prescription Summary Emotion Suggested emotion attribute • Mentioned here, although prosody is only one of its aspects • Complementary to speaking-style, therefore separate attribute is suggested • Can be used where the xml:lang element, i.e. voice, speak, p, s, w • Possible values: "happiness", "sadness", "anger", "surprise", "disgust", "fear".

  30. Overview Text structure Text-to-phoneme conversion Text normalization Prosody prediction Prosody prescription Summary Part-of-speech • Part-of-speech (POS) of word may affect emphasis and other aspects of prosody • Not always possible to automatically determine • More desirable to specify POS than to prescribe prosody (higher level, speaking style can override it) Example in Hungarian: • “Mondd, hogy vagy?” (“Tell me, how are you?”)– interrogative adverb, strong (focus) emphasis • “Igaz, hogy jól vagy?” (“Is it true that you are alright?”)– conjunction, reduced emphasis

  31. Overview Text structure Text-to-phoneme conversion Text normalization Prosody prediction Prosody prescription Summary

  32. Overview Text structure Text-to-phoneme conversion Text normalization Prosody prediction Prosody prescription Summary • Analytic languages (e.g. English, Chinese) • Words are usually short • They convey only one portion of the meaning • Individual words can be stressed • Synthetic languages (e.g. Hungarian, Korean) • Words are often long • Made up of several morphemes and have very complex meanings • Stress, pitch changes, etc. may need to be realized on certain morphemes (~syllables)

  33. Overview Text structure Text-to-phoneme conversion Text normalization Prosody prediction Prosody prescription Summary Example 1: contrastive sentences • English:“The book is not in the box, but on the box.” • Speaker can emphasize one word. • Hungarian:“Nem a dobozon, hanem a dobozban van a könyv.” • Speaker sometimes has to emphasize one syllable. • Stress expressed mainly by pitch; may be aided by short pause, slower rate, higher volume.

  34. Overview Text structure Text-to-phoneme conversion Text normalization Prosody prediction Prosody prescription Summary Example 2: pitch change on syllable • “Elmentek.” – “They are gone.” Pitch is continuously falling • “Elmentek?” – “Are they gone?”Pitch rises at the beginning of the second syllable and falls down on the third syllable 1. 2.

  35. Overview Text structure Text-to-phoneme conversion Text normalization Prosody prediction Prosody prescription Summary Suggestion for extensions to prosody: • Stress and prosody can be described on a per-syllable basis • Extension to prosody: time can be syllable position • decimal fractions can also be used • negative values indicate nth position from end • special symbol syl_end indicates end of expression E.g.: <prosody contour=“(syl1,…) (syl1.5,…) (syl2,…) … (syl-1,…)(syl_end,…)”>

  36. Overview Text structure Text-to-phoneme conversion Text normalization Prosody prediction Prosody prescription Summary Suggestion for optional extensions: some synthesis processors may process • pitch-contour (=contour), rate-contour, volume-contourtime positions: the same as in contourrate / volume: described as in rate / volume • emphasis and break extended with a position attribute; value can be syllable position.In this case break will not be an empty element.

  37. Overview Text structure Text-to-phoneme conversion Text normalization Prosody prediction Prosody prescription Summary

  38. Overview Text structure Text-to-phoneme conversion Text normalization Prosody prediction Prosody prescription Summary Suggested extensions • <w [syllables=“…-…”] [POS=“…” [number=“…” …]]</w> • <phoneme lang=“…” | “x-unknown” [ph=“…” [alphabet=“…”]]> …</phoneme> • <voice | speak | p | s | w [speaking-style=“spelling” | “syllabification” | “causal” | “news reading” | “story telling” | …] [emotion=“happiness” | “sadness” | “anger” | “surprise” | “disgust” | “fear”] [<xml:lang=“…” | “xml-unknown”>]</voice> • <prosody contour=“(syl1,…) (syl2,…) (syl2.5,…) … (syl-2,…) (syl-1,…) (syl_end,…)”>optionally: pitch-contour (=contour), rate-contour, volume-contour; break, emphasis

  39. Overview Text structure Text-to-phoneme Text Prosody Prosody Summary conversion normalization prediction prescription

  40. Thank youfor your attention!

More Related