1 / 23

Chinese-Speaking 3D Talking Head

Chinese-Speaking 3D Talking Head. Project No: H08040 Sang Siew Hoon Supervisor: Dr Ng Teck Khim. System Objectives. create a realistic Chinese-speaking 3D talking model - automatic - accurate and realistic - portable. Problems. Lack of references on Chinese Involves various fields

lynde
Download Presentation

Chinese-Speaking 3D Talking Head

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Chinese-Speaking3D Talking Head Project No: H08040 Sang Siew Hoon Supervisor: Dr Ng Teck Khim

  2. System Objectives • create a realistic Chinese-speaking 3D talking model - automatic - accurate and realistic - portable

  3. Problems • Lack of references on Chinese • Involves various fields - Face modeling - Text-to-speech - Face Animation (Focus)

  4. System Overview 5 modules • Face Model Preparation • Chinese Visemes • Text-to-Speech (TTS) • Animation • Rendering

  5. Face Model Preparation

  6. Face Model Preparation • Face Modeling 3DS Max

  7. Face Model Preparation • Defining MPEG-4 feature points

  8. Face Model Preparation • Assigning lip vertices

  9. Face Model Preparation • Cloning morph targets FAP#3 open_jaw FAP#4 lower_t_midlip

  10. Chinese Visemes Visemes • the visual equivalent of a phoneme • Classification of Chinese Syllables • Definition of Chinese Visemes • Refining Dynamic Visemes

  11. Chinese Visemes • Definition of Chinese Visemes

  12. Chinese Visemes • Definition of Chinese Visemes

  13. Chinese Visemes • Definition of Chinese Visemes

  14. Chinese Visemes • Refining Dynamic Visemes • <i> +e only replace <f> er • zh, z, y + I  drop i • d, l, g, j, zh, z, y, w + u | ü  drop <i> • j, y + <f> ending with an  replace the an by en • j + complex finals headed by i and followed by more than one <f>  drop i in complex finals

  15. Text-to-Speech

  16. Text-to-Speech

  17. Animation • Coarticulation • Automatic Generation of Face Animation • Enhancing Realism

  18. Animation • Enhancing realism - Eye blinks - Eyebrow raising - Gaze - Head Rotation

  19. Results • Accurate and realistic • No discrepancies for new face models • Automated results need slight manual intervention http://www.slamstudio.com/HYP/videos

  20. Conclusion • Contributions • own system to define Chinese visemes - adds to the current research works of Chinese talking heads (currently limited). • an automatic Chinese lip-synchronization system - saves an animator much time and effort

  21. Conclusion • System Limitations • TTS limitation • FAPs conflicts • Not integrated – inconvenient for untrained users

  22. Conclusion • Future Works • Automatic audio signal processing • Resolve FAPs values conflicts • Automatic feature points assignment system • Integration of system

More Related