1 / 19

Interactive multimedia news presentation on very low bitrates

Interactive multimedia news presentation on very low bitrates. Igor S. Pand žić Department of telecommunications Faculty of electrical engineering and computing University of Zagreb, Croatia www.tel.fer.hr. Objectives. Rich multimedia news delivery Animation, voice, graphics Interactive

jaclyn
Download Presentation

Interactive multimedia news presentation on very low bitrates

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Interactive multimedia news presentation on very low bitrates Igor S. Pandžić Department of telecommunications Faculty of electrical engineering and computing University of Zagreb, Croatia www.tel.fer.hr

  2. Objectives • Rich multimedia news delivery • Animation, voice, graphics • Interactive • News on demand • Easy access • Low bandwidth, low processing power • Automatic • Content generation from news data base TNC03 - Pandzic

  3. The system: user’s view MPEG-4 FA Player Applet Dynamic graphics area News selection area TNC03 - Pandzic

  4. The system: how it works Newscaster 3D face model Facial Motion Cloning Newscaster morph targets creation Newscaster now ready for animation Generic MPEG-4 morph targets Making the Newscaster – one-time process Structured news (XML) News processing Visual speech synthesis Dynamic web pages creation Images & graphics Making the News – repeated process MPEG-4 FA Player Applet Dynamic graphics area News selection area TNC03 - Pandzic

  5. Making the newscaster Newscaster 3D face model Facial Motion Cloning Newscaster morph targets creation Newscaster now ready for animation Newscaster 3D face model Facial Motion Cloning Newscaster morph targets creation Newscaster now ready for animation Generic MPEG-4 morph targets Generic MPEG-4 morph targets Making the Newscaster – one-time process Making the Newscaster Structured news (XML) News processing Visual speech synthesis Dynamic web pages creation Images & graphics Making the News • Create a 3D face model • Create morph targets for the model • Construction of models and animations using standard 3D tools • Problem: morph target creation tedious • Solution: Facial Motion Cloning TNC03 - Pandzic

  6. Facial Motion Cloning Newscaster 3D face model Facial Motion Cloning Newscaster morph targets creation Newscaster now ready for animation Generic MPEG-4 morph targets Making the Newscaster Structured news (XML) News processing Visual speech synthesis Dynamic web pages creation Images & graphics Making the News • Fast production of new face models for animation • Automating the morph target approach, the common way of animating faces TNC03 - Pandzic

  7. Facial Motion Cloning Newscaster 3D face model Facial Motion Cloning Newscaster morph targets creation Newscaster now ready for animation Generic MPEG-4 morph targets Making the Newscaster Structured news (XML) News processing Visual speech synthesis Dynamic web pages creation Images & graphics Making the News TNC03 - Pandzic

  8. Making the news Newscaster 3D face model Facial Motion Cloning Newscaster morph targets creation Newscaster now ready for animation Generic MPEG-4 morph targets Making the News – repeated process Making the Newscaster Structured news (XML) Structured news (XML) News processing Visual speech synthesis Dynamic web pages creation News processing Visual speech synthesis Dynamic web pages creation Images & graphics Images & graphics Making the News • News organised by topic in simple XML format • News items contain references to image files • Visual speech synthesis: create voice and animation • Create web site structure with appropriate links TNC03 - Pandzic

  9. Making the news: example Newscaster 3D face model Facial Motion Cloning Newscaster morph targets creation Newscaster now ready for animation Generic MPEG-4 morph targets Making the Newscaster Structured news (XML) News processing Visual speech synthesis Dynamic web pages creation Images & graphics Making the News <?xml version="1.0" encoding="ISO-8859-1"?> <news> <logo>newslogo.jpg</logo> <introduction>Welcome to the interactive news.</introduction> <voice>Mary</voice> <topic> <name>Headlines</name> <item> <text>U.S. airstrike hits Baghdad.</text> <image>baghdad.jpg</image> </item> <item> … </item> … </topic> … </news> TNC03 - Pandzic

  10. Making the news Newscaster 3D face model Facial Motion Cloning Newscaster morph targets creation Newscaster now ready for animation Generic MPEG-4 morph targets Making the Newscaster Structured news (XML) News processing Visual speech synthesis Dynamic web pages creation Images & graphics Making the News • XML format can be extracted from a content management system • Structure can easily be extended • News on demand can be supported with personalised news composition TNC03 - Pandzic

  11. Delivering the news Newscaster 3D face model Facial Motion Cloning Newscaster morph targets creation Newscaster now ready for animation Generic MPEG-4 morph targets Making the Newscaster Structured news (XML) News processing Visual speech synthesis Dynamic web pages creation Images & graphics Making the News • Requirements: • Virtual newscaster (streaming face animation and audio) • Dynamic graphics synchronised with presentation • Must work in standard browser TNC03 - Pandzic

  12. MPEG-4 Face Animation Newscaster 3D face model Facial Motion Cloning Newscaster morph targets creation Newscaster now ready for animation Generic MPEG-4 morph targets Making the Newscaster Structured news (XML) News processing Visual speech synthesis Dynamic web pages creation Images & graphics Making the News • ISO standard since 1999. • Work started in 1996. • Only standard for FA • Widely accepted in academia • Spreading into industry as new products emerge • Standard also includes Body Animation TNC03 - Pandzic

  13. MPEG-4 Face Animation: summary Newscaster 3D face model Facial Motion Cloning Newscaster morph targets creation Newscaster now ready for animation Generic MPEG-4 morph targets Making the Newscaster Structured news (XML) News processing Visual speech synthesis Dynamic web pages creation Images & graphics Making the News • Complete • Large range of expressions • Efficient • No redundancy • Compressed to low bitrates • Relatively simple • Face animation parameters displace feature points from neutral position • Face definition parameters allow personalisation in a scalable manner • Portable & reusable • Animation parameters are model-independent TNC03 - Pandzic

  14. Facial Animation Player Newscaster 3D face model Facial Motion Cloning Newscaster morph targets creation Newscaster now ready for animation Generic MPEG-4 morph targets Making the Newscaster Structured news (XML) News processing Visual speech synthesis Dynamic web pages creation Images & graphics Making the News • MPEG-4 FBA decoder • Simple FA: weighted morph targets • Lightweight implementation • Easily portable to any platform + + = TNC03 - Pandzic

  15. Current implementations Newscaster 3D face model Facial Motion Cloning Newscaster morph targets creation Newscaster now ready for animation Generic MPEG-4 morph targets Making the Newscaster Structured news (XML) News processing Visual speech synthesis Dynamic web pages creation Images & graphics Making the News • Java applet, Shout3D rendering engine • Symbian, iPAQ early prototypes • PC standalone • 3ds max plugin • Softimage dotXSI export TNC03 - Pandzic

  16. Applet player bandwidth requirements Newscaster 3D face model Facial Motion Cloning Newscaster morph targets creation Newscaster now ready for animation Generic MPEG-4 morph targets Making the Newscaster Structured news (XML) News processing Visual speech synthesis Dynamic web pages creation Images & graphics Making the News • Applet size: ~150K • FBA bitstreams • viseme-encoded: ~0.3 kbit/s • low-level FAPS: 2-6 kbit/s • Audio (GSM 6.10): 13 kbit/s • Face models: ~50K reasonable Acceptable for modem connections TNC03 - Pandzic

  17. Applet player graphics performance Model Polygons Size (KB) Frames/second C1 C2 C3 Demy 2800 32 11,5 16 21 Newscaster 3D face model Facial Motion Cloning Newscaster morph targets creation Newscaster now ready for animation Dummy 1362 50 16,5 21 32 Jörgen 168 40 22 30 60 Generic MPEG-4 morph targets Candide 168 4 35 60 60 MIRAface 3692 67 10,5 16 32 Making the Newscaster Lake 16917 284 0,97 1 1,5 Structured news (XML) News processing Visual speech synthesis Dynamic web pages creation Images & graphics Making the News Acceptable for AVERAGE PC C1= P3/600 C2 = P3/1000 P3/1000, OGL acceleration TNC03 - Pandzic

  18. Comparison with other news media TNC03 - Pandzic

  19. Demonstration • An automatically created web site with some (old) news TNC03 - Pandzic

More Related