1 / 37

University of East Anglia, UK Dicta-Sign Partner 3

University of East Anglia, UK Dicta-Sign Partner 3. Prof John Glauert Virtual Humans Group School of Computing Sciences. Major Roles in Dicta-Sign. WP3 Synthesis and Animation Lead WP6 Sign Language Corpora BSL Lexicon and Corpus WP7 Bidirectional Integration Lead. Partnerships.

yeo-fernandez
Download Presentation

University of East Anglia, UK Dicta-Sign Partner 3

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. University of East Anglia, UKDicta-Sign Partner 3 Prof John Glauert Virtual Humans Group School of Computing Sciences

  2. Major Roles in Dicta-Sign • WP3 Synthesis and Animation • Lead • WP6 Sign Language Corpora • BSL Lexicon and Corpus • WP7 Bidirectional Integration • Lead

  3. Partnerships

  4. Virtual Signing at UEA • Dicta- Sign • John Glauert • Richard Kennaway, Post Doc • Vince Jennings, Engineer • Face Tracking, Audio-Visual Speech • Barry Theobald • Crowd Modelling, Urban Modelling • Prof Andy Day

  5. The ViSiCAST Consortium Instituut voor Doven Hamburg University University of East Anglia Independent Television Commission Televirtual The Post Office Royal Institute for Deaf People Institut National des Télécommunications Institut für Rundfunktechnik

  6. The eSIGN Consortium Viataal Hamburg University University of East Anglia Televirtual Norfolk County Council Royal National Institute for Deaf People

  7. Sign Language Animation from Sign Gesture Notation Prof John GlauertUniversity of East Angliawith: Norfolk County Council, RNID, Televirtual Ltd.

  8. Some Signing Avatars

  9. Virtual Characters

  10. Virtual Characters that can Sign

  11. Virtual Human Signing Approaches

  12. Motion Capture • Very lifelike animation • Time-consuming to set up • Blending of signs • Combining signs from different signers

  13. Hand-Crafted Animation • Define Key Frames • Interpolate between Key Frames • Can give very lifelike animation • Time consuming (maybe hours per sign) • Blending of signs still required

  14. Synthetic Signing • Synthesis from Abstract Representation • Quick to create lexicon • a few minutes to transcribe a sign • Instantly retargettable to any avatar with humanoid topology • Automatic Blending • Low Bandwidth

  15. Uses of Signing Avatars:Supporting Communication

  16. Simon the Signer, 1997-1999 • Simon-the-Signer [Broadcast TV] • Generate signed accompaniment to broadcast • Teletext stream as source • SSE – Sign Supported English • Funded by the ITC

  17. Simon the Signer, 1997-1999 • Developing transmission technology • virtual signer in set-top boxes • transmission of signing from text subtitles Audio/Video Stream TV Capture Card Avatar Software Mixer Teletext Stream Computer

  18. SiSi: Say it Sign it, 2007 • IBM with UEA and RNID • Uses Speech recogniser • Might help some meetings • Not pure BSL • www.youtube.com/watch?v=RarMKnjqzZU

  19. SiSi: Say it Sign it, 2007 • “I’ve seen an exciting movie”

  20. TESSA, 1998-2002 • TESSA [Retail, PO] • Using speech recognizer • Convert counter-clerk’s voice input to text • Generate sign stream from text • BSL – limited repertoire • Funded by the Post Office

  21. Post Office Counter Services • Post Offices transact business with almost all Deaf people • Counter Clerk asks questions using speech - No ‘back channel’ yet • Customer • Listens or • Reads or • Watches Virtual Human Signer • Real trials at 5 sites

  22. VANESSA, 2002 – 2005 • Council Information Centres (CIC’s) • Signing avatar conveys signed information • Text version also displayed • Signs created from special notation • Developed and enhancedversion of TESSA (Post Office)

  23. VANESSA – Customer’s View

  24. VANESSA – Clerk’s View

  25. Uses of Signing AvatarsInformation Services?

  26. Web ApplicationsWeather Forecast: NGT,BSL,DGS

  27. Weather Forecasts Sunday Tonight Today Mostly cloudy and dry in the morning. Rain breaking out during the afternoon. Maximum temperature 13 °C. Rain becoming heavy and persistent in the evening and continuing through the night. Met Office for London and South East England

  28. Web ApplicationsWeather Forecasts • Grammar for normal Weather Phrases • Sign Language version for each Phrase • Forecast is sequence of Phrases • Generate Common XML Weather Model • XSLT processing for each Sign Language • XSLT processing for Spoken Languages • Content creation by non- signers

  29. Uses of Signing AvatarsEducation?

  30. Signing Avatars in Education • PC Passport • SQA computer skills qualification • Need to be fair to all deaf candidates • Not all interpreters have IT knowledge • Different signs used by different schools • Students will have a dictionary of IT signs

  31. PC Passport Assessment

  32. PC Passport Dictionary

  33. Performing Hands, 2006-2009

  34. Performing Hands, 2006-2009 • GameLab London, RNID, UEA • Teach literacy for Deaf Children • BSL and English literacy • Mixes Video, 2D Animation, Avatars • Games and Stories • Shockwave Application • BAFTA Nominated • www.youtube.com/user/blahdeblahdeblahdebl • www.performinghands.com

  35. LinguaSign 2006-2009

  36. LinguaSign 2006-2009 • Primary Modern Foreign Language Learning • EU Socrates Lingua 2 • Vocabulary supported by iconic gestures (“signs”) • Speech in French, Dutch, English • Mix of conventional and synthetic animation

  37. Thanks for your attention

More Related