1 / 43

ATLAS : Automatic Translation into Sign Languages

ATLAS : Automatic Translation into Sign Languages. Paolo PRINETTO Politecnico di Torino (Italy) University of Illinois at Chicago, IL (USA) Paolo.Prinetto@polito.it prinetto@uic.edu www.testgroup.polito.it www.comitato-girotondo.org. Expected key questions …. What are Sign Languages?

jeslyn
Download Presentation

ATLAS : Automatic Translation into Sign Languages

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. ATLAS : Automatic Translation into Sign Languages Paolo PRINETTO Politecnico di Torino (Italy)University of Illinois at Chicago, IL (USA) Paolo.Prinetto@polito.itprinetto@uic.edu www.testgroup.polito.itwww.comitato-girotondo.org

  2. Expected key questions … • What are Sign Languages? • Why are you going to talk about them? • Why are you going to talk about them here at EWD&T 2009? • …

  3. What are Sign Languages?

  4. Sign Languages • Sign Languages are real languages like any other human languages and not just “pictures in the air”. • The European Parliament, on June 17th, 1988 and November 18th, 1998, and the UNESCO in Salamanca Resolution, art. 21, on May 15th, 2001, stated the official status of Sign Languages of Deaf people as official languages.

  5. Sign Language is not Universal • Lingua dei segni italiana (LIS) • American Sign Language (ASL) • British Sign Language (BSL) • Langue des Signes Française (LSF) • Lingua dei segni nicaraguense (ISN) • New Zealand Sign Language (NZSL) • …

  6. Deaf people in Italy • In Italy 1 person out of 1,000 is affected by serious deafness • In Italy there are today about 50,000 persons who were born deaf or got deafness in the first years of life

  7. Impacts • The integration difficulties of persons who were born deaf or got deafness in the first years of life are higher because they could not acquire the knowledge of the spoken language. • Sign Languages allow deaf children acquiring a full cognitive development within their community composed by hearing and deaf persons. • Such cognitive development represents the prerequisite to a full access to the education, the culture and the inclusion in working and social environment.

  8. Why am I going to talk about them? • I’m chairing and managing the ATLAS Project: • a 3-year project, started January 2009 • co-funded by Regione Piemonte, Piedmont Region, Turin, Italy • www.atlas.polito.it

  9. The ATLAS Project • The ATLAS project exploits the convergence between cognitive sciences and ICT to build innovative services to provide deaf people with the access to broadcast communi-cations, through the automatic translation of the written Italian natural language into the animated Italian Sign Language (LIS)

  10. The ATLAS Project (3) • Such a challenging goal will be achieved by pursuing the following objectives: • Develop an automatic translation system from Italian language written text (subtitles or general texts) to LIS. The translation will be accomplished resorting to the most advanced statistic and rule-based techniques • Develop appropriate “corpora” of information to properly train the automatic translation system.

  11. The ATLAS Project (4) • Develop a visual assisted editor to support interpreters in the manual translation from Italian texts to ISL. The editor will serve as the reference tool for building the corpora • Develop an engine for rendering and visualization of a virtual interpreter

  12. The ATLAS Project (5) • Define avatar commands for different delivering platforms and client devices • Design and deploy of client server platforms to demonstrate the service on different media, including • several types of Digital Video Broadcasting (DVB) • mobile phones • web applications • additional physical media such as DVD.

  13. ATLAS Architecture Partial Overview

  14. ATLAS Architecture Partial Overview

  15. ATLAS Architecture Partial Overview

  16. The challenge • Representing sentences in Sign Languages resorting to the minimum set of information needed to drive the virtual character.

  17. Open issues in ATLAS_Written_ISL • Sign spatial positions (i.e., 3-dimensional starting location & ending locations) are context sensitive • Sign classifications: • Fixed position • “Movable” • Placeholders are extensively used

  18. “The car is parked between the cat and the house” • Placeholder for car, house, cat

  19. An (incomplete) example • Input: • La mamma ha dato al bambino un grosso gelato alla pesca. (The mother gave the kid a big peach ice cream). • Output: • MAMMA(1) • BAMBINO(2) • GELATO(3; scaled_of=big) • PESCA$1(4; Attr_of=3) • DARE(src=1; dst=2; obj=3; temp_attr=done)

  20. Sign Representation At least 5 parameters: Movement Hand Shape Location Orientation Non-Manual Gesture Signs (NMGSs) or facial expressions are required to describe signs properly.

  21. ATLAS Architecture Partial Overview

  22. 3D (Character) Animation • Keyframe interpolation • Manually edited static poses (keyframes) • Algorithmic interpolation • Motion Capture • Animation data curves recorded from the real world • Physical simulation • Weight and forces to body parts • Purely algorithmic

  23. Keyframe animation

  24. Geometries

  25. Motion Capture Recording real human or (cooperative) animal motion

  26. Motion capture

  27. Screen Translation Chain Simplified Italian text NL → EWISL Translator EWISL text EWISL → AL Translator Animation Language text “Actionary” [Badler00] Action/Animation Dictionary. A map between words and actor movement. Character 3D Model AL text structure: AL Player Renderer <time> <action> <params ...> <time> <action> <params ...> <time> <action> <params ...> <time> <action> <params ...> <time> <action> <params ...> ... Sequencer (Parse the timeline to issue commands at the right moment) Ogre & C. Broadcast point

  28. Delivery Architecture Delivering center archivio Automatic Translation Assisted Translation pc hh stb ip ip dvb [ip] User terminals

  29. A very first result

  30. Cпаcибo!!! Малые Автюхи, Калинковичский район, Республики Беларусь

More Related