Download
1 / 19
190 likes | 359 Views
MAJORDOME. Gérard CHOLLET, Richard CROCE, Laurence LIKFORMAN, Dijana PETROVSKA-DELACRETAZ, Pascal VAILLANT ( chollet,croce,lauli,petrovsk,vaillant ) @ tsi.enst.fr ENST/CNRS-LTCI 46 rue Barrault 75634 PARIS cedex 13 http://www.tsi.enst.fr/~chollet/. Majordome Outline. What is it ?
E N D
MAJORDOME Gérard CHOLLET, Richard CROCE, Laurence LIKFORMAN, Dijana PETROVSKA-DELACRETAZ, Pascal VAILLANT (chollet,croce,lauli,petrovsk,vaillant)@tsi.enst.fr ENST/CNRS-LTCI46 rue Barrault75634 PARIS cedex 13http://www.tsi.enst.fr/~chollet/
Majordome Outline • What is it ? • What it does for you ? • Research and application topics: • The SIROCCO project • The EUREKA !2340 MAJORDOME project • VoIP, VoiceXML, Human-Computer Interaction • Perspectives
Majordome is a distributed Personal Digital Assistant • It is your digital slave. It is personal. It remembers everything that you told him. • It uses resources from you mobile (wireless) device, from your home, from your office, from the Internet, from the environment, … • You interact with him using voice, pen, graphics, …
Interactions with your Majordome • Majordome recognizes your identity, your voice, your handwriting, ... • His speech recognizer is adapted to your voice, • His handwriting recognizer is adapted to your writing style, • He can speak to you, • He can display information for you, • He can talk with other persons either locally or over the phone.
What Majordome does for you ? • Answers your phone, • Receives and interpret your faxes, your emails, … • Supplements your memory (address book, agenda, bookmarks, alarm clock, health record, bank account, documentation, …) • Serves as an interface between you and the (digital) world, • Searches the web, internet forums, … • Controls your home, your car, your children, your parents, …
A framework: A L I S P Automatic Language Independent Speech Processing with applications in Speech Coding, Synthesis, Recognition, Speaker Verification and Language Identification
SIROCCO project Unlimited Vocabulary Speech Recognition INRIA (IRISA et LORIA), LIA, IRIT, ENST-LTCI http://www.irisa.fr/sirocco/
SIROCCO • Unlimited vocabulary speech recognition system • French lexicon (MathLex) with 64kwords (AUF task) • Feature extraction with Spro (G. Gravier) • Context-dependent HMM phone models • Word pronunciation graph • Uses CMU-Toolkit for Language modeling • Beam search for word hypothesis • Rescoring of word hypothesis by A*
Holistique EDF «MAJORDOME» Unified Messaging System Eureka Projet no 2340 D. Bahu-Leyser, G. Chollet, R. Croce, K. Hallouli , J. Kharroubi, D. Kofman, L. Likforman, E. Matta-Sanchez, D. Petrovska, M. Sigelle, P. Vaillant, F. Yvon
Participants • speech : G. Chollet, R. Croce, J. Kharroubi, D. Petrovska • fax : K. Hallouli, L. Likforman, Marc Sigelle • language : P. Vaillant, F. Yvon • platform : D. Kofman, E. Matta-Sanchez, R. Croce • ergonomy : D. Bahu-Leyser
MAJORDOME ( E-mail • Speaker verification • Dialogue • Routing • Updating the agenda • Automatic summary Voice Fax Majordome’s Functionalities
Overview of Majordome • Background tasks (server-side only): • sorting and filtering messages from different sources (E-mail, voice, fax, SMS,…); • extracting relevant information for reporting to user (names of senders, subject,…). • Dialogue with the user: over phone or Web. • The system presents the state of the mailbox, the type of messages, their sender, subject, and may sum them up or read them on request; • The users access their mailbox, addressbook, time schedule, or URIs (Web addresses).
Voice technology in Majordome • Server side background tasks: continuous speech recognition applied to voice messages upon reception • Detection of sender’s name and subject • User interaction: • Identification of the speaker (and Verification if necessary) • Speech recognition (receiving users’ commands through voice interaction) • Text-to-speech synthesis (reading text summaries, E-mails or faxes)
Network192.168.111.0/11 Cisco Catalyst 6507 Unisphere ERX-700 1Gbps (FO Interne) 1Gbps Salle C-234 VTHD Intranet DistanceLearningService Renater GK GW IPVR Salle C-234 Salle C-234 Video Server Salle PBX ( RTC/RNIS PBX ENST-Paris Voice Over IP Platform Network 192.168.222.0/11 Network192.168.223.0/11 Visioconference
Is the called person here ? NetCentrex user called Majordome / NetCentrex project PABX /Gateway ENST -Call Control Server -Application Server Calling person NetCentrex # Usual # IP-VR NetCentrex Recorder Machine No response Usual user called Vocal E-mail
NetCentrex user called Majordome / NetCentrex project PABX /Gateway ENST -Call Control Server -Application Server Calling person NetCentrex # Usual # Voice Interactive call No response MAJORDOME IP-VR NetCentrex • Speaker verification • Dialogue • Vocal e-mail • Routing • Updating the agenda • Automatic summary Usual user called
Perspectives • Add Vision, Hearing and Understanding to Mobile Terminals (UMTS) • Multimedia for Distance Education and Conference Indexing • Semantic Web, • ‘Universal Networking Language’ • ‘Smart Home’, ‘Smart Car’, ‘Smart Office’
Perspectives • The application context of the Majordome project could be of interest to COST-278. • The Majordome/NetCentrex platform could be made available to interested partners. • HTK, ISIP and SIROCCO softwares are available as freeware. One of them will be used on the NetCentrex platform.
