10 likes | 153 Views
<catch event="Insert+F1 help"> <prompt> You are at Circuit City check out </prompt> </catch> <catch event=“Ctrl+S submit"> <submit namelist=“fname lname …”> </catch> …. Cached Files main.vxml allLinks.vxml history.vxml favorites.vxml keyList.vxml commands.vxml ….
E N D
<catch event="Insert+F1 help"> <prompt> You are at Circuit City check out </prompt> </catch> <catch event=“Ctrl+S submit"> <submit namelist=“fname lname …”> </catch> … Cached Files main.vxmlallLinks.vxml history.vxmlfavorites.vxmlkeyList.vxml commands.vxml… …<rule id="phone_number"> <item repeat="0-1">d</item> <tag><![CDATA[$.phone="ddd-ddd-dddd";]]></tag> </item></rule> … fname=“Yevgen”lname=“Borodin”zip=“11790” … INS + F1 A Flexible VXML (Voice XML) Interpreter Yevgen BorodinComputer Science Department, Stony Brook University Back-End Processing Motivation VXMLSurfer in Action • VoiceXML dialog files are sent to the interpreter for processing • Variables are returned to the calling application (HearSay) • HearSay invokes form-filling module to fill and submit the form • Inadequacy of screen-readers in Web browsing • Development of interface Manager for HearSay voice-browser • Absence of fully-implemented open-source VXML interpreters • Need for specialization in Web browsing applications • Need for a multiplatform, extensible, modular, flexible system <?xml version='1.0'?><vxml> … <form id=‘shipping'> <field name="fname"> <prompt>Enter first name</prompt> <grammar src="customer.xml#fname“ type="application/grammar+xml"/> </field> <field name="lname"> <prompt>Enter last name</prompt> <grammar src="customer.xml#lname" type="application/grammar+xml"/> </field> … </form></vxml> Variables | Files | Return | Events user=“Yevgen”session=“browse”… Uses of VoiceXML title=“circuit city”mode=“formFill”… • VXML is typically used in telephony applications • Computer games use VXML to program interactive dialogs • VXML dialogs disseminate information through public terminals • VoiceXML can be used in voice browsing! fname=“Yevgen”lname=“Borodin”zip=“11790”… SGRS Grammar Features of VXMLSurfer • Compliant with VoiceXML 2.0 specifications • Geared to Web-browsing as opposed to telephony • Modular, Extensible, Multi-Platform (Java) • Extends VoiceXML 2.0 to give more control over dialog flow • Loaded with add-ons: Spell Check, TTS, SR, etc. Application of VXMLSurfer Big Picture: HearSay Browser • VXMLSurfer is the interface of the HearSay voice-browser • Users interact with VXMLSurfer through keyboard and mic • HTTP request is forwarded to the Mozilla browser engine • The loaded Web page is converted into a frame tree • The frame tree is processed and converted to VXML dialogs VXML Interpreter Mozilla Engine Future Work • Complete VoiceXML 2.0 Specifications • Messaging between VXMLSurfer and calling application • Speech Recognition (CMU Sphinx) • Java Script Interpreter and Grammar Interpreter Modules • Multilingual TTS, etc. HELP This material is based upon work supported by the National Science Foundation - Awards: IIS-0534419, CNS-0751083, IIS-0808678 and National Institute on Disability and Rehabilitation Research (NIDRR) - Award: H133S090065.