1 / 22

S.D.M.A.P.A.

S.D.M.A.P.A. Spoken Dialog Management for an Astronaut’s Procedure Assistant Presented by: Dan Bohus (dbohus@cs.cmu.edu) Collaborators: Gregory Aist, RIALIST Group NASA Ames/RIACS Summer Student Research Program August 2002. RIALIST Group @ NASA Ames.

ron
Download Presentation

S.D.M.A.P.A.

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. S.D.M.A.P.A. Spoken Dialog Management for an Astronaut’s Procedure Assistant Presented by: Dan Bohus (dbohus@cs.cmu.edu) Collaborators: Gregory Aist, RIALIST Group NASA Ames/RIACS Summer Student Research Program August 2002

  2. RIALIST Group @ NASA Ames • Location: NASA Ames / Moffett Field / Mountain View, CA (CMU West) • People • Jim Hieronymous • John Dowding • Beth-Ann Hockey • Greg Aist • + 7 summer students • Projects • Personal Satellite Assistant • Mobile Robots • Intelligent Procedure Assistant Dialog Management for an Intelligent Procedure Assistant

  3. Motivation • Crew members on International Space Station (ISS) have to perform a wide variety of procedural tasks: • ISS maintenance and repair • Operations (i.e. EVA Suit Checkout) • Setup and running scientific experiments • Medical Exams Dialog Management for an Intelligent Procedure Assistant

  4. Typical setup • Paper documentation • 2 or more astronauts involved: one reads, one executes the procedure Dialog Management for an Intelligent Procedure Assistant

  5. Proposed alternative • Build a spoken dialog system for procedural task guidance and support • Create more crew-time! • Used also for ground training • Potential for increased safety Dialog Management for an Intelligent Procedure Assistant

  6. Outline • Motivation • The Intelligent Procedure Assistant • A Dialog Management Component for IPA • Demo • Future work Dialog Management for an Intelligent Procedure Assistant

  7. The Intelligent Procedure Assistant Multi-modal spoken dialog system providing crew support and training for the execution of procedural tasks • Guide astronauts through procedural tasks • Provide information access (diagrams, docs) • Act as a training tool • Facilitate task delegation • Provide support for voice annotations, intercom Dialog Management for an Intelligent Procedure Assistant

  8. IPA Version 0: June 2002 • RIALIST IPA Version 0 • Load procedure • Next • Previous • Drill-down • Describe remaining steps • Increase volume • Decrease volume • Say again Dialog Management for an Intelligent Procedure Assistant

  9. Outline • Motivation • The Intelligent Procedure Assistant • A Dialog Management Component for the IPA • Demo • Future work Dialog Management for an Intelligent Procedure Assistant

  10. Contributions: Dialog Management & Resources U D SI SpeechRecognition Semanticizer(parsing) Dialog Management Resources/Agents LanguageGeneration Synthesis S NL SO Dialog Management for an Intelligent Procedure Assistant

  11. Audio Dialog Agent U D SI Nuance [recognition] Gemini[parsing] U D SI Nuance[recognition] Gemini [parsing] Dialog Task Specification RavenClaw DM framework (CMU) Dialog Management Resources/Agents The actual domain task (IPA domain) Dialog Management Resources/Agents Gemini [generation] CORE/ Conversational Skills Festival Gemini [generation] Handles dialog flow, initiative, focusshifts, universal dialog mechanisms S NL SO Festival S NL SO Contributions: Dialog Management & Resources 1 2 3 Dialog Management for an Intelligent Procedure Assistant

  12. 1. Resources • Procedures represented in XML (DTD, XSLT) • Title, Description. • Steps: Basic Steps, Conditional Steps, Messages, Images • Messages: Notes, Cautions • Images: Sub-images • Water Sampling Procedure • 28 steps, 150 sub-steps, 18 conditional steps, 1 image with 5 sub-images • HTML version (using XSLT) Dialog Management for an Intelligent Procedure Assistant

  13. Help Suspend/Resume Step1 Step28 Image1 SubStep1 SubStep4 2. Dialog Task Development • In RavenClaw, a dialog task is implemented as a hierarchy of dialog agents (C++ classes) IPA Welcome LoadProcedure Do Procedures DoProcedure1 DoProcedure2 Intro Closing Dialog Management for an Intelligent Procedure Assistant

  14. 2. Dialog Task Development (cont) • 52 dialog task agents (types) were implemented: • Loading procedures by name • Guidance • Step-by-step execution • Drill-down • Branching • Presenting notes / cautions / images • Information access • Talking on/about images, objects • Navigational behavior • History / Trace through procedure • Next, Previous step/substep • Start over Dialog Management for an Intelligent Procedure Assistant

  15. 3. Conversational Skills Enhancements • Timeouts • System reacts after a timeout period • User Barge-in • Requests for repeat • Requests for help • generic “help!” • “what can I say ?” • Context establishment • “where are we ?” Dialog Management for an Intelligent Procedure Assistant

  16. Outline • Motivation • The Intelligent Procedure Assistant • A Dialog Management Component for IPA • Demo • Future work Dialog Management for an Intelligent Procedure Assistant

  17. Demo Guidance Information Access Volume control Dialog Management for an Intelligent Procedure Assistant

  18. Outline • Motivation • The Intelligent Procedure Assistant • A Dialog Management Component for IPA • Demo • Future work • IPA System (RIALIST Group) • Advances in dialog management Dialog Management for an Intelligent Procedure Assistant

  19. Future work for IPA system (RIALIST Group) • IPA system functionality goals • Guide astronauts through procedural tasks • Provide information access • Act as a training tool • Facilitate task delegation • Provide support for voice annotations, intercom • Iterative improvement process • Other features, including those suggested by JSC training personnel and astronauts Dialog Management for an Intelligent Procedure Assistant

  20. Timeline … • Reliable prototype by the end of September • Guidance/Information Access • First fieldable system by March 2003 Dialog Management for an Intelligent Procedure Assistant

  21. Summer @ NASA is cool ! Dialog Management for an Intelligent Procedure Assistant

  22. Being back is also cool ! • 2 new systems to be developed based on the Communicator SDS setup / RavenClaw DM • RoomLine system • Provide conference room reservation services within SCS • Ridy Lie • BusLine system • Provide access to bus line information • Project “Let’s go!” (Maxine/Alan)? • Brian Langner, Antoine Raux • Hope you can call them sometime before Christmas !!! Dialog Management for an Intelligent Procedure Assistant

More Related