1 / 34

XISL language

XISL language. XISL= eXtensible Interaction Sheet Language or XISL=eXtensible Interaction Scenario Language. Characteristics. XML-based markup language for web-based MMI Grammar is defined by DTD Interaction between a XML/HTML element and a user User operation (click, speech input,…)

baker-diaz
Download Presentation

XISL language

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. XISL language XISL= eXtensible Interaction Sheet Language or XISL=eXtensible Interaction Scenario Language

  2. Characteristics • XML-based markup language for web-based MMI • Grammar is defined by DTD • Interaction between a XML/HTML element and a user • User operation (click, speech input,…) • Action (screen update, speech output,…) • Separates XML/HTML content from XML interaction • Allows user initiative interaction, system initiative interaction and mixed initiative interaction • Enables the use of various type of terminals: mobile phone, PC, digital TV, PDA,… • Uses I/O cooperativley: parallel I/O, sequential I/O, alternative I • Flexibility to expand I/O modalities

  3. Goals of XISL • Provide a common language for web-based multimodal interaction • Satisfy 3 conditions: • Control dialog flow/transition: employed from VoiceXML • Synchronize input/output modalities:employed from SMIL • Modality-extensibility:offered by XISL

  4. Exemple

  5. Dialog levels • Exchange • Dialog • Document • Application • Session

  6. Dialog levels • Exchange: primary unit of interaction between a user and a system • non-prompting description (user operation, system action) • prompting description (a prompt, user operation,system action) • Dialog: • composed of exchanges • user is always execution a dialog • Document: XISL file composed of some dialogs • Application: • set of XISL documents • contains 0..N leaf document, 1 application root document • Session • begins when user connects to the XISL execution system • ends when requested by the user or the XISL execution system

  7. Architecture of XISL execution system

  8. XISL execution system • Front-end module: • user interface terminal (microphone, speaker, screen,…) • Depends of application and terminal • Dialog manager module: • interprets XISL documents • manages dialog flow • controls I and O • Independent of application and terminal • Document server module (general web server): • holds XISL, XML/HTML documents • other documents (XISL stylesheet, speech grammar files,…)

  9. A subset of XISL elements

  10. Tree structure of XISL elements

  11. Exemple

  12. XISL interpreter Input integrator Action module Document manager Dialog manager sub-modules

  13. Outline of XISL execution system • XISL interpreter • recives the XISL document from Document Manager • devides it into • <operation>s : sends to Input Integrator • <prompt>s: sends to Action Module • <action>s: sends to Action Module • Input Integrator • extracts <input>s from <operation>s and sends them to front-end module • parses the <operation>s to prepare for input integration • matches the parsed <operation>s with the multimodal inputs sent from front-end module • informs the result to the action module • Action module • executes action corresponding to the <operation>s from Input Integrator • extracts <output>s from <prompt>s and <action>s • 2 types of actions: • terminal-dependent: outputs to a user are sent to the front-end • Terminal-independent: executed inside the action module (<switch>,<goto>)

  14. Separation of content from XISL document (1) • XISL document contains only interaction scenario • Content is held separately into XML/HTML files • Advantages • system developers can reuse XML/HTML documents and XISL documents • improves readabability

  15. Separation of content from XISL document (2) • <output> : output to the user Attributes: • type: output modality (browser, speech, audio,…) • event: output event (navigate, close, play,...) • <input> : accepts the events raised on XML/HTML documents Attributes: • type: input modality (speech, pointing, key,…) • event: input event (recognize, click, press,...) • match : link to XML/HTML element • return: return values • for flexibility strict attribute values and contents of above tags are not specified by XISL (it should be done by terminal developers)

  16. Modalities to control interactions in XISL documents: • comb attribute in <dialog> tag • Comb=par -> all exchange are executed in parallel • Comb=alt -> one of the exchange is executed alternatively • Comb=seq -> all exchange are executed in document order • <par_exchange> <seq_exchange> <alt_exchange> the exchange elements bound by these tags have the same semnification as above

  17. Exemple

  18. Types of front-end • PC terminal • On Line Shopping application • User input: touch-screen display, keyboard, microphone • Outputs: touch-screen(anthropomorphic agent), speeaker

  19. Input specification for PC terminal

  20. Output specification for PC terminal

  21. Exemple

  22. 2. Mobile phone terminal • On Line Shopping application • Sequential scenario due to terminal’s limits • Inputs: user’s speech, DTMF inputs • Output:speech

  23. Input specification for phone terminal

  24. Output specification for phone terminal

  25. Exemple

  26. 3. PDA terminal • On Line Shopping application • Inputs: cursor key, tap-pen (software keyboard), speech • Output: screen, sound device

  27. Input specification for PDA terminal

  28. Output specification for PDA terminal

  29. Exemple

  30. Comparation with other MMI approaches (1) • SALT: offers tags for describing speech recognition and TTS included in HTML documents Advantages of XISL: • better suited to describe complex MMI using sequential, parallel or alternative combination • interaction is explicitly separated from XML/HTML contents

  31. Comparation with other MMI approaches (2) 2. XHTML+Voice: • add VoiceXML to XHTML pages • provides voice interaction as additional modalities to web pages • VoiceXML comes with interaction flow: control more complex MMI then SALT Advantage of XISL: • more flexibility to add modalities

  32. Galatea Interaction Builder – prototyping tool • Runs on PC • Handels input modalities of speech, mouse, keyboard • Handels output modalities of speech(TTS), facial expresion, window display • Implementation of input and output can be made without the knowleadge of XISL • GUI design for domain-specific prototyping (airline ticket reservation, secretary service,…)

  33. Interaction Builder

  34. Conclusions • XML-based markup language for web-based MMI scenarios • XISL is separately described from XML/HTML contents (reusability) • enables the use of various type of terminals • flexibility to describe user’s input and system’s output

More Related