1 / 29

Representational Lego for ECAs

Representational Lego for ECAs. Brigitte Krenn. Motivation. Background representations for multimodal behaviour generation use representations at the interfaces of system components. Motivation. Wish reusable, flexible representational “standards”

clare-davis
Download Presentation

Representational Lego for ECAs

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Representational Lego for ECAs Brigitte Krenn

  2. Motivation • Background • representations for multimodal behaviour generation • use representations at the interfaces of system components HUMAINE Workshop Paris 10./11. March 2005 brigitte@oefai.at

  3. Motivation • Wish • reusable, flexible representational “standards” • to devise interface representations that ease • exchange of system components • integration of new modules HUMAINE Workshop Paris 10./11. March 2005 brigitte@oefai.at

  4. Motivation • Current • everybody does their own language • there is a wealth of different representations • partially overlapping • partially differing HUMAINE Workshop Paris 10./11. March 2005 brigitte@oefai.at

  5. up to date attempts to design a standard representation language for ECAs have failed HUMAINE Workshop Paris 10./11. March 2005 brigitte@oefai.at

  6. Goal to have • reusable • extendable • mappable bits and pieces of representations of ECA relevant information HUMAINE Workshop Paris 10./11. March 2005 brigitte@oefai.at

  7. Formal Requirements • separation of declarative and procedural information • mapping between high-level concepts and their low-level representations • mapping across concepts • extendibility • granularity of descriptions • incorporation of new concepts • ability to embed existing XML representations HUMAINE Workshop Paris 10./11. March 2005 brigitte@oefai.at

  8. The Content Side ? • units of information common to existing ECA systems • information ECA systems ideally should have • allow for optionality HUMAINE Workshop Paris 10./11. March 2005 brigitte@oefai.at

  9. The Current State HUMAINE Workshop Paris 10./11. March 2005 brigitte@oefai.at

  10. The Current StateTerminology • Markup Languages • Representation Languages • Scripting Languages HUMAINE Workshop Paris 10./11. March 2005 brigitte@oefai.at

  11. The Current StateMarkup Languages • for non-expert users • to annotate text • with high-level expert information • e.g. • VoiceXML for creating voice enabled applications • VHML for creating interactive applications with ECAs • APML for annotating text with high-level ECA controls HUMAINE Workshop Paris 10./11. March 2005 brigitte@oefai.at

  12. The Current StateRepresentation Languages • technically detailed annotations of theory-specific information • high- and low-level concepts • for expert use • function as data representation formats inside a system • e.g. RRL HUMAINE Workshop Paris 10./11. March 2005 brigitte@oefai.at

  13. The Current StateScripting Languages • combine declarative and procedural knowledge • comparable to high-level programming languages • e.g. STEP/XSTEP, ABL, PAR/EMOTE HUMAINE Workshop Paris 10./11. March 2005 brigitte@oefai.at

  14. The Current StateSummary • Markup languages • high-level concepts • are indispensable for application development • Representation languages • mix high- and low-level concepts • are crucial in research contexts • Scripting languages • add procedural information HUMAINE Workshop Paris 10./11. March 2005 brigitte@oefai.at

  15. Markup Language The Current StateSummary Representation Language Scripting Language HUMAINE Workshop Paris 10./11. March 2005 brigitte@oefai.at

  16. The Current StateSummary • standardisation efforts up to now concentrate on markup languages • they are • application oriented • to design representations for ECAs in the spirit of VoiceXML • text/utterance oriented • to design multimodal behaviour control as markup for text HUMAINE Workshop Paris 10./11. March 2005 brigitte@oefai.at

  17. Pros for XML-based Representation Languages as Interfaces in ECA systems HUMAINE Workshop Paris 10./11. March 2005 brigitte@oefai.at

  18. Advantages of XML Encoding • XML is • flexible • easy to share • tools for XML processing • standardization efforts (w3c) HUMAINE Workshop Paris 10./11. March 2005 brigitte@oefai.at

  19. Advantages of the Use of Representation Languages • to encode information flow between system components • to map between high-level concepts and low-level realizations • to ease integration/replacement of system components • to support a plug-and-play approach • to support the development of mockup systems HUMAINE Workshop Paris 10./11. March 2005 brigitte@oefai.at

  20. Affective, Interactive ECARelevant Components and Concepts HUMAINE Workshop Paris 10./11. March 2005 brigitte@oefai.at

  21. The Content SideCurrent Foci • speaking ECA • simulation of mm-dialogues • no/little “true” interactivity • APML, RRL • moving ECA • XSTEP, MURML, (?MiraLab) • speaking and moving ECA • there are some gaps to bridge • interacting ECA • approaching • PML, ABL HUMAINE Workshop Paris 10./11. March 2005 brigitte@oefai.at

  22. The Content SideRepresentations & Architectures • information relevant for a certain ECA system depends on the architecture and system components used • Is it possible to identify a common core of relevant components and concepts • Is it possible to provide reusable representations for these concepts • Allow for flexibility of the representations HUMAINE Workshop Paris 10./11. March 2005 brigitte@oefai.at

  23. The Content Side Relevant Parts • World parameters • Scenes and story lines • MM-dialogue generation • Speech • Animation (body, face) • Affect (emotion, personality traits) • Temporal control and synchronization • Interactivity • MM-comprehension HUMAINE Workshop Paris 10./11. March 2005 brigitte@oefai.at

  24. The Content SideTemporal Control and Synchronization • time-alignment of mm-behaviour of an agent • temporal ordering of the actions/behaviours of agents interacting with the outside world • agent-object • agent-agent • agent-user HUMAINE Workshop Paris 10./11. March 2005 brigitte@oefai.at

  25. The Content SideTime-Alignment of MM-Behaviour • speech as guiding medium(phoneme durations) • motion: beats as smallest units(e.g. XSTEP) • synchronization of speech rate and motor activity • motor activity can also constrain voice quality HUMAINE Workshop Paris 10./11. March 2005 brigitte@oefai.at

  26. The Content SideInteractivity • What are the desired smallest communicative units? • speech • dialogue • interactive drama • What are the technically manageable smallest units? • What are the technological challenges? HUMAINE Workshop Paris 10./11. March 2005 brigitte@oefai.at

  27. The Content SideInteractivity • Multimodal understanding • What is relevant information • How do we manage the information flow • agent technology (Lola et al.) • Models of the listening ECA HUMAINE Workshop Paris 10./11. March 2005 brigitte@oefai.at

  28. Next Steps • examine existing languages for a common core • compare their representations • consider architectural aspects • define XML representations for bits and pieces • make them publicly available HUMAINE Workshop Paris 10./11. March 2005 brigitte@oefai.at

  29. talk me off join in ! HUMAINE Workshop Paris 10./11. March 2005 brigitte@oefai.at

More Related