1 / 33

DAI-Labor Fakultät IV – Elektrotechnik und Informatik Technische Universität Berlin

Towards a Model-Based Framework for the Development of Adaptive Multimodal User Interfaces for Ambient Assisted Living Environments. Marco Blumendorf and Sahin Albayrak. DAI-Labor Fakultät IV – Elektrotechnik und Informatik Technische Universität Berlin. Agenda. Motivation

johnna
Download Presentation

DAI-Labor Fakultät IV – Elektrotechnik und Informatik Technische Universität Berlin

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Towards a Model-Based Framework for the Development of Adaptive Multimodal User Interfaces for Ambient Assisted Living Environments Marco Blumendorf and Sahin Albayrak DAI-LaborFakultät IV – Elektrotechnik und InformatikTechnische Universität Berlin

  2. Agenda • Motivation • A Runtime Architecture for Adaptive Multimodal User Interfaces • User Interface Models • The Multi Access Service Platform • Conclusion HCI International 2009

  3. Ambient Assisted Living Environments are Smart Environments and reflect Ubiquitous Computing Paradigms HCI International 2009

  4. “Ubiquitous Computing” (Mark Weiser, 1991) “In the 21st century the technology revolution will move into the everyday, the small and the invisible.” HCI International 2009

  5. Smart Objects Smart Devices Smart Networks Smart Environments HCI International 2009

  6. Interaction in Smart Environments used interaction modalities HCI International 2009

  7. Challenges • Complex, distributed system to interact with • Different capabilities and skills of users • Personalization of UIs • Context adaptivity • Continuous, flexible and coherent interaction • Excellent interaction experience • Lack of development tools, methodologies and runtime support HCI International 2009

  8. Agenda • Motivation • A Runtime Architecture for Adaptive Multimodal User Interfaces • User Interface Models • The Multi Access Service Platform • Conclusion HCI International 2009

  9. Requirements • Connect interaction resources • Integrate context • Host the applications • Support UI features(Multimodality, Distribution, Adaptation, …) HCI International 2009

  10. Connecting Interaction Resources HCI International 2009

  11. Integrating Context Sensors HCI International 2009

  12. Application Hosting Application Model Sensors HCI International 2009

  13. Model State Application Model Sensors HCI International 2009

  14. Monitoring State Application Model Monitoring Sensors HCI International 2009

  15. Stimulating Model Execution Application Model Stimulation Sensors HCI International 2009

  16. Processing Components Application Model Sensors HCI International 2009

  17. Agenda • Motivation • A Runtime Architecture for Adaptive Multimodal User Interfaces • User Interface Models • The Multi Access Service Platform • Conclusion HCI International 2009

  18. Mediating between human and computer 10011010101111100100 “turn the light on” HCI International 2009

  19. Mediating between human and computer ? 10011010101111100100 “turn the light on” HCI International 2009

  20. Mediating between human and computer 10011010101111100100 “turn the light on” HCI International 2009

  21. Abstraction Layers See also Szekely 1996, Cameleon Reference Framework (Calvary et al. 2003) Similar concepts can be found e.g. in UsiXML (Limbourg et al. 2004) and TERESA (Berti et al. 2004) HCI International 2009

  22. Mediation Process See also Szekely 1996, Cameleon Reference Framework (Calvary et al. 2003) Similar concepts can be found e.g. in UsiXML (Limbourg et al. 2004) and TERESA (Berti et al. 2004) HCI International 2009

  23. Adding State and Execution Logic • Definition Elements- static structure and constant elements • Situation Elements - state of the model and changing elements • Execution Elements - interpretation process altering state (and definition) at runtime Dynamic Executable Model See e.g. Breton & Bézivin 2001 for details HCI International 2009

  24. Adding State and Execution Logic HCI International 2009

  25. Agenda • Motivation • A Runtime Architecture for Adaptive Multimodal User Interfaces • User Interface Models • The Multi Access Service Platform • Conclusion HCI International 2009

  26. Implementation The Multi-Access Service Platform (MASP) implements the presented concepts Features • Multimodal interaction • Dynamic distribution • Adaptation Different applications have been implemented. HCI International 2009

  27. A Meta UI for Smart Environments HCI International 2009

  28. The Multimodal Cooking Assistant HCI International 2009

  29. Agenda • Motivation • A Runtime Architecture for Adaptive Multimodal User Interfaces • User Interface Models • The Multi Access Service Platform • Conclusion HCI International 2009

  30. Results • We can build models of multimodal applications that dynamically change the used modalities. • Applications can be distributed and dynamically configured at runtime. • A follow me mode shows automatic reconfiguration based on context information. • Dynamic layout changes according to the distance of the user to the screen have been realized. HCI International 2009

  31. Open issues and future work • Technology and software engineering centric • Next Step: Study how to deploy the features properly to provide integration into environment and the required excellent user experience! • Additional Aspects: • multi-user and multi-application scenarios have not been adequately addressed yet • tools also covering runtime issues and moving towards end-user development and customization • integration of natural language processing and models-based development HCI International 2009

  32. References • [Berti 2004] Silvia Berti, Francesco Correani, Giulio Mori, Fabio Paternò, and Carmen Santoro. Teresa: A transformation-based environment for designing and developing multi-device interfaces. In ACM CHI 2004, volume II, pages 793–794, 2004. ACM Press. • [Breton&Bézivin 2001] Erwan Breton and Jean Bézivin. Towards an understanding of model executability. In FOIS '01: Proceedings of the international conference on Formal Ontology in Information Systems, pages 70–80, New York, NY, USA, 2001. ACM. • [Calvary 2003] Gaëlle Calvary, Joëlle Coutaz, David Thevenin, Quentin Limbourg, Laurent Bouillon, and Jean Vanderdonckt. A unifying reference framework for multi-target user interfaces. Interacting with Computers, 15(3):289–308, 2003. • [Limbourg 2004] Quentin Limbourg, Jean Vanderdonckt, Benjamin Michotte, Laurent Bouillon, and Víctor López-Jaquero. Usixml: A language supporting multi-path development of user interfaces. In EHCI/DS-VIS, volume 3425 of Lecture Notes in Computer Science, p. 200–220. Springer, 2004. • [Szekely 1996] Pedro A. Szekely. Retrospective and challenges for model-based interface development. In DSV-IS, pages 1–27. Springer, 1996. • [Weiser 1991] Mark Weiser, The computer for the 21st century, Scientific American, Morgan Kaufmann Publishers Inc., 1991, 265, 66-75 HCI International 2009

  33. The End … Thank you for your attention! Your questions please … marco.blumendorf@dai-labor.de http://masp.dai-labor.de Please visit our booth at the break! Dynamic Executable Model HCI International 2009

More Related