Middleware for Context-aw are Ubiquitous Computing (In preparation for Distributed System Course)

1. Middleware for Context-aware Ubiquitous Computing(In preparation for Distributed System Course) Hung Q. Ngo Ubiquitous Computing Group (UCG) Real-time and Multimedia LAB Kyung Hee University, Korea Sept., 2004

2. Sequence of presentation • Introduction • Middleware for Context-Awareness • CAMUS Architecture • Feature Extraction Agent • CAMUS Context Model • Reasoning Mechanisms • System Prototyping • Conclusion and Future Work

3. Introduction • Ubiquitous/Pervasive Computing • Calm technology: embedded, invisible, seamlessly, unobtrusive, intelligent. Image source: Friedemann Mattern (ETH Zürich)

4. Introduction (cont.) • Context-awareness • An important aspect of the intelligent pervasive computing systems • Systems that can anticipate users’ needs and act in advance by “understanding” their context.

5. Introduction (cont.) • Context • The specific conditions, external to the application itself, such as audience, speaker (user), situation (place and its surroundings), time, environmental and network conditions, etc., which determine the application behavior, will be called the ‘context’ of the application. • “Out of Context: Computer Systems That Adapt to, and Learn from, Context,” H. Lieberman, T. Selker, MIT • “A Survey of Context-Aware Mobile Computing Research,” by G. Chen, D. Kotz, Dartmouth College • “Context-Aware Applications Survey,” M. Korkea-aho, Helsinki University of Technology

6. Introduction (cont.) Context – in our viewpoint: • the situational conditions that are associated with a user location, surrounding conditions (light, temperature, humidity, noise level, etc), social activities, user intentions, personal information, etc.

7. Introduction (cont.) • Sensing is a key enabling technology • Heterogeneity • e.g. location-sensing techniques: triangulation, proximity, scene analysis • (Jeffrey Hightower, “Location Systems for Ubiquitous Computing”, IEEE Computer August 2001.) In the environment Wearable On devices

8. Introduction (cont.) • Shortcomings of the previous systems • Context acquisition and use was often tightly integrated into a single application, and could not easily be incorporated into other applications. • Individual agents are responsible for managing their own context knowledge

9. Introduction (cont.) • Shortcomings of the previous systems • Lacking an adequate representation for context modeling and reasoning • Existing solutions for Context information are • Name-value pairs or entity relation model • Objects to represent context with methods and fields for retrieval of information • Simple matching mechanisms for context access, and developers must perform low-level programming for reasoning • Users often have no control over the information that is acquired by the sensors • Privacy Concerns

10. Middleware for Context-Awareness • Desired Characteristics: • Support for heterogeneous and distributed sensing agents • Make it easy to incrementally deploy new sensors and context-aware services in the environment • Provide different kinds of context classification mechanisms • Different mechanisms have different power, expressiveness and decidability properties • Rules written in different types of logic (first order logic, description logic, temporal/spatial logic, fuzzy logic, etc.) • Machine-learning mechanisms (supervised/unsupervised classifiers)

11. Middleware for Context-Awareness • Desired Characteristics (cont.) • Follow a formal context model using ontology • To enable syntactic and semantic interoperability, and knowledge sharing between different domains • Facilitate for applications to specify different behaviors in different contexts easily, as well as privacy policy and security mechanism • Graphical development tool to ease developers in writing code.

12. CAMUS Core Architecture CAMUS ---------------------- Context- Aware Middleware for Ubicomp Systems

13. CAMUS Details • Feature Extraction Agents • These sensing agents extract the most descriptive features for deducing contexts in upper layers • Feature - Context Mapping: performs the mapping required to convert a given feature into elementary context based on the meta-information saved in the ontology repository • Self-configuring components: • Access wrappers • Unification interface for acquiring contexts from sensors and delivering to consumers.

14. Context Recognition Feature Encapsulation Feature Segmentation Feature Extraction Sensor signal Processing Feature Extraction Agent (cont.) Table 1. Meta-data for feature encapsulation

15. Feature Extraction Agent (cont.) Table 2. Examples of features and their semantic meaning. Sensor_IDs {3,7} are mapped to “Bedroom” FT= {Sesnor_ID, Type_ID, Feature_ID, Feature_Value, Probability, Timestamp} FT1= {3, 1, 1, 1, 0.9, xxxxx} {Location.Bedroom, Environment.Sound.Intensity = Silent, Probability = 0.9, TimeStamp = xxxxx}

16. Formal Context Modeling • In order to have common understanding among HW/SW entities, they must have invariant meanings, i.e. the context semantics must be formalized • Advantages • Storing context for long-term • Communicating context universally with other systems • Leads to its testability of being a formalized knowledge • Result • A growing pool of well-tested context knowledge available to different context aware systems

17. Formal Context Modeling (cont.) • Formal Context Modeling using OWL • Context entities are concepts in a domain under investigation • Ontologies are used for above purpose, defined as • Specification of a conceptualization of a domain • Domain contains the vocabularies of concepts and relationships among them • @syntactic level • They are XML documents • @structure level • They consist of a set of triples • @semantic level • They constitute one or more graphs with partially pre-defined semantics

18. CAMUS Context Model Basic Model

19. Context Model in detailsAgent ontology

20. Context Model in detailsDevice ontology • Based on FIPA device ontology specification: knowledge reuse

21. Reasoning Mechanisms • The contextual information provided by the environment leads to only elementary contexts • Some contexts are useful only when they are combination of some elementary and/or composite contexts, and also need consistency of contextual information

22. Reasoning Mechanisms (cont.) • Our framework supports various pluggable reasoning modules and developer of the context Aggregator services can exploit any kind of reasoning mechanism based on application requirements • These reasoning modules are broadly classified intoontology & context reasoning mechanisms

23. Reasoning Mechanisms (cont.) • Inference services in DL (Description Logic) Terminological Reasoning • Subsumption check - checks if one concept is a sub-concept of another • Consistency check - checks for (in)consistency of concept definitions • Taxonomy construction - explicit concept hierarchy • Classification - determines the concepts that immediate subsume or are subsumed by a given concept

24. Reasoning Mechanisms (cont.) • Inference services in DL (Description Logic) Instance Reasoning • Consistency check - existence of a model of А • Realisation - given a partial description of an instance, finds the most specific concepts that describe it • Individual retrieval - finds all instances that are described by a given concept

25. Examples for Ontology Reasoning

26. Examples for Ontology Reasoning

27. Reasoning Mechanisms (cont.) *An example of Context Reasoning: Using Bayesian Net to deduce User Activity

28. System Prototyping • A Smart Home has been set up for system prototyping. • RFID tags, Door Sensors, Bluetooth and WLAN access points are used for location and identity detection • Audio/Video based activity recognition

29. System Prototyping • A Smart Home has been set up for system prototyping. • Jini is used as underlay communication and discovery mechanisms • Jena2 Semantic Web Toolkit is used to implement the Context Repository, Context Reasoner, and Context Query Engine. • OSGi Gateway for interconnection with other systems

30. Our Research Contributions • A Middleware Framework for building Context-Aware applications • Unified Sensing Framework for Context acquisition from disparate sensors • Abstractions to relieve the developers of the burden of low level interaction with various hardware devices • Formal Context Modeling in terms of ontologies using OWL • To enable syntactic and semantic interoperability, and knowledge sharing between different domains • Ontology and Context Reasoning Mechanisms • Pluggable Modules

31. Conclusion and Future work • We believe thatformalizing domainsshould be seen as emergent phenomenon constructed incrementally, leading to the sharing of contextual information among heterogeneous context-aware systems • API for application developers to access system’s functionalities while hiding the complexity of underlying context processing (gathering contexts from sources, managing contexts in knowledge base, handling application queries, and reasoning about contexts based on rules) • Privacy policy and security mechanism

32. Selected References • [1] M. Weiser, “The Computer for the 21st Century”, Scientific America (Sept. 1991) 94-104; reprinted in IEEE Pervasive Computing. (Jan.-Mar. 2002) 19-25 • [2] M. Satyanarayanan, “Pervasive Computing: Vision and Challenges”, IEEE Personal Communications (Aug. 2001) 10-17 • [3] Dey, A.K., et al., “A Conceptual Framework and a Toolkit for Supporting the Rapid Prototyping of Context-Aware Applications”, Anchor article of a special issue on Context-Aware Computing, Human-Computer Interaction (HCI) Journal, Vol. 16. (2001) • [4] S. Jang, W. Woo, “Ubi-UCAM: A Unified Context-Aware Application Model”, Context 2003, Stanford, CA, USA. (Jun. 2003) • [5] J. Hong, “The Context Fabric”, http://guir.berkeley.edu/projects/confab/ • [6] Kumar, M.; Shirazi, B.A.; Das, S.K.; Sung, B.Y.; Levine, D.; Singhal, M, “PICO: a middleware framework for pervasive computing”, IEEE Pervasive Computing, Vol. 2 Issue 3. (July – Sept, 2003) 72- 79 • [7] Chen Harry, Tim Finin, and Anupam Joshi, “An Intelligent Broker for Context-Aware Systems”, Ubicomp 2003, Seattle, Washington. (Oct. 2003) • [8] Hung Q. Ngo, Anjum Shehzad, S.Y.Lee, “Developing Context-Aware Ubiquitous Computing Systems with a Unified Middleware Framework”, accepted for publication, The 2004 International Conference on Embedded and Ubiquitous Computing (EUC04), Aizu-Wakamatsu City, Japan, 25-27 August, 2004. • [9] Anjum Shehzad, Hung Q. Ngo, S.Y. Lee, “Formal Modeling in Context Aware Systems”, KI-Workshop Modeling and Retrieval of Context (MRC2004), German.

33. For further discussion: Hung Q. Ngo Ubiquitous Computing Group RT&MM LAB Kyung Hee Univ. KOREA Email: sylee@oslab.khu.ac.kr www: http://ucg.khu.ac.kr/~nqhung