Context-awareness in Digital Library

1. Context-awareness in Digital Library Soon J. Hyun Database Systems Lab. Information and Communications University (ICU)

2. Contents • Introduction • Context Management • Context Model • Taxonomy • Context Awareness in Digital Library • Further Issues

3. Ubiquitous Computing Architecture Introduction

4. Introduction (cont.) • Ubiquitous Requirements • Invisibility • System makes a decision, triggers a command, selects services, and reconfigures itself on behalf of users. • Adaptability • System reconfigures application according to context changes. • Flexibility • System continues services in a dynamic and heterogeneous environment. • Transparency • System allows applications to spontaneously reconfigure with new services or established services without distracting users.

5. Introduction (cont.) • Context Management supports: • Invisibility • Context-awareness enables a computing system to adapt to users proactively without user’s intervention. • Transparency • New Contexts (Sensors, Services, User Models, etc.) can be easily extended to enable a ubiquitous computing system to adapt dynamically to the evolution of user’s environment.

6. Introduction (cont.) • What is Context ? • location, identities of nearby people and objects and changes to those objects [Schilit & Theimer, 1994] • Context is any information that can be used to characterize the situation of an entity [Dey & Abowd, 2000] • Entity - a person, place, or object relevant to the interaction between a user and an application, including themselves • Types of context • Location – position, orientation, velocity, trajectory, etc. • Identity – preference, profile, social relationship, biometrics, etc. • Time – Sequence of events, duration, etc. • Activity – walking, sleeping, sitting, etc. • Task – meeting, reading, working, etc. • Environment – temperature, humidity, brightness, loudness, etc. • Computing resources – device, appliances, etc. • Emotion

7. Introduction (cont.) • What is context-aware computing ? • A system is context-aware if it uses “context” to provide relevant information and/or services to the user, where the relevancy depends on the user’s tasks. • Context-aware functions • Presentation of context information and services to a user • Automatic execution of a service for a user • Attaching context information for later retrieval

8. Context Management • Context Modeling • Ontology-based Modeling • Represents context as hierarchical knowledge structure • Infers implicit high-level context from explicit low-level context • ER Modeling • Define context as a relation and relationship • High speed retrieval and efficient storage but, no more context except a pre-defined context schema • Application-oriented Modeling • Define context for a specific application • Flexible but, lack of formality and reusability

9. Context Management: modeling • Application Oriented Modeling • Context entities and attributes for the “Conference Assistant application” [Dey et al., 2001] • disadvantages • Many context-aware systems model and represent context only for specific application • Lack of formality and reusability • HP’s Cooltown project , Context-Toolkit project

10. Context Management: modeling • Entity-Relationship (ER) • ER model between ‘Conference Room’ and ‘Users’ and ‘User activity’[Wu et al., 2002] • disadvantage • Cannot obtain additional context except for pre-defined context information

11. Context Management: modeling • Ontology approach • Home environment context ontology [Wang et al., 2004] • Pros and cons • Have benefit of reasoning based on Semantic Web technologies • A description logic (DL), theoretical foundation of the ontology is not enough for context model • Only restricted set of first-order logic is allowed for hierarchical classification. • Need to be added additional rules and inference logic according to application requirements • Event-Condition-Action (ECA) rules, Temporal logics, etc.

12. Ontology-based Context Management Light Controller Bed Sensor Light Door Sensor Assertion TurnOff Query Service request Service Discovery Context Widget Context Widget Context Widget Dynamic Reconfiguration Open/Closed On/Empty On/Off/Brightness Light.TurnOff Context Management Context Interpreter Sleep? Sleep(Park, Bedroom) -> TurnOffLight Inference Engine Sleep(Park, Bedroom) <= Status(Door, Closed) ^ Status(Bed, On) ^ Entered(Park, Bedroom) Sleep Context Aggregator Entered Entered(Park, Bedroom) <= Status(Door, Open) ^ Location(Park, Bedroom) Context Base Context Aggregator

13. Taxonomy of Context Management App App App Context-Oriented Programming (COP) RCSM(CA-IDL), GAIA, m3(CA-Rule) Context-aware Application Programming Interface VTT(API), MUSE(4GL) Context Access High-level Context Representation GAIA(FOLogic), MeCentric(DL) Logic-based Inference Probabilistic Machine-Learning TEA (MM), VTT (Bayesian) Low-level Context Representation VTT, COBRA Context Toolkit Context Driver Context Cues(TEA) Context Abstraction, Quantization Service Abstraction STEER(DAML-S) Sensors Services Devices

14. Low-level Context Representation • Context Broker Architecture (CoBrA) [Chen, 2003]

15. Low-level Context Representation (cont.) • VTT Research Center in Finland [Korpipaa et al., 2003] • Leverage the part of MPEG-7 terminology for audio-based contexts Sensor-based Context Ontology Main Categories of Context Ontology Audio-based Context Ontology

16. Low-level Context Representation (cont.) • SOCAM [Wang et al., 2004] • Develop ontology-based context model using OWL that addresses several existing problems such as context dependency and quality of context. • Divide into High-level ontology and Domain-specific ontology. • However, High-level ontology only used to support for categorization of context.

17. Low-level Context Representation (cont.) • SOCAM

18. Logic-based Inference • Gaia [Ranganathan and Campbell, 2003] • first order predicate logic

19. Logic-based Inference (cont.) • HP MeCentric [Perich, 2002] • Atwork domain • An entity can be member of atwork domain only when it is a member of morning or afternoon domain and at the same time the entity must be a member of inCubicle domain. Fig. 5 Graph Representation of the atwork domain

20. VTT Research Center in Finland Low level acquisition Fuzzy Membership Function High level acquisition Naïve Bayes Classifier for higher level contexts Probabilistic Machine Learning

21. Probabilistic Machine Learning (cont.) • Technology for Enabling Awareness (TEA) [Van Laerhoven et al., 2001] • Adds an array of hardware sensors to the system • Use machine learning techniques to make the system learn the context-description from its user on the spot • Eight Sensor outputs in five different contexts • “inside a dark office”, “inside an artificially lit office”, “moving inside an artificially lit office”, “outside still”, “outside moving” • The sensor board was turned off after each contexts was recorded and turned back on when it was brought in a new context. TEA hardware board (light, temperature, accelerometers, microphone, pressure, IR, touch, CO Sensors) An example of 8 sensor outputs during 832 secs

22. Probabilistic Machine Learning (cont.) • Neurons are activated topologically for tasks depending on the sensory input. • It is possible to monitor the activation of the neurons and plot the resulting matrix as a landscape, where different hills ideally represent different contexts. • The only necessary user interaction is the labeling of clusters produced by the SOM. • A cluster is labeled, so classification is possible • There is no label, then a distance weighted K-Nearest Neighbors algorithm to search for the closest label on SOM Activity-plot of a SOM Schematics of the user involvement

23. Probabilistic Machine Learning (cont.) • Probabilistic finite state machine architecture(Markov Chain) • SOM has correctly identified the first few context profiles, but there remains some doubt on what the next will be. • If the previous context was “in a train”, the next would be “in a train station” rather than “in a bedroom”. • Each context is represented by a state. • Transitions are represented by edges between states. • The model keeps a probability measure for each transition, so every time a transition occurs, the supervision model can check if this really is likely. • SOM can be interpreted as a short-term pattern recognition of specific contexts, while Markov chain model can be seen as long-term pattern recognition of general user behavior with relation to contexts. Fig. 3 Overall architecture

24. Context-awareness for Digital Library • Adaptation • Beyond anytime, anywhere information access • Recommendation • manages information overload • helps a user choose from among an overwhelming number of possibilities • Multimedia Support • Images • Antiques, Arts, Medical archives, etc. • AoD/VoD • Music, Seminars, etc.

25. Digital Library Application • Books with RFID Tags [Kyushu University Library] • Efficient Circulation (Loan/Return) • Self-Checkout system • Efficient Inventory • 10~30 times faster than barcode system • RFID Library Card • Possible new services • Recommendation of books based on patrons’ profile/preference • Navigation to books • Automated Storage/Retrieval System

26. Digital Library Application • Automated Storage/Retrieval System

27. Embedding, Partial Links Van Gogh's: Masterpieces from the Van Gogh Museum, Amsterdam, will be based entirely on the holdings of the Van Gogh Museum. The exhibition will illustrate Van Gogh's entire career, from the Potato Eaters of 1885 through Wheatfield of Crows of 1890, the year of his death. It will include such famous works as the Self Portrait as an Artist (1888) The Zouav (1888), The Bedroom (1888e,) and The Harvest (1888). Embedding, Total Link Referential Link VDoc Instantiation Van Gogh The exhibition will illustrate Van Gogh's entire career, from the Potato Eaters of 1885 through Wheatfield of Crows of 1890, the year of his death. It will include such famous works as the Self Portrait as an Artist (1888) The Zouav (1888), The Bedroom (1888e,) and The Harvest (1888). Digital Library Application • Virtual Documents [Myaeng, 2000] • “documents for which no persistent state exists and for which some or all of each instance is generated at runtime.”

28. Digital Library Application • According to user’s context/situation • compose and provide virtual documents proactively • moving to a conference session • Walking - Titles only • Standing - Titles & Images • Sitting on a chair - Full documents • according to user’s preferences • device capability, network bandwidth, battery • Environment, etc.

29. Further Issues • Dynamicity • Context information changes continually, so all context information cannot be pre-defined when developing context ontology. • Uncertainty • Context information cannot be consistent because of highly dynamic nature of pervasive computing. • Privacy • Social Contexts • Sharing information & Group contexts

30. Q & A

31. References [1] M. Satyanarayanan, “Pervasive Computing: Vision and Challenges,” IEEE Personal Communications, Vol. 8, Issue 4, 2001, pp. 10-17. [2] D. Garlan, D. P. Siewiorek, A. Smailagic, and P. Steenkiste, “Project Aura: Toward Distraction-Free Pervasive Computing,” IEEE Pervasive Computing, Vol. 1, Issue 2, 2002, pp. 22-31. [3] A. Chen, R. R. Muntz, and S. Yuen, “A Support Infrastructure for the Smart Kindergarten,” IEEE Pervasive Computing, Vol. 1, Issue 2, 2002, pp. 49-57. [4] T. Kindberg and A. Fox, “System Software for Ubiquitous Computing,” IEEE Pervasive Computing, Vol. 1, Issue 1, 2002, pp. 70-81. [5] B. Johanson, A. Fox, and T. Winograd, “The Interactive Workspace Project: Experiences with Ubiquitous Computing Rooms,” IEEE Pervasive Computing, Vol. 1, Issuee 2, 2002, pp. 67-74. [6] S. S. Yau, F. Karim, Y. Wang, B. Wang, and S. K.S. Gupta, “Reconfigurable Context-Sensitive Middleware for Pervasive Computing,” IEEE Pervasive Computing, Vol. 1, Issue 3, 2002, pp. 33-40. [7] J. Anhalt, et. al., “Toward Context-Aware Computing Experiences and Iessons,” IEEE Intelligent Systems, Vol. 16, Issue 3, 2001, pp. 38-46. [9] K. Cheverst, N. Davies, K. Mitchell, A. Friday, and C. Efstratiou, “Developing a context-aware electronic tourist guide: some issues and experiences,” Proc. CHI 2000, Hague. [10] H. Yan and T. Selker, “Context-aware office assistant,” Proc. 5th Int’l Conf. on Intelligent User Interfaces, 2000, New Orleans. [11] A. Harter, A. Hopper, P. Steggles, A. Ward, and P. Webster, “The anatomy of a Context-aware application,” Wireless Networks, Vol. 8, Issue 2/3, March-May 2002. [12] K. Cheverst, N. Davies, K. Mitchell, and A. Friday, “Experiences of developing and deploying a context-aware tourist guide: the GUIDE project,” Proc. Int’l Conf. on Mobile Computing and Networking, Boston, 2000. [13] G. Chen, and D. Kotz, “Context aggregation and dissemination in ubiquitous computing systems,” Proc. 4th IEEE workshop on Mobile Computing and Applications, pp. 105-114, 2002. [14] A. Ranganathan, R. H. Campbell, and A. Mahajan, “ConChat: a context-aware chat program,” IEEE Pervasive Computing, Vol. 1 Issue 3, pp. 51-57, July-Sept. 2002.

32. References [15] D. Mandato, E. Kovacs, F. Hohi, and H. Amir-Alikhani, “CAMP: a context-aware mobile portal,” IEEE Comm. Magazine, Vol. 40, Issue 1, pp. 90-97, 2002. [16] N. H. Cohen, H. Lei, P. Castro, J. S. Davis II, and A. Purakayastha, “Composing pervasive data using iQL,” Proc. 4th Int’l Workshop on Mobile Computing Systems and Applications, pp. 94-104, 2002. [17] K. Van Laerhoven and K. Aidoo, “Teaching Context to Applications,” Journal of Personal and Ubiquitous Computing, Vol. 5, Issue 1, Feb. 2001, pp. 46-49. [18] P. Korpipaa, J. Mantyjarvi, J. Kela, H. Keranen, and E. J. Malm, “Managing context information in mobile devices,” IEEE Pervasive Computing, Vol. 2, Issue 3, 2003, pp. 42-51. [19] J. Mantyjarvi and T. Seppanen, “Adapting Applications in handheld devices using fuzzy context information,” Journal of Interacting with computers, Vol. 15, Issue 4, pp. 521-538, 2003. [20] S. Madden and M. J. Franklin, “Fjording the Stream: An Architecture for Queries over Streaming Sensor Data,” Proc. of IEEE ICDE, pp. 555-566, 2002. [21] N. H. Cohen, A. Purakayastha, L. Wong, and D. L. Yeh, “iQueue: A Pervasive Data Composition Framework,” Proc. of IEEE MDM, pp. 146-153, 2002. [22] P. Bonnet, J. Gehrke, and P. Seshadri, “Querying the Physical World,” IEEE Personal Communications, Vol. 7, Issue 5, pp. 10-15 2000. [23] W. Y. Lum and F. C.M. Lau, “A Context-aware Decision Engine for Content Adaptation,” IEEE Pervasive Computing, Vol. 1, Issue 3, pp. 41-49, 2002. [24] T. Saridakis, “Dependability and Configurability: Partners or Competitors in Pervasive Computing?” SAFECOMP 2002, LNCS 2434, pp. 309-320. [25] K. Henricksen, J. Indulska, and A. Rakotonirainy, “Modeling Context Information in Pervasive Computing Systems,” LNCS 2414, pp. 167-180. [26] L. Rudolph, “Project Oxygen: Pervasive, Human-Centric Computing - An Initial Experience,” CAiSE 2001, LNCS 2068, pp. 1-12.

33. References [27] F. Michahelles, M. Samulowitz, and B. Schiele, “Detecting Context in Distributed Sensor Networks by Using Smart Context-Aware Packets,” ARCS 2002, LNCS 2299, pp. 34-47. [28] L. Capra, W. Emmerich, and C. Mascolo, “CARISMA: Context-Aware Reflective mIddleware System for Mobile Applications,” IEEE Trans. on Software Engineering, Vol. 29, Issue 10, pp. 929-945, Oct. 2003. [29] A. Ranganathan, and R. H. Campbell, “An Infrastructure for Context-Awareness based on First Order Logic,” Journal of Personal and Ubiquitous Computing, Vol. 7, Issue 6, pp. 353-364, Dec. 2003. [30] M. Roman, C. Hess, R. Cerqueira, A. Ranganathan, R. H. Campbell, and K. Nahrstedt, “A Middleware Infrastructure for Active Spaces,” IEEE Pervasive Computing, Vol. 1, Issue 4, pp. 74-83, 2002. [31] H. Chen, “An Intelligent Broker Architecture for Context-Aware Systems,” PhD. dissertation proposal, the University of Maryland Baltimore County, USA, Jan., 2003. [32] R. Masuoka, Y. Labrou, B. Parsia, and E. Sirin, “Ontology-Enabled Pervasive Computing Applications,” IEEE Intelligent Systems, Vol. 18, Issue 5, pp. 68-72, 2003. [33] Pervasive Computing Group, “Middleware Infrastructure for Active Surroundings,” Proc. of Korean Ubiquitous Computing Workshop, Pyungchang, Korea, Feb. 9, 2004, pp. 134-141. [34] P. Bonnet, J. Gehrke, and P. Seshadri, “Querying the Physical World,” IEEE Personal Communications, Vol. 7, Issue 5, pp. 10-15, 2000. [35] H. Wu, M. Siegel, and S. Ablay, “Sensor Fusion for Context Understanding”, Proceedings of IEEE Instrumentation and Measurement Technology Conference, Anchorage, USA, May 2002. [36] Xiao Hang Wang, Da Qing Zhang, et al. “Ontology Based Context Modeling and Reasoning using OWL”, pervasive Computing and Communication Workshops, 2004. Proceedings of the Second IEEE Annual Conference pp. 18-22 [37] A. K. Dey, G. D. Abowd, and D. Salber, “A Conceptual Framework and a Toolkit for Supporting the Rapid Prototyping of Context-Aware Applications, HCI Journal, Vol. 16(2-4), pp. 97-166. [38] A. Rakotonirainy, “Context-Oriented Programming for Pervaisve Environments” University of Queensland Technical Report September 2002.