Mr. Amjad Usman 19-July-2014 KHU

1. Mr. Amjad Usman 19-July-2014 KHU

2. Presentation Outline Introduction Motivation Related Work Limitations Proposed Architecture Tools and Technologies Development Timeline Current Status

3. Introduction • According to Dey, Abowd and Salber (2001) • Context is any information that can be used to characterize the situation of entities (i.e. whether a person, place or subject) that are considered relevant to the interaction between a user and an application, including the user and the application themselves • Means Contextis very dynamic and transient • Low-level Context is the raw data coming from sensors / source • High-level Context is the abstract information extracted from logically relevant low-level context

4. Motivation • End-users are interested in high-level context, not raw data • For example “It’s raining” (high-level) is preferred rather than “humidity: 77%“ (low-level) • To discover Higher-level Context Information • Context fusion is required • Find the semantic relationship • By integrating all the information relevant to one’s life • It can give us a real sense of Life logging

5. Motivation Context Information Lifestyle Analysis, Prediction, and Recommendations Behavior Modeling Context Information Context Information • Recommending Services • Lifestyle & Lifecare • Exercise and nutrition • Social interaction • Behavior analysis & prediction • Context Information • Sensor data from sensors • Activity data from smartphones • Nutrition data • Social media data

6. Proposed Idea • Activity Activity Information Diet Information Integration Social Media Information Behavior Modeling Environment Information ProfileInformation Context Information  Independent logs Life log Information  Semantically related

7. Related Work

8. Related Work

9. Limitations • Domain Specific Context Models • Information exist in different types of Logs but are stand alone • No integrated system • Very few systems consider social interaction • None of the existing systems use any behavioral model • High level contexts are extracted using context interpretation and aggregation techniques regardless of • Context verification, validation and fusion

10. Proposed Architecture Context Interpreter Context Awareness Long/Short-term Behavior Analysis Decision Making Decision Propagation Situation Analyzer Behavior Modeling Parser Context Analyzer Pattern Classification Rule base Query Generator Match Making Prediction and Reasoner Feature Selection Pattern Identification Activity Retrieval Rule-based Filtering Life log Extractor Mapper and Transformer OWL Ontology XML2OWL DTD2OWL Life log Modeling Life Log Repository Low Level Context-awareness Parser Recommendation Manager Context Logging Context Fusion Context Verification HDFS Data Interface Intermediate Data Context Representation Context Receiver

11. Long/short term Behavior Analysis Scenario 7 7 7 Recommendation Manager Prediction and Reasoner Context Awareness | Long/Short-term Behavior Analysis  Taking Lunch in Kitchen Ali Exercising Sentiment: tired Context Interpreter 6 Decision Making "SELECT ?activityName ?hasConsequentAction ?type ?performedBy ?time WHERE { <" + strNS + strActivity + "> <" + strNS + "hasName> ?activityName ." + "<" + strNS + strActivity + "> }; Decision Propagation Situation Analyzer  In Kitchen  Eating  12:00:00 3 Parser 4 Context Analyzer Rule 2 Matched (activity=eating AND time=12:00:00)  Taking Lunch Query Generator Rule 1 (activity=eating AND time<8:00:00)  Taking Breakfast Rule 2 (activity=eating AND time=12:00:00)  Taking Lunch Rule 3 (activity=eating AND time=18:00:00)  Taking Dinner Match Making Rule base <activity>eating</activity> <activityLocation>Kitchen </activityLocation> <activityTime>12:00:00 </activityTime> <performedBy>Ali </performedBy> 5 Activity Retrieval Rule-based Filtering 2 Mapper and Transformer XML2OWL OWL Ontology DTD2OWL HDFS Data Interface Activities detected: walking Location detected: Kitchen • Time noticed: 12:00:00 Intermediate Data 1 1 Low Level context-awareness

12. 10 10 Recommendation Manager Prediction and Reasoner Scenario Behavior Analysis Lunch No proper timing Exercising Regular ContextAwareness | Long/Short-term Behavior Analysis 9 Behavior Analysis Pattern Classification Pattern Identification Feature Selection Complete Information of User like profile, activities performed when and where, tweets, etc 5 8 Context Fusion Life Log Extractor Horizontal Context Fusion Vertical Context Fusion Data Processing Data Fetcher Lunch, Exercising 4 Verified Data Context Verification and Logging Life Log Repository 7 Log Context Existence Verification <activity>eating</activity><activityLocation>Kitchen</activityLocation><activityTime>12:30:00</activityTime><performedBy>Ali</performedBy> 6 Consistency Verification Parser Data Extractor Data Logger Query Formulation Semantic Structural 3 Context Representation Context Representation Context Converter Context Mapper walking, sitting, eating, tweet: tired 2 Context Receiver High Level Context: Exercising Structured Data Ali,Lunch,Hotel,2014-05-02, 12:00:00 1 1 1 HDFS Data Interface High Level Context-awareness Low Level context-awareness Activity detected: walking, eating Location detected: Kitchen Time: 12:30:00 Intermediate Data

13. Tools and Technologies • Knowledge Representation • RDF / RDFS / OWL Ontologies • Protégé as IDE • Programming Language & API • Java • Jena, Twitter • Querying Language • SPARQL • Reasoner • Racer Pro / Pellet / FaCt++

14. Development Time Line Four Year Work Break Down Second Year First Year Fourth Year Third Year Literature Study: Context Modeling Literature Study: Life log System Literature Study: Behavior Representation System Development Context Model Life log Model Features Selection Behavioral Model Creation High-level Context Awareness Analytical Study: Context Conversion Context logging in Life log Study: Behavior Modeling Unit Testing Life log Parser Context Mapping Technique Model Selection Pattern Identification Test Report Analytical Study: Context Interpretation Context Integration & Verification Algorithmic Study: Behavior Analysis Integration Testing Context Analyzer Fusion + Verification Techniques Algorithm: Long-term / Short-term Analysis Test Report Context awareness Life log Management Behavioral Analysis Documentation Prototype Development Prototype Development Prototype Development Technical Guide

15. Current Status • Context Awareness • Literature Survey regarding • Context Modeling Techniques • Context Conversion & Mapping Algorithms • Long/Short-term Behavior Analysis • Literature Survey regarding Lifelog Design and Development • LifeLog Model: OWL+Dynamic/Static • Storage: Ont-RDB / JenaTDB / RDF • Nature & type of Logs

16. Context Modeling Work Flow

17. What to store in Life log?

18. Behavior Analysis & Prediction Work Flow

19. Thank You! Email address: amjad.usman@oslab.khu.ac.kr Cell Number: +82-10-4769-8867

20. Long/Short-term Behavior Analysis Deployment Diagram

21. Proposed Architecture Recommendation Manager Prediction and Reasoner Long/Short-term Behavior Analysis Context Awareness Low Level context-awareness HDFS Data Interface Intermediate Data

22. Long/Short-term Behavior Analysis Proposed Architecture Recommendation Manager Prediction and Reasoner Context Awareness | Long/Short-term Behavior Analysis Context Interpreter Decision Making Decision Propagation Situation Analyzer Parser Context Analyzer Rule base Query Generator Match Making Rule-based Filtering Activity Retrieval Mapper and Transformer XML2OWL OWL Ontology DTD2OWL Low Level context-awareness HDFS Data Interface Intermediate Data

23. Proposed Architecture Context Awareness | Long/Short-term Behavior Analysis Prediction and Reasoner Recommendation Manager Behavior Analysis Life Log Extractor Context Fusion (Vertical / Horizontal) Life Log Repository Context Verification and Logging Parser Context Representation Context-Awareness Context Receiver HDFS Data Interface Low Level context-awareness Intermediate Data

24. Proposed Architecture Context Interpreter Context Awareness Long/Short-term Behavior Analysis Decision Making Decision Propagation Situation Analyzer Behavior Analyzer Behavior Modeling Parser Context Analyzer Rule base Query Generator Match Making Behavior Checker Model Validator Prediction and Reasoner Activity Retrieval Rule-based Filtering Behavior Descriptor Model Creater Life Log Extractor Mapper and Transformer OWL Ontology XML2OWL DTD2OWL Lifelog Modeling Life Log Repository Parser Low Level Context-awareness Context Logging Context Fusion Context Verification Recommendation Manager Context Representation HDFS Data Interface Intermediate Data Context Receiver

25. Social Media Information Environment Information Diet Information Activity Information ProfileInformation Long / Short-term Behavior Modeling Motivation • To integrate the different context information emerging from diverse sources to identify user’s behavior in order to analyze the user’s lifestyle and provide recommendations to promote active lifestyle Life log

26. Motivation Activity Information Activity Information Diet Information Short-term Behavior Modeling Diet Information Integration Social Media Information Environment Information Environment Information Social Media Information ProfileInformation Profile Information Long-term Behavior Modeling