1 / 27

Zhiphone: A Mobile Phone that Learns Context and User Preferences

Zhiphone: A Mobile Phone that Learns Context and User Preferences. Jing Michelle Liu Raphael Hoffmann CSE567 class project, AUT/05. Objective. Adapt the alarm type of a mobile phone to its context. Objective.

serge
Download Presentation

Zhiphone: A Mobile Phone that Learns Context and User Preferences

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Zhiphone: A Mobile Phone that Learns Context and User Preferences Jing Michelle Liu Raphael Hoffmann CSE567 class project, AUT/05

  2. CSE567 project

  3. Objective • Adapt the alarm type of a mobile phone to its context. CSE567 project

  4. CSE567 project

  5. Objective • Adapt the alarm type of a mobile phone to its context and the preferences of its user. CSE567 project

  6. Track context Receive call User reaction Learn user preferences This is what you want CSE567 project

  7. Outline • Related Work • Architectural Overview • Feature Extraction • Learning Context • Learning User Preferences • Future Work CSE567 project

  8. Related Work • SenSay: A Context-aware mobile phone • Four states: uninterruptible, idle, active and normal • Five functional modules: the sensor box, sensor module, decision module, action module and phone module • Sensors include light, motion and microphone • Query sensor data and the electronic calendar of the user • Use thresholds to classify into different states • There is no learning from user preferences CSE567 project

  9. Related Work • SenSay: A Context-aware mobile phone • Four states: uninterruptible, idle, active and normal • Five functional modules: the sensor box, sensor module, decision module, action module and phone module • Sensors include light, motion and microphone • Query sensor data and the electronic calendar of the user • Use thresholds to classify into different states • There is no learning from user preferences CSE567 project

  10. Architectural Overview ringing tone vibration voice mail alarm type  Learning user preferences Reinforcement Learning online soft sensors close to body conversation extr. noise outdoor physical act.  Learning context Supervised Learning offline features FFT1 variance1 variance2 derivative3 Feature Extraction raw data CSE567 project

  11. Feature Extraction • potentially useful features • running mean • running variance • derivative • exponential smoothing • frequency components (FFT) • short (10 sec) and long (1 min) running windows 010111011010010011010100011110101010111010010111010100011010101 CSE567 project

  12. Feature Extraction (cont’d) • system of pluggable components(using observer/composition design patterns) FFT filter window filter mean filter window filter mean filter joinfilter raw sensor window filter var. filter CSE567 project

  13. Feature Extraction - Experiments • measured execution times on the iPAQ CSE567 project

  14. soft sensors close to body conversation extr. noise outdoor physical act.  Learning context Supervised Learning offline Architectural Overview ringing tone vibration voice mail alarm type  Learning user preferences Reinforcement Learning online features FFT1 variance1 variance2 derivative3 Feature Extraction raw data CSE567 project

  15. Learning Context • identified five relevant soft sensors • Close to Body • Conversation • Extreme Noise • Outdoor • Physical Activity • Using annotated user traces, we build a Support Vector Machine classifier for each soft sensor offline CSE567 project

  16. ? ? Learning Context (cont’d) • Support Vector Machines ? + + + + + Feature2 + - + + + + + + + + - - - - - - Feature1 CSE567 project

  17. Learning Context (cont’d) • Support Vector Machines + + + + + Feature2 + - + + + + + + + Maximize distance between closest point and boundary(Optimization) + - - - - - - Feature1 CSE567 project

  18. Learning Context (cont’d) • Support Vector Machines w/ Gaussian Kernel - - Feature2 - + + - + + + + + + + + + + - + + - - - Feature1 CSE567 project

  19. Learning Context – Experiments • recorded and annotated traces at Allen Center CSE567 project

  20. Learning Context – Experiments (cont’d) • 2-fold Cross-Validation results (in %) CSE567 project

  21. Learning Context – Experiments (cont’d) • measured execution times on iPAQ and desktop online offline CSE567 project

  22. CSE567 project

  23. ringing tone vibration voice mail alarm type  Learning user preferences Reinforcement Learning online Architectural Overview soft sensors close to body conversation extr. noise outdoor physical act.  Learning context Supervised Learning offline features FFT1 variance1 variance2 derivative3 Feature Extraction raw data CSE567 project

  24. Learning User Preferences • Scenario 1 • Soft sensors detect CloseToBody, Conversation • Call comes in and phone is ringing • User hangs up phone without taking call • Scenario 2 • Soft sensors detect CloseToBody, Physical Activity • Call comes in and phone vibrates • User does not notice call big negativereward small negativereward CSE567 project

  25. Learning User Preferences • Reinforcement Learning (Q-Learning) • Neural Network soft sensor states new alarm type + current alarm type CSE567 project

  26. What we can do in the future • use an open-source VoIP library, e.g. JVOIPLIB, to enable real cell phone capability • apply more advanced reinforcement learning on user preferences • evaluate the effectiveness of learning user preferences CSE567 project

  27. End CSE567 project

More Related