WalkSafe : A Pedestrian Safety App

1. WalkSafe: A Pedestrian Safety App Presented by, Ramya Deepa Palle CS 541

2. Paper Overview: • Summarizes a historical view on a conference about mobile sensors development • Discuss about path toward cognitive phones • Explains few phone applications based on mobile sensors • Discuss problems and solution while analyzing Big Sensor Data

3. Sensors in mobile: • Mobile phone or smartphone becomes the most important communication device in people lives. • A smartphone is a mobile phone offering advanced capabilities, often with PC-like functionality • Hardware (Apple iPhone 3GS as an example) • CPU at 600MHz, 256MB of RAM • 16GB or 32GB of flash ROM • Wireless: 3G/2G, WiFi, Bluetooth • Sensors: camera, acceleration, proximity, light • Functionalities • Communication • News & Information • Socializing • Gaming • Schedule Management etc.

4. Embedded sensors to phone as a sensor • The view moved away from traditional embedded sensors to phone as a sensor.

5. Small history: • In early 2007 Nokia released a phone with an embedded accelerometer. • They did not mention about this in N95 because it was only there for video stabilization and photo orientation • When Nokia’s peter Boda (the SensorPlanet leader) visited Dart-mouth college, he casually mentioned that the N(% included an embedded accelerometer. • This, along with the phone’s GPS sensors, led researchers to study and develop new phone-based sensing applications, such as the CenceMe app.

6. CenceMe • CenceMe, the first social sensing application running on mobile phones. • Sensor application • Accelerometer • Microphone • GPS • Bluetooth • Track user information • Physical activity • Sitting, walking, running or driving • Social interaction • Public conversation • Share using social networking applications • Ex. Facebook

7. CenceMe App Analysis • Data pushed to servers for analysis • Reconstruct status of users • Phone Classifiers: • Audio - Fourier transform to determine talking pattern • Understand behavior, mood, and health • Activity - Accelerometer data to determine if user is sitting, standing, walking, or running

8. CenceMe App Analysis Cont… • Backend Classifiers: • Conversation • combines multiple audio primitives to determine whether someone is in a conversation, which may contain pauses • Social Context • Neighborhood conditions - are there CenceMe friends around? • Social status - combination of classifiers • Partying  • Dancing  • Mobility Mode • GPS to determine if user is traveling in a vehicle or not • Am I Hot • Nerdy - often alone and spends time in locations such as libraries • Party animal - often at parties and often near others • Cultured - often visits theaters/museums • Healthy - often walking, jogging, cycling, or at the gym • Greeny - low environmental impact - not typically in a car and often walks, cycles, or runs

9. Smart phone sensing: • Today’s top-end smartphones come with 1.4-GHz quad-core processors and a growing set of inexpensive yet powerful embedded sensors. • They include: • An accelerometer, a digital compass, a gyroscope, a GPS, quad microphones, dual cameras, near-field communication, a barometer, and light, proximity, and temperature sensors. • They also have multiple radios for body, local, and wide area communications; 64 Gbytes of storage; and the touchscreen.

10. BeWell: A Mobile Health App • The BeWell app continuously tracks user behaviors along three key health dimensions without requiring any user input — the user simply downloads the app and uses the phone as usual. • Monitor and promote • Physical and emotional well-being • Persuasive feed back techniques • Tracks along three dimensions • Physical activity • Social interaction • Sleep duration • Weighted scorecard • Between 0 to 100 • Ex. 8 Hrs/day sleep is score 100

11. BeWell : Health App Cont… • Physical activity tracking • Walking, stationary or running • Daily metabolic equivalent of task value • Centers for Disease Control and Prevention guidelines • Sleep tracking • Over 24 hrs period • No phone interaction • Daily bed time activity • Recharging phone • Keeping at same place longer time • Guidelines by National Sleep Foundation

12. BeWell : Health App Cont… • Social Isolation tracking • Total speech time during day • Phone microphone usage • No recordings • Social apps presence on the phone and usage • Low score linking to depression • Work as stand-alone and interwork with cloud

13. BeWell : Health App Cont… • Persuasive feedback techniques • Animated aquatic ecosystem as wallpaper • Orange clown fish • Reflects user’s activity • Blue clown fish • Reflects user’s social interaction • Ocean’s ambient lighting • Reflects user’s sleep duration • Score history accessibility

14. Ambient Display:

16. BeWell+ • Store data on clouds • Share with friends on social networks • Compare with others • Targeted messages • Challenges • Low energy consumption • More accurate classification • Population Diversity Problem

17. BeWell : Big Sensor Data • No raw data to server • Upload features, inferences, scores and usability data • 20MB data traffic per day per user • Population Diversity Problem • Walking patterns changes for a child and an adult • Community Similarity Networks (CSN) • Personalized model per user • Measure similarity between people

18. WalkSafe : Pedestrian Safety App • Research in social science • Safety app for pedestrians crossing roads while on phone • Back camera to detect vehicles • Machine learning algos • Difficulty • Minimum battery utilization • During active calls only • Trained model • Decision tree

19. WalkSafe: Pedestrian Safety App • A mobile-phone user deep in conversation while crossing a street is generally more at risk than other pedestrians not engaged in such behavior. • WalkSafe uses the smartphone’s back camera to detect vehicles approaching the user, alerting the user of any potentially unsafe situations.

20. Algorithms used: • More specifically, WalkSafeuses machine-learning algorithms implemented on the phone to detect moving vehicles. It also exploits phone APIs to save energy by running the vehicle-detection algorithm only during active calls. • The WalkSafe app offers real-time detection of the front and back views of cars, noting when a car is approaching or moving away from the user, respectively . It alerts the user using sound and phone vibration.

21. Image recognition: • The core WalkSafe car-detection technology is based on image-recognition algorithms. Image recognition is a computationally intensive process that, if not carefully designed, can easily drain the smartphone’s computational resources and batteries.

22. Continued…… • To address this, WalkSafe bases its vehicle recognition process on a model that’s first trained offine and then uploaded to the phone and used for online vehicle recognition, which runs automatically whenever there’s an ongoing phone call. • WalkSafeactivates the smartphone’s camera and captures video of the surroundings. Each video frame is preprocessed to compensate for the phone tilt and illumination variations, and is then analyzed by the decision tree model built during the offline training phase. If the decision tree detects a car in the picture, it triggers an alert to warn the user of possible danger.

23. NeuralPhone: A Brain-to-Smartphone Interface • There’s a growing interest in new handsfree interfaces for smartphones based on voice and face recognition systems. • They developed the EyePhone, which lets the user select and activate applications with the blink of an eye.1 We then wondered if a thought could also drive a smartphone application—and it turns out it can.

24. NeuralPhone : A brain to smartphone interface • Hands free interfaces for smart phone • Thought driving smart phone app • EEG headsets • Brain controlled address book dialing • P300

25. Summary • Towards cognitive phones • Pushing intelligence towards phone • Robust classification techniques • Mobile sensors to improve life conditions

26. Queries?

27. If time permits: • A small video based on how accelerometer inside the cell phone are made and How a Smartphone Knows Up from Down. • http://www.youtube.com/watch?v=OjwLnAZoDFE