PeopleTones: a system for the detection and notification of buddy proximity on mobile phones

1. PeopleTones: a system for the detection and notification of buddy proximity on mobile phones Kevin A. Li Timothy Sohn Steven Huang William G. Griswold University of California, San Diego

2. ubiquitous ample computational power a few sensors a few actuators  proactive context-awareness

3. messaging location-based reminders You are driving by Safeway. Reminder: Buy steak.

4. unobtrusive notifications The slopes on Beaver Run have opened! Bzzzzt!

5. crappy sensors crappy actuators cheap sensors could lead to many false notifications cheap actuators could lead to misunderstood cues proactive notification + commodity hardware  flood of meaningless notifications

6. PeopleTones two proximity states: far and near (< 2 city blocks) when a buddy becomes near, play her sound or vibration cue runs on commodity hardware (Windows Smartphone)

7. “It was so cool to see who was home by the time I got home. I could tell if Jenny was home when I passed by University. So if we were going to go eat or something I could ask her. Oh she’s home, so let’s call her and see if she wants to eat.”

8. approach focus on application goals to avoid over-engineered, impractical solution proximity is easier than location acceptable if notifications missed or not understood

9. contributions privacy-friendly proximity detection algorithm technique for reducing sensor noise without sapping power method for generating a language of understandable vibrotactile cues exploratory study of buddy proximity cues

10. proximity detection

11. build on location? GPS doesn’t work indoors, in urban canyons tower-based systemsmust keep database current require wardriving how about tower overlap? NearMe[Krumm 2004] [LaMarca et al. 2005] iPhone

12. initial data collection used a GSM phone to record the cell towers it saw every 5 minutes 3 GSM phones, kept 1 stationary gather data at a variety of distances (0 - 1.2 miles)

13. initial measurements a and b are the sets of cell towers seen by each phone

14. initial measurements a and b are the sets of cell towers seen by each phone

15. evaluating proximity algorithm can our overlap-ratio algorithm detect proximity accurately enough to support nice-to-know information?

16. requirements cannot be annoying  when the system detects a buddy is near, they should really be near OK to not detect every time  if a buddy is nearby and stationary, we’ll have multiple chances

17. the dataset [Chen et. al., 2006] used the dataset collected by wardrivingseattle

18. coverage Suburb Downtown

19. metric: precision and recall precision 100% precision every report is valid recall 100% recall  every near incident is detected

20. how do we extract the relevant data? only care about when two phones are near or far from each other why not pull out each set of data by different distance thresholds? turns out mobile phone tower readings fluctuate over time (e.g., due to load balancing) we can crosscut the dataset to approximate precision and recall for different scenarios

21. nearby extract pairs of readings taken within 90s 569,264 pairs from Suburb 379,285 pairs from Downtown calculate precision and recall for different ratio threshold values

22. nearby precision suburb downtown 100% precision  every report was valid 100% recall  every near incident was detected

23. nearby precision suburb downtown 100% precision  every report was valid 100% recall  every near incident was detected

24. nearby precision suburb downtown 100% precision  every report was valid 100% recall  every near incident was detected

25. nearby recall 100% precision  every report was valid 100% recall  every near incident was detected

26. reducing sensor noise

27. initial approaches • wait for 2 consecutive-same-readings • Too many false positives • wait for 3 consecutive-same-readings • Too much delay

28. 2-bit-filter (“eventually 3 more”)

29. filter evaluation for noise filtering, interested in transitions from far to near and vice-versa extract seattlewardrive readings at 30s intervals try three algorithms on this subset, baseline is single report

30. filter evaluation

31. adaptive sampling rate sampling once every 20s kills the phone in less than a day increasing sampling rate to once per 90s helps but introduces a worst-case delay of 270s sample at 90s when in steady state, 20s when transitioning

32. buddy cues

33. mapping music to vibrations

34. problem we want to convert music to vibrations… …but the phone’s vibrator only turns on and off …at single frequency, single amplitude

35. pulse width modulation electric motors do this to save power in the case of vibrotactile motors this also decreases its rotational frequency perceived as different vibration levels can produce 10 levels of 20ms pulses

36. capturing the essence of music

37. overview of approach just using beat doesn’t always work mapping lyrics doesn’t work well basic idea: convey the current energy level of the music

38. remove noise isolate 6.6kHz to 17.6kHz components using 8th order Butterworth Filter use amplitude threshold, to keep only components greater than the average

39. take running sum take running sum of absolute value, generate 1 value every 20ms this keeps length consistent

40. exaggerate features compose output from previous step with power function: Axn, x is sample, A and n are constants, 10<=A<15, 1<=n<=2

41. examples(requires imagination) Beethoven’s 5th Symphony matching vibration sequence Michael Jackson – Smooth Criminal matching vibration sequence

42. so far…

43. would the techniques we used for proximity detection, sensor noise filtering and vibrotactile cues work in the wild? can peripheral cues be deployed on mobile phones despite poor sensors and actuators? (what experiences can such a system enable?) field study

44. PeopleTones two proximity states, far and near (< 2 blocks) when a buddy is near, play their song if phone is in vibrate mode, play a matching vibrotactile sequence

45. participants 3 groups of friends, 2 weeks

46. could you tell who it was?

47. could you tell who it was?

48. user response to the cue

49. designing peripheral cues for the wild higher comprehension rate when users select their own cues obtrusiveness of music cues was not a concern mapping music to vibration was most successful for people who knew the songs well semantic association is key to learnability

50. userexperience