SurroundSense: Mobile Phone Localization via Ambience Fingerprinting

1. SurroundSense: Mobile Phone Localizationvia Ambience Fingerprinting Publish: Mobicomp 2009 Presenter: Vincent Data: 2009 / 10 / 27

2. Outline • Introduction • Relative work • Hypothesis • Architecture • Fingerprint generation / filtering / matching • Evaluation • Conclusion

3. Introduction • Notion of location • Physical coordinates(latitude/longitude) • Logical labels(like Starbucks, Mcdonalds) • User want to proposed a logical localization Application of logical localization

4. Introduction (cont) • Application developer are assuming that physical coordinate can be reversed geo-code to logical location.

5. Relative work • Active RF • Install special hardware • Passive RF • GPS, GSM or WIFI based • Behavior Sensing • Imaging matching 1. Lack accuracy 2. Need pre-installed infrastructure

6. Their hypothesis • Combine effect of ambient sound, light, color, user motion • Sound (microphone) • Coffee machine (cafe shop) versus forks and spoons clinking (restaurant) • Light / Color (camera) • Shops may have thematic light, colors and different floors. • Human movement (accelerometer) • Restaurants: short queuing -> long duration of sitting • Grocery store: walking up and down aisles • Place may not be unique based on any one attribute • The combination is likely to exhibit diversity. • Increasing number of sensors on mobile phones presents new opportunities for logical localization. Starbucks McDonald’s

7. SurroundSense architecture Candidate list 1.Xxx 2.Yyy 3.zzz 1.Xxx 2.Yyy 1.Xxx 2.Yyy 1.Xxx 2.Yyy 1.Xxx

8. Fingerprint Generation- sound • Convert signals to time domain • Similarity of fingerprints • Euclidean metric in 100 dimension Normalized occurrence count amplitude value 50 0 -50 time time Normalized amplitude value Normalized occurrence count Normalized amplitude value

9. Fingerprint Generation– sound (cont) • Unreliable to be a matching scheme • Sound from the same place can vary over time. • Only use as a filter If distance > threshold τthen discard from the candidate list • Select threshold τ • Collect the acoustic fingerprint from each location at different time. Choose a 95 percentile self-dissimilarity among the locations.

10. Fingerprint Generation– Motion • Use support vector machine(SVM) as classifier • Movement is prone to fluctuation • Some users may browse for a long time in a clothing store, while others may purchase clothes in haste. • Use as a filter Raw data SVM 1 moving -1 stationary

11. Fingerprint Generation– Motion (cont) • Differentiate place from 3 categories • Sitting place • Slow-browsing place • Speed-walking place • Compute motion fingerprint Ratio = tmoving / tstatic Bucket 1: 0.0 <= Ratio <= 0.2 Sitting (ex: cafe) Bucket 2: 0.2 <= Ratio <= 2.0 Browsing (ex: clothing) Bucket 3: 2.0 <= Ratio <= ∞ Walking (ex: grocery) • Multi-commodity shop problem • Customers vary from long DVD browser to quick grocery shopper. • Assign both buckets to be the training data moving stationary Sitting Slow-browsing Speed-walking

12. Fingerprint Generation– Color / Light • Where to capture the picture? • random picture of surrounding • floor • Advantages of taking floor pictures • Privacy concern • Less noisy • Rich diversity in floor color • Often point downward the camera while using the phone too many noise

13. Fingerprint Generation– Color / Light (cont) • How to extract colors and light intensity? • RGB to HSL(Hue-Saturation-Lightness) space • Find color cluster and its size k=2 sk: the sum of distance from all pixels to their centroid. t: convergence threshold k-mean clustering k++ sk-sk-1 < t no yes < c1, c2 …, ck >

14. Fingerprint Generation– Color / Light (cont) • Similarity of fingerprints • Assume C1 = {c11, c12, …, c1n}; C2 = {c21, c22, …, c2m} • Fingerprint matching • The candidate list with maximum similarity is declared to the matching fingerprint Total size in C1 or C2 distance of centroid

15. Fingerprint Generation– Wi-Fi • Wi-Fi fingerprint • Record MAC address from APs every 5 second <{AP1_MAC_Addr, AP1_fraction_time}, {AP2_MAC_Addr, AP2_fraction_time}, {AP3_MAC_Addr, AP3_fraction_time}>

16. Fingerprint Generation– Wi-Fi (cont) • Similarity of fingerprints • Use as filter/matching module • in the absence of light/color, we use it as matching module. • Accuracy depend on location of shops. fraction of time M: union of MAC address of f1 and f2

17. Evaluation • SurroundSense(SS) test environment • we war-sensed 51 shops organized in 10 clusters • 4 students visited the clusters at different time, and evaluate SS by cross validating the fingerprints. • Offer 4 mode of sensor combination • Wi-Fi only • Sound, Accelerometer, Light and color • Sound, Accelerometer, Wi-Fi • SurroundSense

18. Evaluation – Per-Cluster Accuracy

19. Evaluation – Per-Shop Accuracy • To understand the localization accuracy on a per-shop basis 30% shops 47% shops

20. Evaluation – Per-User Accuracy • Simulate 100 virtual user, each assign 4~8 stores from cluster 1~9

21. Evaluation – Per-Sensor Accuracy • Hand-picked 6 samples to exhibit the merits and demerits of each sensor filter out false negative

22. Conclusion • Author presented SurroundSense which fingerprint a location based on user’s ambient sound, light, color, RF and user movement • In conjunction with GSM based macro-localization, SurroundSense can perform micro-localization. • SurroundSense is a early step of indoor localization. Further research in fingerprinting techniques, sophisticated classification, and better energy management schemes could make SurroundSense a viable solution of the future.