Did You See Bob? Human Localization using Mobile Phones

Did You See Bob?Human Localization using Mobile Phones Ionut Constandache Duke University Presented by: Di Zhou Slides modified from Nichole Stockman

The Issue • Finding someone in a public place can be difficult • Might not know their location • Might be unfamiliar with the area • Maps and floor plans are not always available

Hypothetical Scenario • Mobicom conference – in a big hotel • Alice wants to meet her colleague Bob… • Can walk around • Can ask “Did you see Bob?” • But these can take a long time and he may be moving • Can call him • But he may be in a meeting already • Best to have someone escort you… How?

Current Solutions • GPS • Drains the battery (and not very precise) • Outdoor scheme • WiFi/GSM schemes • Not enough accuracy than GPS • Require special infrastructure, RF transmitter or war-driving

Escort • Guide a user to the vicinity of a desired person • mobile phone sensors • Opportunistic encounters • Client/server model • Does not require: - Physical coordination • GPS/Wi-Fi • War-driving • Maps or floor plans • Can be easily installed and provides localization service when GPS/WiFi not available

Outline • Motivation Recap: • We need efficient and accurate software for people localization: Escort • Outline • Basic Design of Escort • Challenges • Solutions • Experiment Specifics • Results • Future Work

Basic Design • Escort consists of two parts: • Navigation • - Get directions to the person • Visual Identification • - End to end human localization

Part 1 - Navigation Clients report “Movement trail” periodically A(t),B(t),C(t) <displacement, direction, time> Use accelerometer and compass measurements Displacement = # of steps multiply step size Direction read from compass Also report “Encounters” T_AC, T_BC <movement trace, encounter time> Definition ->Audio signals in inaudible frequencies within 5m B(t) B(t) T_BC C(t) T_AC • Server builds a virtual graph • Global view of users’ positions and paths C(t) A(t)

Challenges • Accelerometers & Compasses are noisy • Measured path error over time • No global reference frame • How to correct errors? • Even if correct user position, difficult to correct entire trail - Yet is necessary! (routing) 4. Trail graph grows over time

Solutions - 1 • Accelerometers & Compasses are noisy • Use: (step size) * (step count) Signature from up and down bounce of human body while walking Displacement error using double integration

Solutions - 1 Accelerometers & Compasses are noisy Use: (step size) * (step count) • Take into account varying step size (Vary step size with error factor drawn from Gaussian distribution centered on 0 and standard deviation 0.15m) Error with step count method (avg 4%)

Solutions - 2 • No global reference frame • Use a fixed beacon transmitter • Beacon Transmitter • Location is origin of a virtual coordinate system • Location diffusion ( single point updates ) • Drift cancellation ( path correction )  sol’n 3

Solutions - 3 • Drift Cancellation • Amortize the correction vector over time • Assume that user’s projected path deviates from the true path linearly over time User encounters beacon at t_r1,and another beacon or recentlyupdated user at t_r2. Solid Line: actual path Dotted Line: user-computed trail Dashed Line: corrected user trail

Solutions 4 • Graph computation done by Server • Pruning heuristic -> eliminate duplicates • Floyd-Warshall algorithm -> shortest paths Graph – 4 users, 10 min After pruning After Floyd-Warshall alg

Basic Design • Escort consists of two parts: • Navigation • Visual Identification • Help you identify the person if she is someone you have not met before (e.g. first-time meeting with a professional colleague at a conference)

Basic Design – Visual Identification • Perhaps Alice has not met Bob before… • Picture of face not enough • In a trusted environment like Mobicom, can • Take many photos of user to generate a “fingerprint” • Alice takes photos of surroundings. Image processing is done to identify Bob from the photos • Totally works in theory! • Currently only implementedoffline and requires user input…

Experiment Specifics • Parking Lot Experiment • 4 users, 13 min, phones in hand, 40 routing exp’mts • Used parking spot lines & markers for ground truth (GPS not fine-grained enough) • Indoor Experiment • 2 users, 6 min, 10 routing exp’mts

Results – Parking Lot Instantaneous location error over time

Results – Parking Lot • Instantaneous location error < 10m: • Intertial ------------------------ ~6% of cases • Beacon & Encounter ------- ~ 68% of cases • Drift Cancellation ----------- ~ 84% of cases • Final destination distance error 8.2m on average

Results – Indoor • Instantaneous location error: • Better overall (due to indoor structure and user guesswork) • Intertial --------------------- (N/A because no GPS indoors) • Beacon & Encounter ------ ~ 85% of cases • Drift Cancellation ---------- ~ 90% of cases • Final destination distance error was 7m on average

Results – Visual Identification • Is this a good result? • 80% accurate with 8 people in surroundings

Future Work • “Escort is not designed for energy efficiency” • Turn off sensors (except accelerometer) when user not moving • Less audio signaling when many users or beacons nearby • Less frequent uploading of data to server • Routing through physical obstacles • People are smart • Visual representation of path may help • Long routing paths • Include true-direction arrow to Bob’s location

Future Work • Routing instructions under low location accuracy • If update too far in past, prompt user to approach beacon • Update path as more info becomes available • Phone placement • Need advancement of phone gyroscopes to infer orientation • Behavior under heavy user load • Scalability needs to be explored but should improve performance (more beacons/users = more updated info, etc)

Thank You Any Questions?

Did You See Bob? Human Localization using Mobile Phones