Ferret: RFID Localization for Pervasive Multimedia

1. Ferret: RFID Localization for Pervasive Multimedia Xiaotao Liu, Mark Corner, Prashant Shenoy University of Massachusetts, Amherst

2. Scenario: I’ve Lost my Keys • People frequently misplace common items • books, keys, tools, clothing, etc. • difficult due to the sheer scale: we interact with >1000s of items • Need a system to find objects quickly and efficiently • then tell the user where the object is

3. Problems • Tracking objects can be broken into sub-problems • Locate: find position, perhaps not exact, but a general idea • Store: keep object locations in a convenient place • Update: when objects move, need to change store • Display: Present locations to user in a helpful way

4. Solution: Ferret • Provides a real-time augmented reality service • locates, stores, updates, and displays object locations • intended for nomadic objects not mobile ones • Leverage passive RFID, multimedia, and location systems • passive RFID: inexpensive, scalable, maintenance-free • multimedia systems: provide convenient display and storage • location systems: bootstrap process of finding locations • Goal is to pack all functions into a hand held device • including RFID detection, storage, and display • a combination of video camera and RFID reader

5. Outline • Motivation and Applications • Overview of Use • Design of Ferret • Sensor model • Offline location algorithm • Online location algorithm • Display • In paper: Storage, Update for nomadic objects • Prototype implementation • Experiments • Speed and accuracy • Robustness to different movement patterns • Related Work • Conclusions

6. Overview of Operation • User selects some object(s) that she is looking for • She wanders around a room, or building, holding Ferret system • During this process, the reader scans for nearby RFID tags • Ferret detects the RFID tag of interest, localizes tag • It then displays an outline of where the object is on the screen • willing to settle for a probable region of where the object is • depend on human skill to find the exact location • refine region as system runs • present improved results in real-time

7. RFID Localization 1. energy 3. id 2. use RF energy to charge up • Passive RFID tags are not self-locating • Instead we depend on the handheld to locate tags • Passive RFID tags have significant error rates • false negatives are frequent • false positives due to reflections • Locate using probabilistic model • inspired by [Hähnel et. al] RFID reader

8. Bayesian Probability Model • Goal:p(x|D1:n): Probability of tag at x given readings • Initially, without readings, p(x|D0) is uniformly distributed • Assume we have p(x|D1:n) • Positive reading • p(Dn+1=True|x) • Bayes’ rule p(x|D1:n+1) = α p(x|D1:n) p(Dn+1|x) • α– normalization factor • Similarly, for negative readings • p(Dn+1=False|x) = 1 - p(Dn+1=True|x)

9. Tag Detection Probability Manually measure probability of detecting tag (positive reading) p(D =True|x) x – tag’s position

10. Ferret Localization Algorithm (+ reading) • Multiple readings come from user mobility, previous, or shared readings

11. Ferret Localization Algorithm (- reading) Repeated intersection of positive and negative readings

12. Offline Algorithm Complexity • We refer to the previous algorithm as the “offline” algorithm • Each + or - reading Ferret performs O(n^3) operations • n is the number of sample points • it must rotate, translate the RFID sensor model • multiply each sample point against every other sample point • must do this for each object! • Computational requirements at least 0.7s on a laptop • reader is producing at least 4 readings per second • some readings include multiple objects • Algorithm most useful for back-annotating video

13. Online Algorithm • To address real-time concerns use an “online” algorithm • instead of intersecting all interior points, just find convex intersection • only uses positive readings, not negative ones (keeps shape convex!) • Complexity reduced to O(n^2) or 6ms per reading

14. Display • Each RFID location is a 3-D shape • To display we simply project this 3-D shape onto a 2-D screen

15. Ferret Prototype • ThingMagic Mercury4 RFID reader • 30dBm (1 Watt), monostatic circular antenna • Alien Technology “M” RFID Tag • EPC Class 1, 915 MHz • Sony Motion Eye web-camera • 320x240 at 12fps • Cricket Ultrasound 3-D locationing system • global location not necessary, but need relative locations at least • Sparton SP3003 Digital Compass • Pan, tilt, and roll • Software • translate between coordinate systems, rotate, and display

16. Ferret Prototype Built-in Camera Cricket locationing sensor Compass ThingMagic RFID reader RFID antenna

17. Evaluation • Evaluation metrics: • Size of location region for many objects • Speed of localization for a particular object • Robustness of localization to mobility patterns • Evaluation setup for many objects: • Place 30+ objects with passive tags around the room • Move Ferret system around the room by human for 20 minutes • CDF of localization over 30 objects • Evaluation setup for single object: • Place single object in room with passive tag • Move Ferret system in and out of view randomly and using a specific pattern • Size of localization after some amount of time

18. Online Vs Offline (CDF-30 Objects) Offline algorithm outperforms online, but most objects localized to 0.2 m^3

19. Refinement: Relative Volume (1 Object) Volume size drops down 100 times to 0.02m3 in 2 mins When starting with previous readings, localization is faster

20. Refinement: Relative Projection Area Final projection area decreases 33 times in 2 mins to a 54 pixel diameter circle

21. Different Movement Patterns • Circular motion pattern performs the worst: no diversity in views • Offline algorithm’s advantage comes from negative readings • so head-on and circular perform similarly

22. Related Work • Grown out of our work on Sensor Enhanced Video Annotation • SEVA ACM Multimedia 2005 (Best Paper Award) • Used active sensors for location • RFID Localization inspired by techniques from [Hähnel et. al] • 2-D sensor model, application of Bayes rule positive readings • we add 3-D model, negative readings, and online technique • focuses on SLAM/localizing reader, we focus on reverse • LANDMARC and SpotON RFID locationing • active RFID and signal strength

23. Conclusions • Ferret: a scalable, RFID-based, augmented reality system • localize objects augmented with passive RFID tags • display probable location regions to a user in real-time • Uses two algorithms: online and offline • both are accurate and efficient (localizes objects to 0.2m^3 in minutes) • robust to a variety of user mobility patterns • Ferret lays the ground work for other augmented reality applications

24. Ferret: RFID Localization for Pervasive Multimedia Xiaotao Liu, Mark Corner, Prashant Shenoy University of Massachusetts, Amherst

26. Location Storage • Locations (3-Dimensional probability maps) • Storage on reader • simple to implement, but must acquire readings as it goes • Database • any Ferret readers can take advantage of prior knowledge • also permits offline searching, but privacy/authorization concerns • Storage on writable tags • tags self-locating and provide locations to non-Ferret systems

27. What if objects move? • Nomadic objects may have moved since previous readings • when online algorithm detects empty intersection, reset • offline algorithm more complex, uses a probability threshold

28. Ferret Software Architecture Ferret System Visualization Module (modified from FFmpeg) Intercept original display function Display projection boundary Use optics model Compute projection of location estimates Fuse video, tag’s location together Deal with large amount of data, Optimized for real-time usage Bayesian Locationing Module Video Recording via TCP, Use SQL-like language RFID Module (operate RFID reader) Device Drivers for Cricket and Compass

29. [Hähnel et. al] • “To each of the randomly chosen potential positions we • assign a numerical value storing the posterior probability • p(x | z1:t) that this position corresponds to the true pose of • the tag. Whenever the robot detects a tag, the posterior is • updated according to Equation (1) and using the sensor model • described in the previous section.” • In this paper we analyze whether recent Radio Frequency Identification (RFID) technology can be used to improve the localization of mobile robots and persons in their environment.