Pervasive Location-aware Computing Applications and Issues

1. Pervasive Location-aware ComputingApplications and Issues Professor Lionel M. Ni Dept. of Computer Science Hong Kong Univ. of Science & Technology ni@cs.ust.hk

2. Outline • Motivation and Background • Pervasive computing • Location-aware computing • Applications • Technical challenges • Conclusions

3. Evolution of Computing Devices Supercomputer $10,000,000 Mainframe $1,000,000 $100,000 Server $10,000 Workstation $1,000 PC $100 PDA $10 Watch $1 Smart Card $0.1 RF Chips $0.01 ???

4. Technologies Drive New Thing • Natural technology trends • Computation is becoming essentially free • Communication is becoming ubiquitous • Smart devices • Huge numbers of computing devices in the world • What are we doing with them? • Modes of operation • Programs controlling other programs in our environment • Human-in-the-loop: computing should be only as visible as I desire; no more, no less...

5. Next Wave Computing • Artificial intelligence? • Anywhere and anytime services • All are interconnected • Benefit from computers without any knowledge of computers • Making computers invisible • Making networks invisible • Computers can act intelligently upon and within the environments

6. Trend: Pervasive Computing • Mutual dependency and interaction between computers and environments • Computers • Obtain the information from the environment • Utilize the information to dynamically build models of computing • Environments • Detect other computing devices entering it • Challenge: how to teach a computer about its environment?

7. Computing Input Output Processing

8. Networked Computing Processing + communication Processing + communication Processing + communication Processing + communication Network

9. Pervasive Computing Sensors Processing + communication Processing + communication Processing + communication Location information Processing + communication Resource information Network Actuators Environmental Context

10. Pervasive Computing • Shorthand for the strongly emerging trend toward: • Numerous, casually accessible, often invisible computing devices • Frequently mobile or imbedded in the environment • Connected to an increasingly ubiquitous network structure • The aim is for easier computing, more available everywhere it's needed

11. Ubiquitous Computing • Integration of large-scale mobility with the pervasive computing functionality • Any computing device, while moving with us, can build incrementally dynamic models of its various environments and configure its services accordingly • Proactively build up services in new environments • Remember past environments when we re-enter

12. Fine-Grained Networking • The next explosion in growth of the Internet will come from connecting to billions of cheap, low-power sensors, effectors, and “smart things” • Cybernetic servants in the Internet will interpret and respond to the environment, such as location, gestures, and movement • Connectivity will be wireless • Transceivers must be comparably small, cheap, and low-power

13. Makes use of different kinds of information User Identity Location Social “In a meeting” Physical Temperature Light and noise levels Computational Network bandwidth CPU speed Display size Temporal Geographic Environmental Context

14. Environmental Context • Environmental features are not created explicitly for input to the system • Help for more intelligent behavior • Hard to give detailed instructions in mobile environment • Interfaces must be more intuitive • Mobile environment is much more variable than desktop

15. Examples • Take your current environment into account in making decisions • Turn off cell phone when you enter the lecture hall. • When you ask where to go for a meal, notes that it is morning and you are in Hong Kong before making a recommendation. • Know who wrote on the whiteboard so a copy of the ink can be emailed to the author. • Play music you like when you enter an empty elevator. • Notify your doctor when your heart rate goes too high.

16. Example: More on Cell Phone • How should a cell phone respond to an incoming call • Noisy environment in street  ring loudly • Quiet environment in office  ring gently • Inside a cinema  no ring, but vibrate • Attending classes  switch to voicemail • Manual switching is possible but undesirable

17. More Examples Meeting Event Enter Area Event • Proactive is important in pervasive computing • Need some event observation/notification mechanism Ask forautograph ... You wanted to buy shoes ... Shoes

18. Other Context-aware Behavior • Present information and services proactively • Your wearable sunglass shows you the names and the research interests of the people around • Execute service automatically • When a sensor you wear senses that you feel cold, it turns on the heater automatically • Tag “context” for later retrieval • When attending a lecture, your computer detects that you feel sleepy and starts recording. Later It can show you parts of the lecture that you have missed

19. Why Indoor Location Sensing?

20. Motivation location-based action nearby local printer, doctor nearby remote phone directions/maps location-based information real person’s location history/sales/events virtual walkthrough story of city augmented touring machine Location-aware Computing

21. Location-aware Computing • Location-aware services allow to offer value-added service to the user, depending on their current geographic position and will be a key feature of many future mobile applications • Many scenarios in pervasive computing • Navigation • Resource discovery • Embedded applications, sensor systems • Monitoring and control applications

22. Location Sensing Techniques • GPS: used in outdoor • Indoor location sensing criteria • Accuracy • Granularity • Latency • Direction of movement • Power consumption • Wiring constraints • Physical packaging • Regulation constraints • Cost

23. GPS • The most popular option of obtaining location information today. • 24 satellites controlled by US • Need to be exposed to 4 satellites • GPS lets suitably equipped receivers determine their position within 3-5 meters. • It is generally not workable indoors.

24. GPS Trimble GeoXM The Casio Pathfinder GPS watch

25. Mobile Commerce

26. Mobile Commerce Applications • Retails • Shopping malls • Health care system • Manufacturing • Inventory tracking • Mobile IP phone • Your imagination ….

27. Retail Stores & Shopping Malls • Customer: enjoy the convenience and fun • Easily find the direction to the target items/stores • Find and locate your friends if they nearby (can even setup to get notified if they are close-by) • Retailers: better management and customer relationship • Easily get information to the customer, like special deals/coupons for items/stores nearby • More efficient management, e.g. just-in-time customer service • Establish better relationship with customer through the provision of better services and experience

28. M-commerce Illustration Click for details info or text chat Initiate a voice chat with friends or for customer service person • Direction to target places, or pinpoint location of friends • Help manage customer service employees

29. 5.Stop eating 6.Back to desk 1.walk Desk 2. Tracking movement Refrigerator Desk 3. Notify where you are (Location sensing) 7. Time sensing 10. Walk around 8. immobility sensing An Obesity Care Case 4. Notify your eating schedule 9. Notify to move 13. Distance& Time sensing 18. Refer Healthy food 11. Proximity sensing 14. Notify to stop 16. Notify “ok to eat” 17. Go to kitchen 12. Walk away from sofa 15. Go back to desk Refrigerator Desk Sofa 19. Eat matched food

30. Other Applications • Health care system • Tracking patients with wheelchairs • Dietary control • Manufacturing • Auto industry • Inventory tracking

31. SWIM Project Infrastructure • Sensing Network • Locate a mobile user or object within certain accuracy and granularity • WIreless Network • Allow the communication between mobile devices and the resource-rich Internet/Intranet • Mobile Ad-hoc Network and routing: How mobile devices communicate through close-by in-range mobile device when it’s out of the range from the Wireless Access Point (WAP)

32. SWIM Prototype

33. Mobile Ad-hoc Network • Ad-hoc communication & routing • Allow mobile devices to communicate with each other in the absence of WAP • Enable mobile devices to communicate with the wired-network/far-away devices through peers forwarding • VoIP between handheld devices • Allow hands-free communication on handheld devices

34. Who Are the Players? • The design of good location-aware computing systems cuts across many areas of CS/EE • E.g., sensors, signal processing, networking, mobility, data management, graphics/visualization, planning, HCI, … • Interdisciplinary Research • Explore new business models • Establish new customer behavior analysis • Develop new applications for m-commerce

35. Fundamental Challenges • Collecting sensor data • Scalability, heterogeneity • Raw format • Measurement error, noisy environment • Processing sensor data into “context” • Adjust format, correct for error • Aggregate for higher-level context • Disseminating to interested applications • Scalability, heterogeneity • Securing access to secret and private data

36. More Challenges • In-door location sensing techniques • Resolution and range proportionate to size and mobility of objects • Most applications want relative location, not latitude and longitude • Commodity components: Berkeley MOTE? • Multi-hop ad hoc networking • Infrastructure mode and ad-hoc mode • IP address assignment, Name and IP mapping • Service discovery • Low-power multi-hop ad hoc routing

37. More Challenges: Context • We are constantly receiving information • Problems • Information only received once or twice • Not received when and where we need it • Large amount of distributed data • Data movement, storage, retrieval, etc. • Possible solutions • Place information into the context in which it will be most useful • Devices accept and/or deliver information • Distributed query processing

38. Software Infrastructure • Need a software infrastructure to put all those together • Software models • Standard middleware support • Service discovery • Security and privacy issues • More intuitive HCI

39. Current Pervasive Machines Still Mostly “Modern” • More, faster, refined: • RAM • Processor power • Network bandwidth • Disk storage • Windows • Icons • Menus • Pointers • Wireless networks • Portable • Pen Input

40. Passive Nominal Individual Single machine Local data Fixed External Perceptual/Reactive Personal Collaborative Distributed Global data Portable/wearable Implanted (medical) User Interface TrendsBeyond Twentieth Century “Modern”

41. Into the Twenty-first Century... • Wireless networking • Extreme portability • Longer battery life • Perceptual interfaces • Sense: sounds, video images • Recognize: words, people, gestures, gaze • Respond: process commands, access information

42. e911 Scenario • Assumptions • John, a physician, volunteers to help patients and carries a PDA • Bill, over 70, has heart disease and carries a PDA with his disease history and other information • PDA: cell phone, wireless LAN/WAN, Bluetooth • Both John and Bill are in a shopping mall and Bill has heart attack. • Bill pushes one button on his PDA • PDA calls 911 and e911

43. Challenges • How is people’s mobility supported? • How are Bill’s location and John’s location known and integrated with the medical service (mobile service)? • How can the system provide John’s expertise while keeping his privacy? • How is Bill’s medical information guaranteed to be secure and not revealed to the shopping mall? • How are practical jokes protected? • How are service requests made non-repudiate? • How are quacks excluded? • Etc. • One solution: SPLENDER (PerCom 2003)

44. Conclusions • Most of the exciting stuff will happen in the next few years! • Many technical challenges • Commercial and economical impacts • New business models • Security and privacy issues • Pricing and charging models • Social impacts

45. Questions?