LOCATION-AWARE RESOURCE MANAGEMENT IN SMART HOME ENVIRONMENTS

1. LOCATION-AWARE RESOURCE MANAGEMENT IN SMART HOME ENVIRONMENTS Sajal K. Das, Director Center for Research in Wireless Mobility & Networking (CReWMaN) Department of Computer Science and Engineering (CSE@UTA) The University of Texas at Arlington, USA E-mail: das@cse.uta.edu URL: http://crewman.uta.edu [Funded by US National Science Foundation]

2. Saturated with computing and communication capabilities to make • intelligent decisions in an automated, context-awaremanner •  pervasiveorubiquitous computingvision. • Technology transparently weaved into the fabric of our daily lives • technology that disappears. (Weiser 1991) • Portable devices around users networked with body LANs, PANs • (personal area networks) and wireless sensors for reliable commun. • Environment that takes care of itselfor users intelligent • assistantsprovide proactive interaction with information Web. • Examples: Smart home, office, mall, hotel, hospital, park, airport What is aSmart Environment ?

4. Smart/Pervasive Healthcare • Consider a heart attack or an accident victim • Desired actions • Coordinate with the ambulance, hospital, personal physician, relatives and friends, insurance, etc. • Control the traffic for smooth ambulance pass through • Prepare the ER (Emergency Room) and the ER personnel • Provide vital medical records to physician • Allow the physician to be involved remotely … • On a Timely, Automated, Transparent basis • PICO (Pervasive Information Community Organization) http://www.cse.uta.edu/pico@cse M. Kumar, S. K. Das, et al., “PICO: A Middleware Platform for Pervasive Computing,” IEEE Pervasive Computing, Vol. 2, No. 3, July-Sept 2003.

5. Ambulance Physician Hospital Cardiac Surgeon Nurse Pervasive Healthcare Heart attack victim Victim- Ambulance Community • Spouse • Police • Traffic control • Insurance Co. Larger community to save patient

6. PICO Framework • Creates mission-oriented, dynamic computing communitiesof software agents that perform tasks on behalf of the users and devices autonomously over existing heterogeneous network infrastructures, including the Internet. • Provides transparent, automated services: what you want, when you want, where you want, and how you want. • Proposes community computingconcept to provide continual, dynamic, automated and transparent services to users.

7. Bluetooth 802.11b Cellular … Access point Internet Gateway Access point Gateway Camileuns PICO Building Blocks • Camileuns (Physical devices) (Context-aware, mobile, intelligent, learned, ubiquitous nodes) • Computer-enabled devices: small wearable to supercomputers • Sensors, actuators, network elements • Communication protocols

8. Community Delegents PICO Building Blocks • Delegents (IntelligentDelegates) • Intelligent SW agents and middleware • Location/context-aware, goal-driven services • Dynamic community of collaborating delegents • Proxy-capable: exist on the networking infrastructure • Resource discovery and migration strategies • QoS (quality of service) management

9. Surveillance Police Community Automobile Community Traffic Monitor Information Kiosk Visitor’s Delegent Camileuns + Delegents = Chameleons Streetlamp

10. Community Delegents PICO Middleware Services Bluetooth 802.11b Cellular … Access point/ Gateway Access point/ Gateway Camileuns PICO Architecture

11. Smart Homes: Objectives • Use smart and pro-active technology • Cognizantof inhabitant’s daily life and contexts • Absence of inhabitant’s explicit awareness • Learningand predictionas key components • Pervasivecommunications and computing capability • Optimize overall cost of managing homes • Minimize energy (utility) consumption • Optimize operation of automated devices • Maximize security • Provide inhabitants with sufficient comfort / productivity • Reduction of inhabitant’s explicit activities • Savings of inhabitant’s time “The profound technologies are those which disappear” (Weiser, 1991)

12. Smart Home Prototypes /Projects • Aware Home(GA-Tech) –Determination of Indoor location and activities • Intelligent Home(Univ. Mass.) –Multi-agent systems technology for designing an intelligent home • Neural Network House(Univ. Colorado, Boulder) –Adaptive control of home environment (heating, lighting, ventilation) • House_n(MIT) –Building trans-generational, interactive, sustainable and adaptive environment to satisfy the needs of people of all age • Easy Living (Microsoft Research) –Computer vision for person-tracking and visual user interaction • Internet Home(CISCO) –Effects of Internet revolution in homes • Connected Family(Verizon) –Smart technologies for home-networking

13. MAVHome at CSE@UTA • MavHome: Managing an Adaptive VersatileHome • Unique project – focuses on the entire home • Creates an intelligent homethat acts as arational agent • Perceives the state of the home through sensors and acts on theenvironment through effectors (device controllers). • Optimizes goal functions: Maximize inhabitants’ comfort and productivity, Minimize house operation cost, Maximize security. • Able to reason about and adapt to its inhabitants to accurately route messages and multimedia information. http://ranger.uta.edu/smarthome S. K. Das, et al., “The Role of Prediction Algorithms in the MavHome Smart Home Architecture”, IEEE Wireless Communications, Vol. 9, No. 6, pp. 77– 84, Dec. 2002.

14. Automated blinds Door/lock controllers, Surveillance system Face recognition, automated door entry Climate control Intelligent appliances Remote site monitoring and control Assistance for disabilities Robot vacuum cleaner Lighting control Robot lawnmower Intelligent Entertainment Smart sprinklers MavHome Vision

15. MavHome: Bob Scenario • 6:45 am: MavHome turns up heat to achieve optimal temperature for waking (learned) • 7:00 am: Alarm rings, lights on in bed-room, coffee maker in the kitchen (prediction) • Bob steps into bathroom, turns on light: MavHome records this interaction (learning), displays morning news on bathroom video screen, and turns on shower (proactive) • While Bob shaves, MavHome senses he is 2 lbs overweight, adjusts his menu (reasoning and decision making) • When Bob finishes grooming, bathroom light turns off, kitchen light and menu/schedule display turns on, news program moves to the kitchen screen (follow-me multimedia communication) • At breakfast, Bob notices the floor is dirty, requests janitor robot to clean house (reinforcement learning) • Bob leaves for office, MavHome secures the house and operates lawn sprinklers despite knowing 70% predicted chance of rain (over rule) • In the afternoon, MavHome places grocery order (automation) • When Bob returns, grocery order has arrived and hot tub is ready (just-in-time).

16. MAVHome: Multi-Disciplinary Research Project • Seamless collection and aggregation (fusion) of sensory data • Active databases and monitoring • Profiling, learning, data mining, automated decision making • Learning and Prediction of inhabitant’s location and activity • Wireless, mobile, and sensor networking • Pervasive computing and communications • Location- and context-aware middleware services • Cooperating agents – MavHome agent design • Multimedia communication for entertainment and security • Robot assistance • Web monitoring and control

17. MAVHome Agent Architecture • Physical • Sensors • Actuators • Networks • Agents • Communication • Routing • Multimedia • download • Information • Data Mining • Action Prediction • Mobility Prediction • Active database • Decision • MDP/policy • Reinforcement • learning • Multiagent • systems/ • communication House Agent Rooms/ robots Agent Agent … Agent Network / mobile network Appliances/ robots Agent Agent … Agent Network / mobile network Transducers/actuators User Interface External resources • Hierarchy of rational agents to meet inhabitant’s needs and optimize house goals • Four cooperating layers in an agent • Decision Layer Select actions for the agent • Information Layer Gathers, stores, generates knowledge for decision making • Communication Layer Information routing between agents and users/external sources • Physical layer Basic hardware in house

18. Indoor Location Management • Location Awareness • Location(current and future) is the most important context in any smart computing paradigm • Why Location Tracking ? • Intelligent triggering of active databases • Efficientoperation of automated devices • Guarantees accurate time-frame of service delivery • Supports aggressive teleporting and location-aware multimedia services -- seamless follow of media along inhabitant’s route • Efficient resource usage by devices -- Energy consumption only along predicted locations and routes that the inhabitant is most likely to follow

19. Location Representation • Location Information • Geometric – Location information in explicit co-ordinates • Symbolic - Topology-relative location representation • Blessings of Symbolic Representation • Universal applicability in location tracking • Easy processing and storage • Development of a predictive framework

20. Indoor Location Tracking Systems

21. Inhabitant’s Movement Profile • Efficient Representation of Mobility Profile • In-building movement sampled as collection of sensory information • Symbolic domain helps in efficient representation of sensor-ids • Role of Text Compression • Lempel Ziv type of text compression aids in efficient learning of inhabitant’s mobility profiles (movement patterns) • Captures and processes sampled message in chunks and report in encoded (compressed) form • Idea:Delay the update if current string-segment is already in history (profile) – essentially a prefix matching technique using variable-to-fixed length encoding in a dictionary – minimizes entropy • Probability computation: Prediction by partial match (PPM) style blending method – start from the highest context and escape into lower contexts

22. MavHome Floor Plan and Mobility Profile Graph-Abstraction SampleFloor-plan • Bob’s movement profile:a j k k o o j h h a a j k k o o j a a j k k o o j a a j k k … • Incremental parsing results in phrases: a, j, k, ko, o, jh, h, aa, jk, koo, ja, aj, kk, oo, jaa, jkk, ... • Possible contexts:jk (order-2), j(order-1), (order-0)

23.  a (7) j (7) h (2) k (8) o (6) a (2) j (1) a (2) k (2) h (1) o (2) k (2) o (2) a (1) k (1) o (1) Phrases:a, j, k, ko, o, jh, h, aa, jk, koo, ja, aj, kk, oo, jaa, jkk, ... Trie Representation and Phrase Frequencies Phrases and frequencies of different orders • Probability of jaa: • Absence in order-2 and order-1; escape probability in each order: ½ • Probability of jaa in order-0: 1/30 • Combined probability of phrase jaa : (½) (½ )(1/30) = 0.0048

24. Probability Computation of Phrases = • Probability of k • ½ at the context of order-2 • Escaping into next lower order (order-1) with probability:½ • Probability of k at the order-1 (context of “kk”): 1/(1+1) = ½ • Probability of escape from order-1 to lowest order (order-0): ½ • Probability of k at order-0 (context of ): 4 / 30 • Combined probability of phrase k = ½ + ½{ ½ + ½(4/30) } = 0.509

25. Phrase ProbabilityPhrase Probability k kk ko koo o oo h j ja jaa jk jh a aa aj 0.0048 0.0048 0.0048 0.0048 0.0905 0.0809 0.0048 0.5905 0.0809 0.0048 0.0048 0.0195 0.0095 0.0809 0.0095 • Bob’s movement profile:a j k k o o j h h a a j k k o o j a a j k k o o j a a j k k … Phrase Probabilities Probabilities of individual locations can be estimated by dividing the phrase probabilities into their constituent symbols according to symbol-frequency and adding up all such frequencies for a particular symbol (location) Total probability for location k is: 0.5905 + 0.0809 + 0.0048/2 + 0.0048/3 = 0.6754

26. k h j o a Probability Computation of Individual Locations • Bob’s movement profile:a j k k o o j h h a a j k k o o j a a j k k o o j a a j k k • Phrases:a, j, k, ko, o, jh, h, aa, jk, koo, ja, aj, kk, oo, jaa, jkk, ... • Probabilistic prediction of locations (symbols) based on their ranking • Prime Advantages of Lempel-Ziv type compression – most likely location is predicted • Prediction starts from k and proceeds along a, h, o and j

27. Characterizing Mobility from Information Theory • Movement history: A string “v1v2v3…” of symbols from alphabet  • Inhabitant mobility model:V = {Vi}, a (piece-wise) stationary, ergodic stochastic process where Vi assumes values vi • Stationarity:{Vi} is stationary if any of its subsequence is invariant with respect to shifts in time-axis • Essentially the movement history “ v1, v2, …, vn” reaches the system as C(w1), C(w2), …,C(wn) where wis are non-overlapping segments of history vi and C(wi)’s are their encoded forms • Minimizes H(X) and asymptotically outperforms any finite-order Markov model • The number of phrasesis bounded by the relation:

28.  a (7) j (7) h (2) k (8) o (6) a (2) j (1) a (2) k (2) h (1) o (2) k (2) o (2) a (1) k (1) o (1) • Bob’s movement profile:a j k k o o j h h a a j k k o o j a a j k k o o j a a j k k … • For a particular depth d of an LZ trie, let H(Vi) represent entropy at ith level. Running-average of overall entropy is: Entropy Estimation

29. A paradigm shift from position based update to route based update • Encoder: Collects symbols and stores in the dictionary in a compressed form Decoder: Decodes the encoded symbols and update phrase frequencies LeZi-Update: Location Prediction Scheme Encoder Decoder Init dictionary, phrase w loop wait for next symbol v if (w.v in dictionary) w := w.v else encode <index(w), v> add w.v to dictionary w := null forever Initialize dictionary := empty loop wait for next codeword<i, s> decode phrase := dictionary[i].s add phrase to dictionary increment frequency of every prefix of every suffix of phrase forever

30. Predictive Framework: Route Tracking • Probability of a set of route sequences depends exponentially on relative entropy between actual route-distribution and its type-class • Route-sequences away from actual distribution have exponentiallysmaller probabilities • Typical-Set –Set of sequences with very small relative entropy • Small subset of routes having a large probability mass that controls inhabitant’s movement behavior in the long run • Concept of Asymptotic EquipartitionProperty (AEP) helps capture inhabitant’s typical set of routes

31. FromAEP, typical routesclassified as: { : 2 -1.789 L() -   Pr[]} where L() is the length of phrase  and  is a very small value • Threshold-probability of inclusion of a phrase into typical-set depends on its length L() Probability Computation of Typical Routes • At our context:

32. PhraseProbabilityPhraseProbability k kk ko koo o oo h j 0.5905 0.0809 0.0048 0.0048 0.0195 0.0095 0.0809 0.0095 ja jaa jk jh a aa aj 0.0048 0.0048 0.0048 0.0048 0.0905 0.0809 0.0048 Capturing Typical Routes • At this point of time and context, the inhabitant is most likely to move around the routes along Bedroom 2, Corridor, Dining room and Living room • Typical Set of route segmentscomprises of :{ k, kk, koo, jaa, aa }

33. k j o a Bob’s Movement along Typical Routes Typical Route:k o o k j a a Bedroom 2, Corridor, Dining room and Living room

34. Energy Consumption • StaticEnergy Plan • Devices remain on from morning until the inhabitant leaves for office and again after returnat theend of the day. • Let Pi : power of ith device; M :maximumnumber of devices; t : device-usage time; p(t) : uniform PDF. • Expected average energyconsumption: • Using typical values of power, number and usage-time for lights,air-conditioning and devices like television, music-system,coffee-maker from standard home, static energy plan yields ~12–13 KWHaverage daily energy consumption. Worst-Casescenario

35. Energy Consumption • Optimal (Manual) Energy Plan • Every device turned on and off manually during resident’sentrance and exit in a particular zone. • Pi,j : power of ith device in jth zone; :max # devices in a zone; R : # zones; t : device-usage time in a zone; p(t) : uniform PDF. • Expected average energyconsumption: • Using standard power usage, optimal energy plan results in ~ 2–2.5 KWHof average daily energy consumption. OptimalScenario But lacks automation and needs constant manual intervention

36. Energy Consumption • PredictiveEnergy Plan: • Devices turned on and off based on the predictionof resident’stypical routes and locations(Incorrect prediction incurs overhead) • Devices turned on in advance – existence of time lag (t) s : predictive success-rate. As s 1, E[energypredict]  E[energyopt] • For the scenario, predictive scheme yields ~3-4 KWH consumption • Successful prediction reduction of manual operations and saving of inhabitant’s invaluable time inhabitant’s comfort

37. Event types: Daily actions of a user, e.g., sleeping, dining, cooking, etc. • Event Queue • Priority Queue for buffering events • Events ranked according to time stamp. • Event Initializer • Generates the first event and pushes it into the event queue • Event Processing • Carried out with every event • Calls the event generator to generate next event and pushes it into the queue • Calls various action modules depending upon the type of event Discrete Event Simulator Simulation Structure

38. Simulation Duration: 70 days • Different life-styles at weekdays and weekends • Mobility initiated as the inhabitant wakes up in the morning and starts daily-routine • Inhabitant’s residence-time at every zone – uniformly distributed between a maximum and a minimum value • Negligible delay between sensory data acquisition and actuator activation • Prediction occurs while leaving every zone • In inhabitant’s absence, the house has minimal activity to conserve energy resources Simulation: Assumptions

39. Granularities of Prediction • Predicting next zone • Inhabitant’s immediate next zone / location • A coarse level movement pattern in different locations • Predicting typical routes / paths • Inhabitant’s typical routes along with zones • More granular indicating inhabitant’s movement patterns • Predicting next sensor • Every next sensor predicted from current sensor • Large number of predictions lead to system overhead • Predicting next device • Predict every next device the inhabitant is going to use • Details of inhabitant’s activities can be observed

40. Garage Garage Garage 100 90 80 70 60 50 40 30 20 10 14 12 10 8 6 4 2 Master bedroom Restroom kitchen kitchen kitchen kitchen kitchen kitchen Dining Room Dining Room Dining Room Dining Room Dining Room Success Rate A Snapshot of Simulation Restroom Wash room Closet Corridor closet 0 Bedroom Bedroom Energy Savings Living Room Static Optimal Predicted Predicted Actual Correct Prediction

41. Learning Curve and Predictive Accuracy • 85% – 90% accuracy in predicting next sensor, zone and typical route • Route prediction accuracy slightly lower thanlocation prediction, yet provides more fine-grained view about inhabitant’s movements • Only 4-5 days to be cognizant of inhabitant’s life-style and movements • Highergranularity keeps device prediction accuracylow (63%)

42. Memory Requirements Variation of Success-rate with table-size • 85% success rate with only 3–4 KB memory for inhabitant’s profile • Small size typical set (5.5% -- 11% of total routes) as typical routes

43. Energy Savings Reduction in Average Energy Consumption • Energy along predicted routes / locations only – minimum wastage • Average energy consumption – 1.4 * (optimal / manual energy plan) • 65% – 72% energy savings in comparison with current homes

44. Reduction in Manual Operations • Prediction accuracy reduction of manual operations of devices  brings comfort and productivity, saves time • 80% – 85% reduction in manual switching operations

45. Future Work • Route prediction and resource management in multi-inhabitant (possibly cooperative) homes • Design and analysis of location-aware wireless multimedia communication in smart homes • Integration of smart homes with wide area cellular networks (3G wireless) for complete mobility management solution • QoS routing in resource-poor wireless and sensor networks • Security and privacy issues

46. Selected References • A. Roy, S. K. Das Bhaumik, A. Bhattacharya, K. Basu, D. Cook and S. K. Das, “Location Aware Resource Management in Smart Homes”, Proc. ofIEEE Int’l Conf. on Pervasive Computing (PerCom), pp. 481-488, Mar 2003. • S. K. Das, D. J. Cook, A. Bhattacharya, E. Hierman, and T. Z. Lin, “The Role of Prediction Algorithms in the MavHome Smart Home Architecture”, IEEE Wireless Communications, Vol. 9, No. 6, pp. 77– 84, Dec. 2002. • A. Bhattacharya and S. K. Das, “LeZi-Update: An Information Theoretic Framework for Personal Mobility Tracking in PCS Networks”, ACM Journal onWireless Networks,Vol. 8,No. 3, pp. 121-135,Mar-May 2002. • A. Bhattacharya and S. K. Das, “LeZi-Update: An Information Theoretic Approach to Track the Mobile Users in PCS Networks”, Proc. ACM Int’l. Conference on Mobile Computing and Networking (MobiCom’99),pp. 1-12,Aug 1999 (Best Paper Award).

47. Selected References • D. J. Cook and S. K. Das, Smart Environments: Algorithms, Protocols and Applications, John Wiley, to appear, 2004. • A. Bhattacharya, “A Predictive Framework for Personal Mobility Management in Wireless Infrastructure Networks”, Ph.D. Dissertation, CSE Dept, UTA (Best PhD Dissertation Award), May 2002. • A. Roy, “Location Aware Resource Optimization in Smart Homes”, MS Thesis, CSE Dept, UTA (Best MS Thesis Award), Aug 2002. • S. K. Das, A. Bhattacharya, A. Roy and A. Misra, “Managing Location in ‘Universal’ Location-Aware Computing”, in Handbook in Wireless Networks (Eds, B. Furht and M. Illyas), Chapter 17,CRC Press, June 2003.

48. Technology Forecasts (?) • ‘ Heavier-than air flying machines are not possible’ • Lord Kelvin, 1895 • ‘I think there is a world market for maybe five computers’ • IBM Chairman Thomas Watson, 1943 • ‘640,000 bytes of memory ought to be enough for anybody’ • Bill Gates, 1981 • ‘The Internet will catastrophically collapse in 1996’ • Robert Metcalfe • ‘Long before the year 2000, the entire antiquatedstructure of college degrees, majors and credits will be a shambles’ • Alvin Toffler

49. “A teacher can never truly teach unless he is still learning himself. A lamp can never light another lamp unless it continues to burn its own flame. The teacher who has come to the end of his subject, who has no living traffic with his knowledge but merely repeats his lesson to his students, can only load their minds, he cannot quicken them”. Rabindranath Tagore (Nobel Laureate, 1913) Concluding Remarks