Chapter 1: Introduction to Pervasive Computing

1. Chapter 1:Introduction to Pervasive Computing • A New Start … • What is a Computer? • Computer Systems Vs. Our Needs • Growth of Computer Technologies • The Current Status and Current Trends • The Next Step … • Integration of Technologies: Pervasive Computing • Some Definitions • Pervasive Computing Systems Components

2. A New Start … • Pervasive computing, a new topic • A “new” angle to look at the relationship between our world and the computer world • What is a computer world? • The world created by a computer (logical and NOT physical) • Data structure + algorithms => execute • Computers areNOTComputers …So, what are they • Thus, we need to start to learn it from the fundamental computer concepts and rebuild the concepts • New applications and requirements (become parts of our living) • In hospital, shopping mall, driving and navigation, traveling and sight-seeing, home and office, all aspects of our daily lives, etc. • Design Problems and Solutions • New devices and tools • Moving from mobile devices to pervasive computing devices, smart dusts, sensors; limitations and constraints • Fixed and mobile networks; connectivity everywhere; quality • J2ME, Palm OS, Symbian OS, WinCE, TinyOS, TinyDB, etc.

3. What is a Computer?

4. What is a “Computer”? • Fundamental Question: What is a “Computer”? • Answer One: Structurally, a specially designed machine: CPU, memory and I/O devices + software (OS (+ middleware) + applications) • CPU performs computation • OS manages resources • Applications define what you want to be done by the computer • Memory for temporarily storage of data (information) for computation • Receive inputs and then generate outputs • Compare: computers Vs. human beings (structurally)

5. What is a “Computer”? • Answer Two: Functionally, a machine that can perform computing jobs (computation) • What is computation? • I finished a program then I submitted it for execution (computation) by a computer • Program: a sequence of instructions telling the computer what to do • What will be the consequences if the instructions are incorrect? • Note: computation is not just referring to arithmetic. It also includes logic operations • Compare: the way that we do a task with the way that computers do a task (stepwise)

6. What is a “Computer”? • Answer Three: Functionally and operationally • Mostly, a computer can be used for multiple (general) purposes (operating system + applications) • Mostly, at fixed locations although they are connected to fixed or wireless networks and can be carried around (home uses or office uses) • If you have a job needed to be performed, you have to find a computer sit in front of it, and then submit your job request through the specific interface of the computer. (You know what is a computer and then you know how to find one) • General purpose computers Vs. specific purpose computers • General purposes: loading different programs for different purposes • Specific purposes: pre-loaded programs for specific purposes. May not be able to add more • You perform “computation”. Are you a specific/general purpose computer?

7. What are the relationships between You and Computer Systems? Separated!

8. Computer Vs. Our Needs • Q: How can we finish a task? How about in the past when we did not have computers? I.e., How to communicate, perform computation, finish a task, deal with uncertainty, etc., in our daily??? • Similarity: your real world Vs. the computer virtual world • Computation: 1 + 1 = 2 • Definition and change (operator) “one” to be another “one” • How about your daily life? • Your world: One state => an event => another state • Logically the same • We model our living logics (defined in applications/programs) in the computer world (a simulated virtual world) • Execute the logics and generate results (outputs) • The generated outputs are returned into our world • Real world (you are living in) Vs. computer world (virtual) • Limited interaction, mainly through input/output • Growing in degree of interactions: virtual reality (games)

9. Computer Vs. Our Needs Computer: creates a computer world (data structures and states/values) in its memory and then follows the instructions step by step to “run” the world. Each step changes the current state to a new state (modeling the changes of events in the real world) Environment Events (interaction) Objects (Physical & logical) You Outputs Inputs Computing units Computer world

10. Computer Vs. Our Needs • Computers are similar to us structurally (and functionally) • Processing unit + memory + I/O devices (stepwise) • Why do we build computer systems (create the computer world)? • Find someone to help us to deal with events in our environment • Build a computer system following the ways in our world • Computers and human being both have their limitations and their strong points • Computers: fast in computation, large memory, more accuracy (stable and repeatable) and reliable (no lying) • Human being: …. Think about yourselves • A general trend from the beginning of human history: wants to “improve the quality” of our daily lives • Add more “computers” in our world => more computer worlds • More interactionsbetween our living and computing units • Our needs create more computer systems • Computer systems digitize our world and change our daily lives

11. Computer Vs. Our Needs • How do computers change our living and the way of doing things? • When you want to book a restaurant? • You want to make a stock analysis/trading? • Learning and doing assignments??? • Control of machines and vehicles • Information is modeled in database (digitized) • Once digitized that they can be copied, transmitted, shared and repeated easily • Plagiarisms, privacy and security • THEN, we change the computers and the applications when we see new applications, in particular in interface and the ways of how to use them • Consequences: more computing units, “better performance” and more applications, and new ways of using computer systems in our living environment

12. New Applications Our Needs Applications New Needs Our behavior to deal events occurred in our environment

13. Why separated? Look back the growth of computer systems… How about the future?

14. Growth of Computing Technology • Standalone computer Vs. networked computers • Standalone computers: for computation only (like a calculator) • Network & Distributed computing (starting from about 20 years ago) • A computing job may be divided into sub-tasks to be performed by different processing units (in parallel/ or sequentially) which are connected by a network • Better performance and more reliable (adv. of DC) • Mobile computing (starting from about 10 years ago) • The users may move around and computing units (notebooks, laptop, etc.) are carrying by users • The system supports computing task requests from moving users through a wireless network • Note: wireless ≠mobile (cordless phone ≠ mobile phone) • Wireless: no wire (may be stationary). The mobility is usually quite limited in scope (stationary is only a relative concept) • Mobile: the connectivity is still maintained while you are moving or stationary

15. Growth of Computing Technology • Why do we have such a computing trend in the past? • Distributed: performance reasons • What is the performance requirements? • Throughput, response time, quality, reliability, etc. • Our life, thus our needs (follow us) are mobile • Nothing is static (absolutely) in our world • Our world is distributed (centralized is again a relative concept) • Note: computers simulate our world and are asked to perform logic operations similar to the way that they happen in our lives • Applications need to move (follow) with us

16. The Current Status • Dramatic improvements in hardware technologies and system operation environment • More powerful and smaller devices, higher bandwidth networks, more network connection points • Build on top of existing trends • Driven by our needs and new technologies (needs <=> products) • New needs create new products • New products create new needs • Nowadays, many electronic devices are equipped with a processor (intelligence & computation) and memory for performing computing jobs, i.e., mobile phone, projector, camera and MP3 players • Are they computers? Yes? Perform computing jobs = “Computers” • Remember what is the Chinese meaning of “computing” • Each one simulates a “part” of our daily life

17. The Current Status • Currently, most of these devices are embedded and close systems • Embedded system: predefined functions to be completely performed by the device itself • Close system: standalone system, no direct connection (I/O) with other devices (separated) • Networks (fixed networks, mobile networks, ad hoc networks) are building to connect the devices together and to other parts of the systems • i.e., bluetooth devices, your video camera, HiFi and TV set are connected to your computers • Current limitations: the connections are still not reliable and have limitations in bandwidth/stability. Energy and computation power are still problems. Reliability and lack of good software are other problems

18. Current Trends • System and Architecturally • Further decentralization and getting smaller in size • Continue to shift from a centralized view to a decentralized computing • Many small computing units connected by networks • Price/performance and reliable distributed computing services • Different computing units for different purposes/functions and may work together (collaboration) • Computation (performed by multiple computing units) is performed close to the place (localization) where the event is detected (carrying them) • Computation is performed at device level if possible (what are the advantages?) • From Client/Server to Peer-to-peer (same level) • Advantages of distributed computing • Faster in response (localized processing) • Minimize data transmission workload

19. Current Trends • Diversification Vs Integration • Different types of embedded devices for their own functions (mobile phone, MP3 player, camera and TV) • Diversification: mobile phone, handheld computer, MP3 player, camera… • Integration: one device contains all • Integration in the future??? • Business and performance reasons • High connectivity (both fixed and mobile) • The computing units are connected if necessary (even moving) • The computing units work together cooperatively to complete a task • Heterogeneous networks with different QoS of services • QoS: quality of services (a measure of how good is the service) • Normally, we require a guarantee of QoS • Is it better to have a single high performance global network instead of multiple networks? Yes, no handoffs and easier to manage but it may not be adaptive

20. Heterogeneous Network integration of heterogeneous fixed andmobile networks with varyingtransmission characteristics regional vertical handoff metropolitan area horizontal handoff campus-based in-car, in-house, personal area What is handoff? From one hand to another hand From: Mobile Communications (Schiller)

21. Current Trends • Application and functionally • Real-time tasks • Our world is changing => using computers to assist us to deal with the “real-time” events occurring in our world • I.e., Booking, car park status, navigation, on-line games • Real-time: current state => then, computation, if need action (actuation) => generate action immediately (with a pre-defined time frame) • Real-time: tasks have deadlines on completion • Multiple tasks => multiple deadline constraints need to be met => scheduling problem • Multi-data integration and processing • Integration of (multi-media) data from different sources for processing for better decision-making • Sharing of resources and information

22. Current Trends • Proactive Operations • Event monitoring/surveillance: want to get the latest status of the world, then react to it immediately, i.e., latest stock prices, sport results, ticketing and transportation information • Define monitoring rules • Once a new data item is received, check the new system status • If the conditions are met, trigger an action (actuator • If not ignore (filter out) the received data items • A lot of data values are pushing in front of you and require attention for processing • Prioritizing, filtering and consolidation (summary) • Filtering and pushing interested values => You can concentrate on those interested tasks

23. Pervasive Computing A new direction for the future computer systems Note: Different people may have different understandings about pervasive computing

24. Next Step: Pervasive Computing • Build on top of existing trends => more interactions with the computer world and more convenient to use the systems => to more the computer world into our “real” world • Computers do not look like computers but computations are performed everywhere in our environment • Embedded into our daily life and even materials/devices • Small, cheap, mobile processors and sensors, in almost all everyday objects, i.e., on your body (“wearable computing”) and embedded in environment (“ambient intelligence”) • The environment is embedded with a “logical” assistant to help us to deal with our living environment and events related to us • Connected embedded systems (why connecting them?) • Cooperative operation of devices • Sharing of resources and integration of information • Better performance and higher quality services (reliable)

25. Results: Invisible Computing? • Invisible and everywhere: wireless, dynamically configurable, remote access, adapting, … • Why dynamically configurable? • Adaptive to changing environmental status => why the status is always changing? • The capability of human beings in dealing with the real world (environment) is improved (why? Like a Robot and assisted by other robots) • But, this is not easy to achieve. It requires integration of various techniques and knowledge

26. Smart Spaces (Operation Environment) • We are living within (interact with) our environment • Environment: school, home, office, city, train, etc… • Interactions with the environment: eating, learning, reading, traveling, listening to music, etc.. • Smart space: an environment embedded with computing devices (sensors) to detect your needs/status and perform jobs to react to your needs proactively (interactive and anticipating) • Example: automatic adjustment of heating, cooling and lighting levels in a room based on occupant’s electronic profile

27. Not a Robot But an Intelligent Servant • Conventionally, we submit commands (requests) to instruct the computers to perform computing jobs • I.e., calculate a sum, order a ticket booking • In pervasive computing a computer is more than a robot (a passive machine) • Do we require the robot to follow you? • An intelligent servant : it can anticipate what you want before you submit your requests (proactive) • The servant logically exists everywhere and arrange your life before your any commands based on prediction • How to make the prediction? It is difficult especially to make the right one

28. Proactive Services • Location-based services • Different services are provided based on different locations of the users (or at different places) • Emphasized on data and information delivery (broadcast and dissemination of information) • I.e., shopping malls, schools, toilets • What do you need when you are in a shopping mall? • Context-aware services • Different services are provided based on different context detected in the system (situational) • Surveillance and triggering (monitor & react) • Detecting system status (e.g. intruders? crisis?) • Any emergence events -> timely responses (triggering)

29. Some Definitions • Ubiquitous/Pervasive computing • It is everywhere • It is invisible, you don't notice it (minimal intervention) • It is inside things, in the infrastructure • It is integrated in your everyday life. It is invisible and useful, but it does not imply mobility • Mobile computing • It can move around and no wire connected (wireless) • Distributed computing • May be static (fixed locations and wired connection) • Why wired connections? • A task is divided into sub-tasks for parallel/sequential processing • Everyware: exists everywhere(what? Computing units)

30. Pervasive Computing Vs. Mobile Computing • Mobile computing Vs pervasive computing • Structurally are similar (i.e., mobile devices and mobile networks)but functional and operation requirements are different • Mobile computing: • Support mobile applications • Mobile networks so that users and computing units can be mobile • Applications are mobile and access to mobile data (sources) • Pervasive computing • Also have mobile applications (networks) but more than that • Main requirements are in how to use the applications and how the applications fit into our life. They need to be smart (mobile devices Vs. smart devices) • Requirements: smart spaces, invisible, proactive localization, etc., and applications become part of our life. They become our assistants and secretaries (Fig. 1 of Sat’s paper)

31. Pervasive Computing Vs. Mobile Computing Distributed Systems Pervasive Computing Systems Mobile Computing Systems

32. Pervasive Computing Goals • Pervasive computing is roughly the opposite of virtual reality where virtual reality puts people inside a computer-generated world • Pervasive computing forces the computers to live out here in the world with people (become parts of our world)” • Computers become not computers • “.. Make a computer so imbedded, so fitting, so natural that we use it without even thinking about it.” => unaware and easy to use • Computations are performed in every part of our life to enrich our living and improve our ability in responding to the changing environment => more interaction with environment • “In the 21st century the technology revolution will move into the everyday, the small and the invisible…The most profound technologies are those that disappear. They weave themselves into the fabrics of everyday life until they are indistinguishable from it…”

33. Pervasive Computing Views Environment (Your world) Events Objects (Physical & logical) You Embedded Computing units Computer world

34. How to achieve the Goals of Pervasive Computing? Integration of technologies! NOT just within Computer Science BUT also other disciplines

35. Putting Them Altogether • Progress in • computing speed • communication bandwidth • material sciences • sensor techniques • computer science concepts • miniaturization • energy and battery • display technologies • … • Enables new applications • Create new computing concepts • Computing becomes a part of our life • “Post-PC era” business opportunities • Challenges for computer scientists, e.g., infrastructure

36. Integration of Computer Science Technologies • Distributed processing • Peer to peer computation • Cooperative processing • Distributed transactions management … • Network and mobile Communication • PAN, Wireless LAN and WAN • Sensor networks • Mobile Ad Hoc Network (MANET) … • Database systems • Concurrency control and consistent data management • Data dissemination • Broadcast Vs. Pulling • Update, stream processing and temporal consistency … • Control Theory and Real-time Systems • Scheduling • Feedback control … • Semantic and context modeling (situational and different context general different responses) • Image processing, pattern and voice recognition … (Integration of multi-media information for generating different contexts)

37. Poverty of Existing Computer Systems Heavy, GUI Intrusive, MIPS happy Virtual reality, Agents Situated and behavioral science Philosophy Psychology Behavioral studies Pervasive Computing Technology Trends Displays, Processors Low power, Packaging wireless Integration of Multi-disciplines Enabling your capability in response to the changing environment

38. Overview of Pervasive Computing Systems What are the components? Anything new and special?

39. Components ofPervasive Computing Systems • Front end devices, networks, backend server, operating systems, tools and development, and applications • FE devices: for capturing the system status (generating updates) and for input/output • User devices: PDA, notebook computers, mobile phones, wearable computers, etc.. • Smart devices and spaces: sensors and actuators, smart control (responsive and proactive to the environment) • Smart materials: electronic ink, smart clothing, smart glasses, smart label, etc. (not all are electronics) • Smart appliances: networked electronic appliances

40. Components of Pervasive Computing Systems • Work stations: for integration of data from various sources and devices • Backend servers: for processing the updates and responding to the events occurred in the environment • Networks: connecting front-end devices to the workstations and servers • Satellite communication, GSM, 3G, wireless LAN, personal area network (PAN), ad hoc network and sensor network • Operating systems: build on top of the networks for connection, i.e., Palm OS, WinCE, EPOC, etc. • Tools and standards: for development, i.e., J2ME, Jini, Bluetooth, WAP, GSM, etc. • Smart applications (entertainment and services): MP3 players, game console, digital camera, intelligent toys, etc. (cooperative and work together)

41. Server Store & Process Information Standards HTML GSM HTTP Jini WAP Interoperability XML PC/SC OpenCard Java SQL Bluetooth SyncML Workstation Edit & Manage Information Device Create & Access Information Services Content Communication Infrastructure A generic view of pervasive information technology Pervasive Computing, 2ed. Springer

42. Pervasive Devices Smart Controls Intelligent Appliances Thermostat Medical Monitoring Smart Card ATM Kiosk Manufacturing Process Controls Smart Labels Telecommunication Switching Equipment Point of Sale Terminal Gas Pump Control Vending Machine Home Controls Intelligent Toys Television Pager Screenphone Digital Camera Smart Phone Vertical Market Devices Game Console Set-top box Handheld Cellular Phone MP3 Player Entertainment Systems Information Access Automotive Pervasive Computing 2ed. Springer

43. Wireless Sensors Monitoring the Environment • Why sensors? Work like our sensors. Small and can distributed in large number • What is a sensor? • Sensing the environment (i.e., temperature, pressure, presence of unknown objects, etc.) • Sensors continuously (periodically) capture the current status of their operation environment • MICA Motes, sensors, and TinyOS => go further small and tiny and large in number www.xbow.com

44. A label for identification and counting (simplify the identification process) Like a low-end smart card with memory and processing power (embedded device) Identify objects from a short distance (~10m to 100m) small IC with RF-transponder Wireless energy supply magnetic field (induction) Read/writeable Low Cost => ~$0.1 ... $1 consumable and disposable Flexible tags laminated with paper RFIDs (“Smart Labels and Identifiers”) www.aimglobal.org/technologies/rfid/resources/papers/rfid_basics_primer.asp

45. Smart Clothing • Conductive textiles and inks • Print electrically active patterns directly onto fabrics • Sensors based on fabric • e.g., monitor pulse, blood pressure, body temperature • Invisible collar microphones • Kidswear • Game console on the sleeve? • Integrated GPS-driven locators? • integrated small cameras (to keep the parents calm)? http://www.sensatex.com/

46. Today’s Wearable Computers • Wearable computers: the computers that you can wear • Simplified I/O (reduced display) and voice input together with simple panel to make it more easy to use • Compact design to make it easy to carry (reduce unnecessary components) • What can you do with this devices? • Easy to operate and fit into your working environment • What will be the future wearable computers? www.xybernaut.com

47. Body Area Networks (BAN) • Networks are required to integrate the components to form a system • Very low current (some nA), some kb/s through the human body • Possible applications: • Health monitor • Car recognize driver • Pay when touchingthe door of a bus • Phone configures itselfwhen it is touched • Various network scales: • WAN, LAN, PAN • Different performance • Different connectivity • Like a road system with different connectivity and performance

48. Personal Area Network (PAN) • Various network configurations: • Static vs. dynamic • Dynamic: ad hoc network • Ad hoc network: changing configuration and workload • How to group and connect? PAN: Personal area network

49. Our world Reach out more Indirect connection Peoples Objects Direct connection You Events

50. Example Projects and Applications • Oxygen • Smart home • Data recharging