CAPNET Program Context-aware Pervasive Networking Mobile Forum’s Research Focus Area

1. CAPNET Program Context-aware Pervasive Networking Mobile Forum’s Research Focus Area

2. Contents • Introduction to the CAPNET Program • Further information on application scenarios illustrated in videos

3. CAPNET Program • Aim the essential foundation for new information and communications technologies in the context-aware mobile ubiquitous computing domain. • Focus the mobile user perspective, where other terminals and services are utilised via a personal mobile device in a gateway-like fashion. • Goal iterative and progressive development of technologies enabling truly pervasive networking environments for applications, using existing and emerging computing platforms and environments as a technological basis.

4. Motivation for CAPNET Program • Pervasive networking is a promising area for creation of environment aware personalised mobile services. • The large penetration, capability increase and miniaturising of mobile networks and devices enable the creation ofcheap pervasive services. => huge business potential exists! • Next generation value nets will be built on physical sub-nets & smart personal spaces. (E.g. local mobile commerce, smart communication, p2p interactions & consumer-initiated content creation, sharing of services/content, virtual value communities) Main Aim of the CAPNET Program To create ubiquitous technology enablers – map lucrative discontinuities “Grow” necessary workforce/competence to launch ubi business: spin-offs, new product businesses & services, fill white-spaces Take lead in new exiting “4G” area in a concrete fashion • Support infrastructure for pervasive services: • Flexibility in network programmability • Multi-channel, multi-zoned service offering • Advanced resource and profile management • Easily adaptable, self-configurable services

5. CAPNET Partner Network 2002 Nokia Ventures Organization IBM ISG Univ. of Oulu www.ee.oulu.fi/mvis/research/isg/ Hantro Zoned networking MediaTeam Univ. of Oulu www.mediateam.oulu.fi Context solving Adaptive content Intelligent SW ProximityTransactions Environment management University of Linköping Ubicomp via mobile device CCC Reliability Umiacs/LAMP Univ. of Maryland www.umiacs.umd.edu Advanced UI Tekes Nets Programme Content Processing Ubi concept design CAPNET Areas HCI Univ. of Oulu www.tol.oulu.fi/tutkimus/index.html Sonera Research Nokia Mobile Phones Business

6. CAPNET Research Process

7. User experience • An application scenario driven research approach • to ensure that real human needs are the driving force for the technology development • research problems are derived from actual usage needs and scenarios • requirements derived from the application scenarios are used as a basis for architecture and technology development. • prioritised list of scenarios is used for defining the order of implementation of CAPNET demonstrations • selected application scenarios are illustrated in the visualizations at the end of this presentation and on the videos provided here • User experience evaluation • methods and frameworks for evaluating and measuring the factors affecting to the user experience in ubiquitous environments • evaluation of CAPNET demonstrations and implementations from the user experience point of view

8. CAPNET Universe CAPNET RUNTIME ENVIRONMENT (CRE) CRE ENGINE DYNAMICCOMPONENT DYNAMICCOMPONENT CRE ENGINE DYNAMICCOMPONENT CORECOMPONENT DYNAMICCOMPONENT CRE CORECOMPONENT CRE CAPNET Architecture • Each device contains a CAPNET Runtime Environment - which in turn contains the engines • Engines are the basic entities of a distributed system requiring very little resources as a basic setup • Engines consist of components • Core component offers basic functionality for an engine type • Dynamic component can be tailored according to services needs

9. UI Application Logic … Application level Media Processing Context Management Virtual Storage … Value adding service enablers … Component Management Connectivity Management Service Discovery UBI enablers Engines Are the Building Blocks of the Architecture • Each engine is targeted to a domain area • Component Management Engine controls all components and partitions application optimally • Connectivity Management Engine controls all connections • Service Discovery Engine finds components, engines, and external services • Context Engine provides contextual information for other engines, etc • UBI enabler engines have to be included in every CRE • Light engines in mobile terminals, more capable engines in the network

10. Architecture Taxonomy Main properties (cumulative 1-3): 1. Direct Engine-Application Connection (direct value add) • Engine can be used directly from Web • browser or application • Engine can be installed in mobile device or network server in local/global domain • Engine implements a micro-server functionality (calm: waits to be used, proactive: functions under scripts) Browser Application or E Local/Mobile Environment • Engines can establish temporary hierarchy • Application can use multiple engines • Engine can use engine(s) • Engines can be located in different local domains • Nature of connection may vary • Engine can move to optimize service • Engine functionality can be upgraded 2. Local Domain of Networked Engines (proximity value add) Browser Application or E E E E E Local/Mobile Environment Local Hosted Environment 3. Hybrid Domains of Networked Engines (richest value add) • Engines can establish a fixed hierarchy • Engines can seek services offered by • other engines • Engine can function as a transparent gateway for other engines. • Engines can form functionality islands (larger servers) coping with load, errors, • QoS demands E Browser Application or E E E E E Remote Hosted Environment(Global domain) E E Local/Mobile Environment Local Hosted Environment(Local domain)

11. Prototyping • Implementing the architecture and selected application scenarios • Aim is to • demonstrate service concepts • test developed architecture • test individual technologies within a complex system • e.g. new service discovery and routine learning algorithms • First prototype • Scenario: Know your where abouts • Utilized existing technology: PersonalJava, XUL, XML-RPC, Jini • Implemented in Java but uses language independent communications

12. CAPNET Test environment • Provide a realistic platform for the deployment of software prototypes that are produced within Prototypes program area. • Basic architecture and common components can be tested and evaluated in the environment • Prototype software packet as a new service scenario can be verified in the environment • Provide feedback to refine both architecture and fundamental methodology creations • New invented techniques as well as applications scenarios are to be evaluated in the test environment in terms of usability, from the viewpoint of perceived user experience • How the users experience the ubiquitous environment and its applications, • What parameters affect the user experience. • Evaluation results will provide feedback for the new application scenarios creation • The test environment can be used for the demonstration of the concrete deliverables of the CAPNET program. • 1st Test Environment focuses on indoor Wireless Local Area Network, based on IT-NET • With wireless positioning system; PDA as the main user device; simulating main building & offices • Next Test Environment will be extended to outdoor WLAN & Wireless Wide Area Network • Outdoor WLAN based on SmartRotuaari; wireless positioning; urban downtown & business centre; • WWAN based on Octopus environments; GPRS connection; suburban, rural, general coverage; • Sensor technology for human activity analysis will provide context-information from the test environment.

13. Application Scenarios

14. Lunch Assistant • Simo, Raimo, and other employees of CAPNET program are having a meeting in Tietotalo’s meeting room. While the chairman is speaking the display of Simo’s mobile device starts to blink. The device is in a meeting mode. • The device has recorded its locations from the beginning. The profile changes and the use of the device (times/places where/when he does not answer the phone etc.) have been saved into a database as well. From this data, the device has identified and analyzed Simo’s routines. • Enabling phenomena: routine learning. • The device displays a message informing that Simo has another meeting at 13:00, and he should order a meal in advance so that he has time to eat, and be back in time for the next meeting. The device suggests different restaurants based on the travel time and available time, displays the menu of the restaurant Simo chooses, and orders the meal. • The device knows how much time it will take to travel between locations. The device uses Simo’s calendar information, and combines this information with his context-information. Also, the device has access to restaurant databases, and it has built a profile for Simo according to his preferences. • Enabling phenomena: routine learning, profiling, easy downloading of client applications from the restaurant, WLAN connection between Simo’s mobile device and restaurant servers, advanced human-computer interface for browsing and ordering.

15. Lunch Assistant • While Simo is in the elevator, the mobile device informs that Simo is going to play badminton in the evening, reminds him of his wife’s nameday, and asks wheter he wants the sauna to be warm after badminton game as usual. • The device notices that Simo has time to receive notifications about his evening program. It has learned that Simo wants the sauna to be warm every time after badminton, and offers to do it automatically. • Enabling phenomena: routine learning, location awareness. • Simo goes to the restaurant, his meal is delivered to him, and he eats while Raimo is having his lunch outside at Piccolo fast food. After the meal, Simo pays for the food with his mobile device, and gives the waiter a tip for good food and smooth service. • The mobile device gives the restaurant information about its location through Bluetooth connection, and the meal is ready when Simo enters the restaurant. The payment is done through Bluetooth connection as well. • Enabling phenomena: location awareness, advanced human-computer interface for browsing and payment. • Simo is back in the office in good time, and has time to go through news on an adaptive notice-board. • The mobile device is a remote control for an adaptive notice-board, and Simo can browse information stored in a news server with his device. • Enabling phenomena: adaptive notice boards, WLAN connection between the news server and the device, personal profiling for news preferences.

16. Local Environment User Device Location info through Bluetooth & WLAN Customer-aware info organizing Customer-aware info organizing Menu & order through WLAN Order Menu & Table Info Customer Record Info Menu & Table Info Customer Record Info Remind Menu & Table Info GUI/HCI Lunch Reminder Application & Middleware Menu browse & order GUI Lunch Manager Location informer Time Loc WLAN Bluetooth Internetworking Connectivity Network Connection UMTS Menu & order through UMTS General Environment Lunch Assistant: Technology scenario assessment

17. Shopping and Wish List • Maria is wandering around shops and whenever she sees a nice item, she saves the information about the product onto her wish list, so that she can browse and choose the best ones later on. • Maria selects certain products to be remembered. The code, the price, and the picture of each item are dowloaded, and transferred into her personal database from the store database. • Enabling phenomena: Symbiam-based mobile device with color screen, bar code reader, Bluetooth connection to store database,and WLAN connection to a personal database for wish list. • Simo also has a wish list, he saves the information about a car onto his personal wishlist. During saving, Simo’s mobile device informs him about his wife’s (Maria) birthday. The device asks whether Simo wants to take a look at Maria’s personal wish list, so that he can choose a present for her. • The device knows about Maria’s birthday based on his calendar information. • Enabling phenomena: combining calendar and contex-information.

18. Shopping and Wish List • Simo browses Maria’s wish list, the mobile phone shows him his current location, and the shops nearby that have those items Maria has chosen onto her wish list. Simo chooses the nearest jewellery store and finds a necklace he likes. No more unfit presents for Maria. • Simo has access to Maria’s wish list, so he can browse it with his personal mobile device. The device only suggests the stores that are near Simo’s current location. • Enabling pehnomena: advanced human-computer interface for browsing, WLAN connection to Maria’s personal database, location awareness.

19. Shopping and Wish List Technology scenario assessment Local Environment User Device ProductInformation Selection, products browse Customer &Request Info Payment,request GUI/HCI User Shopping mode ?on-off Network Connection WLAN connection Sense location Landline connection Application & Middleware GPRS connection General Knowledge(Rules, service providers’ locations, maps etc.) Recognizecontext Personal Knowledge(Rules, profile, calendar etc.) Customer Account Info Services: shopping, navigation, localization, map databases, information databases etc. General Environment See legend from next slide =>

20. Context Aware Well Being • Simo is interested in his own well-being. He purchases an active well-being application from a service provider. • Terminal side Java/MIDP Application is downloaded to Simo’s UMTS mobile phone. • Application can access (with Simo’s approval) personal profile and properties that are related to his well being from Simo’s profile manager. • The application starts to monitor Simo immediately. • The application identifies devices Simo is wearing. The application identifies devices (BlueTooth, UPnP, Jini, Salutation) it can use to improve its functionality. (eg. wrist watch that provides detailed energy consumption, blood pressure and pulse information) • The application makes automated daily nutrition plans and produces a shopping list for Simo • The application monitors Simos eating to chart the energy supply and suggests scheduled meals and snacks to help the user to achieve the target body weight and/or fitness level.

21. Context Aware Well Being • The application makes an exercise program for Simo. • The application makes a daily exercise plan for Simo. • The application monitors Simo’s physical condition and the results of Simo’s exercises, and adjusts the exercise plan according Simo’s development. The results are stored to Simo’s personal exercise calendar. • The application can be used to book time from the gym web service (WSDL/SOAP over UMTS) • The application works in collaboration with local grocery shop • The well being application can inform the shop that Simo is coming there and has just finished his exercise. The shop can give Simo special offers.

22. Local Environment User Device GUI/HCI Application & Middleware Connect to 3rd party application Install application Connect to 3rd party hardware Monitor user, habits, consumption Context Recognition Connectivity Network Connection Service catalog (Service provider accessibility, 3rd party) General Environment Context Aware Well BeingTechnology scenario assessment • Access to other wearable devices • Network access • Push content • Disconnected operation • Connected operation Context knowledge (preferred gym, shops, eating habits)

23. Ad Hoc Networking • Simo works as a project manager in a software company. As a first thing tomorrow morning Simo has to inform his co-workers about the new project so he decides to prepare a short presentation at home in the evening. Simo prepares few slides and saves them into the his personal database. Simo dresses himself up overcoat, puts the mobile phone into his pocket and leaves the house. Audio device, TV, PC and coffee maker and lights are turned off when Simo leaves. The house enters into the economy mode. • When user is leaving the house Local Bluetooth server informs Context server about user’s actions. General environment server turns off the audio device, TV, PC and coffee maker . • Connection between mobile phone and Local Bluetooth server is closed. • Bluetooth connection (Local Bluetooth server), Landline connection (Context server), • Mobile phone and home appliances (audio device, coffee machine, lights, PC, TV etc.) must have a Bluetooth chip.

24. Ad Hoc Networking • The next morning Simo arrives at his office meeting room, other participants of the meeting are already sitting around the table. Simo selects an icon in his mobile phone and the mobile phones shows him a menu. Mobile phone presents a list of available services that he can use in this meeting room. Simo sees a list: audio device, data projector, lights and resources. • User’s mobile phones Bluetooth monitoring is automatically activated. • In the meeting room mobile phone observes the local environment around it via Bluetooth and notices Local Bluetooth server’s access point which continuously sends Bluetooth signals (Bluetooth base station feature). The phone starts to communicate with it. The phone gets to know that access point is a meeting rooms local access point and the phone tells this local point who it is. Phone identification is made by SIM-card recognition. • Local Bluetooth server gives the mobile phone the list of activities (data projector, resources etc.). The mobile phone filters the list according the context and user preferences and shows it to the user, and in addition adds ”Communication” to the list for signalling direct Bluetooth connection between other Bluetooth devices. • Bluetooth connection (Local Bluetooth server), Landline connection (Context server), System needs context recognition capability tool in order to sense the arrival of Simo’s mobile phone in the meeting room. • Mobile phone and office appliances (data projector, printer, laptop etc.) must have a Bluetooth chip.

25. Ad Hoc Networking • Simo selects ”Projector” from the list. The projector is turned on. Next Simo selects “Lights” and adjust the lights a bit. • Simo selects ”New minutes” from the list and watches the minutes and especially the list of present and absent people appearing. • Next Simo selects resources and the mobile phone shows the files that have been edited last. Simo selects the slide set he wrote the previous evening. Simo’s slides appear on the wall and Simo goes on with his presentation. • Mobile phone connects to personal database and shows the list of the files that have been edited last. • Mobile phone sends info from the selected task and the selected document to the Local Bluetooth server. Bluetooth connection transmits the document wirelessly from Local Bluetooth server to the data projector. No cables are needed for transmission. • Bluetooth connection (Local Bluetooth server), Mobile phone and office appliances (data ptojector, printer, laptop etc.) must have a Bluetooth chip.

26. Bluetooth 1 2 List of Ad Hoc Services Landline 3 4 inquiry and data saving presentation Document selection 6 5 Presentation Data projector (Bluetooth) Ad Hoc Networking :Technology scenario assessment General Bluetooth Environment User Device Local Bluetooth Environment Mobile phone Bluetooth compatible device availability & user info Personal Knowledge (Rules, profile, calendar etc.) Bluetooth monitoring on User Local Bluetooth server Context server

27. Thank You!For further information please contactmarika.leskela@oulu.fi