Fasten the seat belt, here we Go

1. Fasten the seat belt, here we Go

2. Home sweet home, where the lights along the corridor flicker

3. The 3500 °F Incinerator for the Charcoal Grill 3500°F !!

4. The Duel Between Stew and IceCube Hey, Who Turns Off The Light?

5. Attention! Time for Roll Call!

6. Does Your Smart House Know Better?

7. A Safety-Oriented Programming Model for Pervasive Computing Systems Pervasive & Mobile Computing Lab Hen-I Yang April 21, 2007

8. Characteristics of Pervasive Computing Systems • Interaction with Physical World • Highly Personalized/Customized • Dynamicity • Open • Diversity/Heterogeneity • Complexity and Scalability Issue • Predication on Human Intention • Multiple Modalities in Interface

9. Safety Vs. Expressiveness Proposed New Model

10. Y = X/0 Divided by 0 Interrupt Memory access violation Fire Alarm Impermissible Context Out of Op Range Exception Interrupt & Exception Handling

11. The Four Elements

12. Elements in Pervasive Computing Environment Devices Services Users Space • Sensor • Actuator • Service • Applications • User • Space • System Support

13. Element 1: Device • Sensors, actuators and other smart appliances are the I/O devices that serve as the bridge between the physical world and the pervasive computing systems. • Source of data and/or sink of commands • Accurate description of devices is the key, it defines how to interact with these devices

14. Modeling of Device • Service oriented architecture • Can be interpreted on the fly, or created and compiled prior

15. Safety Measure for Device • Automatically appended exception handling routine • Device Safety: Avoid conflicting directives and unsafe/unacceptable operations, similar to Electric Breaker • Regulate the incoming commands and detect abnormal command/access pattern • Using maximum and minimum values to filter or red-flag abnormal values

16. Element 2: Service • The core active member of pervasive computing system • More complicated than the usual software in desktop because of the heterogeneity and dynamicity • Service/Application • Service Lifecycle management is crucial • Implemented • Aligned (Target Setup, possibly using user preferences) • Binded (to devices or other services) • Context input as conditions • Logics/Actions/Behaviors • Change of Preference • Change of Binding • Change of Conditions • Priority Level

17. Modeling of Services Service Binded Active Active Contexts Impermissible Context Event Active Context Change Event Services Services Service Registry Services Services Services Devices Programmed Binded User Preference Aligned User Preference Change Event Terminated Service/Device Change Event Service Behavior 1 Service Behavior 2 Service Behavior n ….

18. Safety Measure for Service • Service Safety: Whenever device safety, user safety or space safety is violated, the exception handler will suspend or terminate all relevant services • Services have to properly Binded and Aligned before become eligible for execution. The existence of Terminated state ensure each service has a Hand Break that can be activated when needed • Throughout execution, services subscribe to relevant active contexts, and suspend or terminate when impermissible contexts become active • Service is as safe as the safety feature of the language used to implement it, but the safety measure reduce the risk of damage caused by services

19. Element 3: User • In every single pervasive computing system, even those aiming at automation without human interventions, the ultimate goal: to achieve and fulfill functionality defined by users’ desires and intentions. • There will be no need for the existence of any system if there was nothing users want to achieve. • Integral part of pervasive computing system, as their behaviors or mere existence inadvertently affect the system

20. Modeling of User • Digital representation: User Profile • Static profile • Describes users’ relevant attributes and preferences • Used to setup the goal of the service and applications • Critical for customization and users’ intention prediction • E.g. special assistance needed, or the preferred room and water temperature settings, or even users’ calendar • Dynamic profile • Data gathered by the system about the user • Updated by the system • Treated as contexts in the system • E.g. the current location of the user, the blood pressure and glucose level attained by medical sensors

21. Safety Measure for User • 1st and 2nd Laws of Robotics • A robot may not injure a human being or, through inaction, allow a human being to come to harm. • A robot must obey orders given it by human beings except where such orders would conflict with the First Law • User Safety 1: Smart space should monitor users’ status and activate emergency measure should users appear to be in danger [using the mechanism forSpace Safety] • User Safety 2: Smart space should function to serve users’ preference and obey users’ order [using the mechanism for service safety] Humans need to be locked up so they cannot commit crime to harm other humans

22. Element 4: Space • Most models do not consider space itself as a critical element of pervasive computing system • Influence the decision making (context) • Big pictures on the effects of service and applications • Big pictures for user apprehension and visualization • Provide critical perspective when there are resource sharing

23. Modeling of Space Murky air Blown fuse Medium power draw Clean air Low power draw Overcooked Steak Hot Indoor Cooked Steak Warm Indoor Raw Steak Cold Indoor Context Preferred Context Impermissible Context Potential Transitions • Independent Context Manager and Context Graph • Contexts defined using standardized ontology • Categorization of preferred, transitional and impermissible contexts • Visualization in a summary-in-a-glance view on overall context of the space

24. Safety Measure for Space • Space Safety Monitor: Whenever impermissible context is detected, the space safety monitor will invoke exception handling routine, similar to interrupt/signal capture • In the worst case where there is no exception handling available, a prompt would be send to the user or the emergency contact person, so human actions can take place to handle the situation

25. Recap • Enforce Safety at Each h/w and s/w Entity in the intelligent environment • Device Safety Enforcer • Service Signal Handling Vector • Respect Users’ Wishes, and Constantly Look After Them • Look at the Big Picture, and make sure the interactions among users and entities are not harmful

26. Prior Endeavors

27. ATLAS Middleware

28. IDE: Atlas Plug-in for Eclipse

29. What does Atlas plug-in do? • Browse Available Entities and Services • Provide Information on OSGi bundles • Design by Selection • Semi-automatic Environment Setup • Integrate seamlessly with all the nice features and assistance provided by Eclipse • Deploy New Bundles Back to Smart Space

30. Context-Driven Model • Reactive Programming Model • Ontology Based context graph • Categorize contexts into desirable, transitional and impermissible context • Find the actions required to take us from currently active contexts to preferred contexts, and activate special measures when impermissible contexts are active

31. Hybrid Model

32. Epilogue

33. Past, Present and Future • Past • Establish safety-oriented programming model • Implement IDE to support basic OSGi bundle authoring • Present • Implementation of the new safety framework with safety enforcer into both the ATLAS Middleware and the IDE support • Future • Evaluating the safety feature of the intelligent environment created based on this framework

34. Conclusion • Pervasive Computing Systems are Intimate • The failure and unreliability can result in strong feeling of betrayal and frustration • Intelligent Environment Affects Numerous Aspects of the Daily Life • Ensure Safety of the System by Ensuring Safety of All Entities and the Environment as a Whole • Safety Enforcement is accomplished by the collaboration at both implementation/ compilation time as well as runtime.

35. Thank You

36. Design Philosophy • Safety Feature: • Each individual device has a device breaker to monitor abnormal access pattern and invalid values • The smart space as a whole guarded by context manager • Orthogonal Elements: • Simplify the design, implementation and maintenance of the heterogeneous and dynamic system • Dual text/object approach: • Text based implementation allows common, configurable services to be created without coding and maximum portability; • Object embodies more complicated or less-common logic to be created for specific services

37. The Path Forward • Analysis of Pervasive Computing Systems and elements of them • Design programming model and necessary abstractions to simplify design and implementation • Focuses on safety features to protect users, devices and the smart space as a whole

38. Modeling of Services Service/Device Change Event Programmed Service Service Registry Services Services Service Binded Services Devices Devices Devices Binded Service User Preference Aligned Service User Preference Change Event Active Contexts Active Context Change Event Service Behavior 1 Service Behavior 2 Service Behavior n ….

39. Pervasive Computing • “The most profound technologies are those that disappear. They weave themselves into the fabric of everyday life until they are indistinguishable from it.” -- Mark Weiser • “The creation of environments saturated with computing and communication yet gracefully integrated with human users” -- Pervasive Magazine, Inauguration Issue • “Embedding computation into the environment and everyday objects to enable people to interact with information-processing devices more naturally and casually than they currently do, and in whatever location or circumstance they find themselves.” Other alternative terms: Ubiquitous computing, calm technology, things that think, everyware -- Wikipedia

40. Goals of Proposed Programming Model • Safe • Service Oriented Architecture (SOA) • Open • Configurable/Customizable • Orthogonal Description • Easily Programmable

41. Background

42. ATLAS Architecture Programming Tool Applications Intelligent Environment (e.g. Smart Home) ATLAS Sensor/Actuator Technology

43. ATLAS Sensor Platform • The basic building block for programmable pervasive spaces. • Atlas provides • physical nodes for connecting various heterogeneous devices, • a system for translating those devices into software services, • a system for maintaining a library of device services and their interfaces, and • a runtime environment for accessing services and composing applications.

44. Middleware Atlas Developer API Context Manager Originally designed and implemented for Context-Driven Model Implemented using ontology software and inference engine Parts of implementation concerning impermissible contexts and their emergency handling routines are retained and modified Visualization will be added to provide quick snapshot view