UBIWARE Project

1. Resource Agent Resource Agent Resource Agent PI GB SC Industrial Ontologies Group “Smart Semantic Middleware for Ubiquitous Computing” Deliverable 3.1 UBIWARE Project “Expert” “Device” “Service” & shared services University of Jyväskylä

2. Industrial Ontologies Group UBIWARE Team University of Jyväskylä Researchers • Vagan Terziyan (Head) • Olena Kaykova • Oleksiy Khriyenko • Sergiy Nikitin • Michal Nagy Contact Person: Timo Tiihonen • e-mails: • timo.tiihonen@jyu.fi • vagan.terziyan@jyu.fi • phone: +358 14 260 2741 • Artem Katasonov • Michal Szydlowski • Joonas Kesäniemi • Michael Cochez URL:http://www.mit.jyu.fi/ai/OntoGroup

3. UBIWARE Workpackages

4. Project Workpackages • Core Distributed AI platform design(UbiCore); • Managing Distributed Resource Histories (UbiBlog); • Smart Ubiquitous Resource Privacy and Security (SURPAS); • Self-Management, Configurability and Integration (COIN); • Smart Interfaces: Context-aware GUI for Integrated Data (4i technology); • Middleware for Peer-to-Peer Discovery (MP2P); • Industrial cases and prototypes.

5. (2007-2010) “Smart Semantic Middleware for Ubiquitous Computing” UbiCore – CORE DISTRIBUTED AI PLATFORM DESIGN UbiCore – CORE DISTRIBUTED AI PLATFORM DESIGN UBIWARE Project DELIVERABLE D3.1 Workpackage WP1 TASK T3.1_w1 Workpackage leader Sergiy Nikitin

6. WP1 Tasks • How to enable agents to flexibly discover each other, based both on the roles played and on particular capabilities possessed. • What would be concrete benefits of and what mechanisms are needed for accessing and using a role’s script by agents who are not playing that role but wish to coordinate or interact with an agent that does? The WP tasks for the Year 3 are the following: Task T3.1_w1 (research):Answers to the questions above, and other, if appear in the process, related to the agents’ semantic interaction based on each other’s behavior models. Design of the core inter-agent discovery and coordination mechanisms. Task T3.2_w1 (development):Incorporating the research findings to the UBIWARE prototype.

7. A reminder: what is Ontonut? • What is rule? • {A A ?a} => {C C ?a}. (produces effect from the input given) • What is capability? • {B B ?b} C1 {C C ?b}. (produces effect from the input given) • What is the difference? • NO essential difference except the fact that logic of the rule is explicit, whereas capability has internal (hidden) processing logic • Ontonut is a capability that allows free form of implementation • UBIWARE provides a toolset for several typical predefined Ontonut types (e.g. Donuts – for database connectivity)

8. Big board Ontonut DF Platform DF Billboard Local Ontonut DF Local Ontonut DF Local Ontonut DF Direct marketing How to advertise Ontonuts? • Wrap them as services • Distribute service descriptions to a set of possible clients

9. Service contract Service definitions and obligations Parties Functional requirements with test cases Service provider Unique company ID (e.g. digital certificate) Service consumer Security constraints Contract version Quality of Service and Service Level Agreement Address of the entity to be contacted in case of: Provider contact entity Transaction support • Expiry of the current contract • Proposal to change service contract • Problems with service consumption • etc. Wrapping Ontonut as a service • ServiceNut wrapper • Access mode (public or authorized) • Publishing mode (Direct marketing, Big board or Billboard) • Access point (unique agent name and Ontonut name within the agent) • Contracting (QoS, SLAs, pricing, payment methods, etc.)

10. Publishing Ontonut DF I want { You publish {I haveService { accessMode is public. serviceURI is www.iog.jyu.fi/service/id1. //Access point input is {semantic pattern}. output is {semantic pattern}. … }} }

11. Ontonut DF Ontonut DF I want You subscribeMe { ?agent hasService { serviceURI is ?uri. … } } Discovery and subscription I want You answer { ?agent hasService { input is {pattern}. output is {pattern}. serviceURI is ?uri. abstractServiceURI is ?auri. serviceCategory is ?category. … } }

12. WP1: A framework for servicing Standard Web Service stack UBIWARE stack Process Management protocol MOWS (OASIS) SACL Semantic Agent Configuration Language S-APL subsets Process Description protocol Semantic Process Execution Language BPEL, WS-CDL SPEL Service Discovery protocol Ontonuts Directory Facilitator UDDI ODF Service Description protocol WSDL Ontonuts Messaging protocol SOAP FIPA ACL Service Transport protocol HTTP, SMTP, etc. HTTP

13. WP1: Conclusions • Agent component servicing through • Externalization of capabilities (Ontonuts) • Adding missing features (accessibility, contracting, etc.) • Advertising schemes • Directory Facilitator, Environment, P2P • Subscription mechanisms • to ensure service availability • Future development directions • A framework for process management • Protocols of interaction • Process configurability

14. (2007-2010) “Smart Semantic Middleware for Ubiquitous Computing” UbiBlog – MANAGING DISTRIBUTED RESOURCE HISTORIES UbiBlog – MANAGING DISTRIBUTED RESOURCE HISTORIES UBIWARE Project DELIVERABLE D3.1 Workpackage WP2 TASK T3.1_w2 Workpackage leader Sergiy Nikitin

15. WP2 - Tasks Third year phase of the WP2 (Mining phase) as it was specified before would rather be an application case of the Ontonuts technology than a research topic. With respect to this fact we have decided to concentrate our efforts towards the fusion of the WP1 and WP2 3rd year objectives. Shortly, the WP1 objective for the 3rd year is to elaborate a mechanism for advertising of agent capabilities and role scripts (complex capabilities) amongst agents, whereas WP2 research targets data mining capabilities only. Therefore, it is reasonable to integrate WP1 and WP2 under one umbrella of Ontonuts technology which is targeting even more ambitious goal – to enable automated (re)planning and execution of semantically annotated agent actions including distributed data querying, data mining as well as distributed agent-to-agent servicing.

16. Agent service Input Model Output Vector DM result DM model Data Mining as a service in UBIWARE • Data mining applications are capabilities • Agents can wrap them as services • PMML language - a standard for DM-model representations • Data Mining Group. PMML version 4.0. URL http://www.dmg.org/pmml-v4-0.html

17. Data Mining service Mining method Model construction service Computational service Supervised Learning Unsupervised learning Neural networks Fixed model service Model player service Clustering kNN Industry Electrical Engineering Process Industry Paper industry Power networks Power plant Data Mining as a service in UBIWARE Ontology construction Data mining domain Core DM service ontology Problem domain

18. Input Model Output Vector class of V1 is: “Urgent Alarm” according to model M1 Vector to be classified: alarm message: V1={0.785, High, node_23} Paper machine alarms classifier neural network model (M1) Inputs Model Outputs Set up a model M1 Model player Model M1 assigned Paper machine alarms classifier neural network model (M1) Vector class of V1 is: “Urgent Alarm” according to model M1 Vector to be classified: alarm message: V1={0.785, High, node_23} Input Model Output Learning samples and the desired model settings Model constructor Model M1 parameters Data mining service types Fixed model service Model player service Model construction service

19. A use case for data mining service stack • A “Web of Intelligence” case: Input Model Output Distributed query planning and execution Pattern of learning data to be collected: ?V={?p1, ?p2, ?p3} A set of learning samples (vectors) 1 Learning samples and the desired model settings Model constructor Model M1 parameters 2 Set up a model M1 Model player Model M1 assigned 3 Paper machine alarms classifier neural network model (M1) Vector class of V1 is: “Urgent Alarm” according to model M1 Vector to be classified: alarm message: V1={0.785, High, node_23} 4

20. Example application: Data Mining in Cloud UBIWARE for control and management in cloud Semantic Business Scenarios DM model wrapped as a service for paper industry Domain components as services Domain model (Ontology) & components DM model for paper industry Cross-domain Middleware components Componentization & Servicing Cross-layer configuration & management mechanisms Data Mining service player Connectors, Adapters RABs, Scripts UBIWARE in cloud computing stack Cloud computing stack Application as a Service Applications and Software as a Service Application (business logic) Services (Payment, Identity, Search) Platform as a service Solution stack (Java, PHP, Python, .NET) Structured storage (e.g. databases) Raw data storage and network OS-virtualization Infrastructure as a service Virtualization Machine Hardware configuration Technologies in cloud

21. (2007-2010) “Smart Semantic Middleware for Ubiquitous Computing” SELF-MANAGEMENT AND CONFIGURABILITY SELF-MANAGEMENT AND CONFIGURABILITY UBIWARE Project DELIVERABLE D3.1 Workpackage WP4 TASK T3.1_w4 Workpackage leader Michal Nagy

22. WP4 - Tasks • How can we find suitable agents to perform the task based on the goal specified by the user? • How can we transform a goal into a set of scripts to be executed by these agents? • Once the system is configured, how can we maintain this state even if the circumstances of the system change? The WP tasks for the Year 3 are the following: Task T3.1_w4 (research):Answer the questions above and other related questions that may arise during the process. Find the limitations of the proposed approach . Task T3.2_w4 (development):Incorporating the research findings to the UBIWARE prototype.

23. The environment List of abstract process descriptions Manager of a process Ontonut phone book (Yellow pages) Dictionary

24. Initial configuration Pre-configuration Send flowers to somebody Send goal ?x rdf:type c:ThingSent . ?x c:sentTo ?y . ?x c:sentWhat ?z . ?y rdf:type c:Human . ?z rdf:type c:Flower Search for process Process returned

25. Initial configuration Process configuration Process description: • Steps: • Find address • Send flowers • Input: • Name • Surname • Flower type • Output: • Flowers sent to that person Steps: • Find suitable Ontonuts • Specify the inputs • Create an executable process

26. Abstract and executable process Abstract process Executable process

27. Runtime self-configuration 3 layer architecture • Goal management • Planning • Change management • Rebinding • Partial plan execution • Component control • Monitoring and status reporting

28. Runtime self-configuration Rebinding

29. Runtime self-configuration Replanning

30. Conclusion and future work Conclusion • Initial configuration • Runtime self-configuration based on a 3 layer architecture Next steps • Implement the environment (Director agent, Abstract process repository, etc.) • Implement 3 layer architecture for self-management

31. (2007-2010) “Smart Semantic Middleware for Ubiquitous Computing” GENERAL VISION OF 4I TECHNOLOGY AND ITS APPLICATION IN UBIWARE GENERAL VISION OF 4I TECHNOLOGY AND ITS APPLICATION IN UBIWARE UBIWARE Project DELIVERABLE D3.1 Workpackage WP5 TASK T3.1_w5 Workpackage leader Oleksiy Khriyenko 4i (FOR EYE)TECHNOLOGY Intelligent Interface for Integrated Information

32. WP5 - Tasks • What should be the general architecture of the product – 4I(FOR EYE) tool package so that it will be possible to build and further extend a different services based on the product? What are the requirements for the product, for the product components, what are the necessary modifications and the use cases of the product utilization? • What are the commercialization and marketing steps? The WP tasks for the Year 3 are the following: Task T3.1_w5 (research):Answer to the questions above. Design of a 4I(FOR EYE) tool package and utilization business models. Task T3.2_w5 (development):Incorporating the research findings to the UBIWARE prototype.

33. General architecture of the browser General architecture of the browser.GUI-Shell has client-server architecture and represented by frontend web-page (HTML, JavaScript) and backend server part (Apache Tomcat, Java) of the system. MetaProvider is a kind of a service that has client-server architecture and represented by frontend JSP page and backend server part (Apache Tomcat, Java).

34. To become a product To become a product , prototype should pass through a number of stages that bring generalization to the prototype (ability to be used in different domains and arias), provide common information infrastructure and user/programming interfaces of the system to allow extension and configuration of it. • Common vocabulary.So far we considered two ways of Browser usage: • first one is a kind of local system, where all the parts of the system are developed by the same group of specialists. It means that ontology will be built step by step with a new task appearance. • another one is open environment where different parts of the system are developed and provided by different parties. Then we need one common ontology or have to provide mechanism for adaptation and bridging of different ontologies, because we could not avoid appearance of sub-systems (former local systems) that would like to interact with each other and provide its knowledge and experience as external services. Source data adaptation.To make Browser applicable for different tasks and systems we have to elaborate functionality that enables Browser to convert data from any format to the required one. GUI-Shell should provide user interface to select external repository to be imported via appropriate adaptation sub-module from a list of available in Browser as well as interface for search of new sub-modules and their imbedding.

35. To become a product Manipulations with Context and MetaProviders.GUI-Shell should have interfaces and functionality for Context and MetaProvider descriptions exchange and editing. Definition of new context will be accompanied not only by correspondent description, but also by JavaScript code, correspondent functional server part and appropriate html, media and other files. Then extension of the Browser with new context will be conducted via installation of certain add-on package. Extension of the Browser with new MetaProvider is much simple. We just have to extend internal repository of MetaProvider’s descriptions with a new one. MetaProvider description is standardized. It describes address of the service, in- and output-parameters.

36. Commercialization steps General business model of 4I Environment … Scenario 1: Global Use of the Browser Scenario 2: Local Corporate Use of the Browser • Player 1:Semantically-enhanced Search System that plays role of Resource Information Holder. • Player 2: plays role of Holder of the Browser as well as role of Context Generator & Provider and MetaProvider Registry Holder. In case of Global Use of the Browser all these roles can be played by Browser Provider. • Player 3:is a list of parties that play roles of Providers of MetaProviders. The most probable case when the role of Ontology Holder is shared by Player 1 and Player 2 and Player 3 just follows this ontology. The most promising case is when Player1 and Player 2 is one merged Player and autocratically takes care of ontology. • Player 1:is a Browser Provider that provides whole infrastructure as a full-fledged system in one package with all necessary specifications and supportive documentation. • Player 2: plays role of Holder of the Browser, should provide access to a repository of semantically annotated resources as a Resource Information Holder, and play the roles of Context Generator & Provider and MetaProvider Registry Holder as well. Due to domain specifics, it should play a role of Providers of MetaProviders also. • Player 3:is a list of parties that play roles of Providers of MetaProviders. With further evolution of the scenario, some new Players can emerge: those who will take care about common ontology or bridging technology for already existing, those who will be responsible for common Registry of MetaProvider and common context creation and provisioning.

37. Enhancement of the Browser As a next valuable enhancement of the Browser , we consider intelligent way of automatic/semiautomatic context recognition and personalized visualization invocation. To provide automatic context generation, we should have a model that represents a context and can be used by machine learning algorithms to automate the process… • context is a filter of resource representation in certain situation; • resource is characterized by set of properties/attributes as well as other relevant to the resource resources in particular context . Model of the context in this case is a set of coefficients/weights that shows correspondence of the attributes, relevance of them in certain sense:

38. Strategy 1: As a simple case, we can consider State of the User as a part of State of the Environment. Then we will build the model of influence on a vector of weights . Unfortunately such approach does not give us an ideal model, because a user affects an influence of environment on model (in other words, user can be a contextual entity with indirect influence on the model). Enhancement of the Browser Strategy 2: Here we are taking into account an influence of the user on a model and consider State of the User as a separate set of contextual information (let us say – meta-level context). Thus, at the first stage we have to find correlation between State of the User and State of the Environment and then build two-level influence model. Strategy 3: Probably the most correct approach is a combination of previous two. We have to consider State of the User as a part of State of the Environment and at the same time use it as a meta-context to find the most relevant contextual information (data) for resource visualization depending on the User context.

39. Conclusions and future work To become a product… ontology creation No doubts that the best way is to develop common ontology. But, practically, more realistic way is to start from building local domain and task-specific ontologies and further apply adaptation and bridging technologies to allow interoperability between systems based on different ontologies. Current prototype does not have separate ontology as such, and uses some limited (task specific) imbedded one. Thus, ontology/vocabulary creation and manipulation mechanism is one of the challenges that has to be taken into account. source data adaptation GUI-Shell should provide user interface to select external repository to be imported via appropriate adaptation module from a list of available in Browser as well as interface for search of new adaptation modules and their imbedding. Regarding to the plan for Inno-W industrial case, we are going to elaborate adaptation module to convert their RDF repository into internal format. It will be just one adaptation module, but we are going to implement general functionality of the Browser according to modular approach of source data adaptation. manipulations with Context and MetaProviders The same modular approach also can by apply for resource visualization context definition and creation. Definition of new context should be accompanied not only by correspondent description, but also by JavaScript code, correspondent functional server part and appropriate html, media and other files. Then extension of the Browser with new context will be conducted via installation of add-on package.

40. Conclusions and future work Commercialization steps of the Browser… We have considered general model of 4I Environment where we tried to define main players and roles. We described two business scenarios of Browser utilization: Global Use of the Browser and Local Corporate Use of the Browser. So, as in a case of ontology creation we think that the easiest way to start with the second scenario, but this scenario can evolve to a Web of separated Corporate 4I Environments later. In this case we will face a problem of interoperability of the systems especially on the level of ontology specification, resource annotation and context creation. Thus, with further evolution of the scenario we will come to the first scenario with more complex structure. Still, it is more realistic scenario, but of course, the best scenario is the first one, even if it demands more efforts and investments. Enhancement of the Browser … Finally, we have presented intelligent way of automatic/semiautomatic context recognition and personalized visualization invocation as a next valuable enhancement of the Browser. One of the challenges that we can face elaborating such system functionality is how to collect State of the User and State of the Environment. Depending on domain and arias of system usage amount of the contextual resources, their properties can vary. But the algorithm can be elaborated more generally to fit any size of the data sets. This is a first draft of idea and should be elaborated more comprehensive in the future...

41. UBIWARE Project Budget Balance (2007-2009) and planned (2009-2010)

42. UBIWARE Project Budget Balance:

43. UBIWARE Modifications in 3rd Year Project Plan (2009-2010)

44. Changes in the Year 3 Project Plan With the Tekes decision to leave planned (by Tekes) amount of money in the project budget, it was decided to come back to the workpackages WP3(Policy Management Engine in UBIWARE) and WP6(MetaUBIWARE: Global Enterprise Resource Integration) with some modifications. Due to the fact that these two packages were returned to the project after Deliverable D3.1 (research part), we decided to perform correspondent research simultaneously with development part and present the results of design phase and development status during the second checkpoint and include them to the Deliverable 3.2. The project year will have three checkpoints, at the end of October 2009, March 2010, and August 2010, correspondingly. At the first checkpoint, the deliverable D3.1 is to be completed, at the second checkpoint, D3.2, and at the third checkpoint, D3.3. Working seminars at companies will be organized in September and in the 2nd half of February or 1st half of March.

45. Next Meetings • 27th October 2009 Checkpoint 1: D3.1 • end of March 2010 Checkpoint 2: D3.2 • end of August 2010 Checkpoint 3: D3.3 Company visits:xxx,xxx,…

46. Obtain More Information about UBIWARE from: Head of UBIWARE Industrial Consortium (Steering Committee Head) Dr. Jouni Pyötsiä, Metso Automation Oy. Jouni.Pyotsia@metso.com , Tel.: 040-548-3544 UBIWARE Contact Person Prof. Timo Tiihonen, Vice-Rector, University of Jyväskylä timo.tiihonen@jyu.fi , Tel.: 014-260-2741 UBIWARE Project Leader Prof. Vagan Terziyan, Agora Center, University of Jyväskylä vagan.terziyan@jyu.fi , Tel.: 014-260-4618 Project URL: http://www.cs.jyu.fi/ai/OntoGroup/UBIWARE_details.htm