Industrial Ontologies Group

1. Resource Agent Resource Agent Resource Agent S-APL University of Jyväskylä Industrial Ontologies Group UBIWARE: “Smart Semantic Middleware for the Internet of Things” “Expert” “Device” “Service” Vagan Terziyan ICINCO-2008 13 May 2008, Funchal, Madeira, Portugal

2. Industrial Ontologies Group Authors: • Artem Katasonov • Olena Kaykova • Oleksiy Khriyenko • Sergiy Nikitin • Vagan Terziyan URL: http://www.cs.jyu.fi/ai/OntoGroup University of Jyväskylä

3. University of Jyväskylä Kharkov National Universityof Radioelectronics UBIWARE Team:Industrial Ontologies Group 2008-2009 • Researchers • Vagan Terziyan (Head) • Olena Kaykova • Artem Katasonov • Oleksiy Khriyenko • Sergiy Nikitin • Contact Person: Timo Tiihonen • e-mails: tiihonen@it.jyu.fi vagan@cc.jyu.fi • phone: +358 14 260 2741 • Michal Szydlowski • Joonas Kesäniemi • Michal Nagy • Arnim Bleier • Nikos Mouchtaris URL:http://www.cs.jyu.fi/ai/OntoGroup

4. Three alternative trends of Web development Applications, services, agents Machines, devices, computers Human Communities Facilitates Machine-to-Machine interaction Facilitates Software-to-Software interaction Facilitates Human-to-Human interaction Semantic Web Web of Things Metadata Ubiquitous Computing Web 2.0 Smart Spaces Ontologies Wikis RFID Web Services Blogs Embedded Systems Agents Mashups Sensor Networks EAI Portals Community Portals Web

5. F X Y New integral trend of Web development Web of intelligent entities (intelligence services), browseable, searchable, composable, configurable, reusable, dynamic, mobile … Involvement of various mathematical models to be service components results to more general Web of Abstraction Facilitates Intelligence-to-Intelligence (also model-to-model) interaction Distributed AI MAS Data and Web Mining Machine Learning Knowledge Discovery "Smart Services" Web

6. “Semantic Wave” (Web X.0) We may add here: Web 5.0 will come finally and it is about connecting models in a “Global Understanding Environment” (GUN), which will be such proactive, self-managed evolutionary Semantic Web of Things, People and Abstractions where all kinds of entities can understand, interact, serve, develop and learn from each other. [Vagan Terziyan] “The semantic wave embraces four stages of internet growth: Web 1.0, was about connecting information ... Web 2.0 is about connecting people. Web 3.0, is starting now… and it is about … connecting knowledge… Web 4.0 will come later … and it is about connecting intelligences in a ubiquitous web where both people and things can reason and communicate together.” [“Semantic Wave 2008” , Mills Davis ]

7. Beyond Web 5.0 ? Human v2.0 ?! [Ray Kurzweil] 2029 Singularity Semantic Wave Nanotech http://www.youtube.com/watch?v=BywCMkbG-Jg

8. Overall Goal • Based on combination of Semantic and Agent technologies, we aim at designing a new generation middleware platform (UBIWARE) which will support and essentially simplify design, implementation and operation of self-managed, complex, flexible and extendable industrial systems (e.g. information-, business intelligence-, expert-, condition monitoring-, diagnostics-, maintenance-, etc. systems) consisting of mobile, distributed, heterogeneous, self-descriptive, self-aware, shared, reusable and configurable components of different nature.

9. What is UBIWARE ? (1) • UBIWARE is a generic, domain independent middleware platform, which is meant to be able to provide the following support: • integration; • interoperability; • proactivity; • communication, observation, negotiation, coordination and collaboration; • automation, design and installation; • lifecycle management, execution monitoring, diagnostics, maintenance; • self-descriptiveness, semantic search, discovery, sharing, reuse; • planning and decision-making; • adaptation; • learning, mining, knowledge discovery; • context-awareness; • self-management including self-configuration; • security, privacy and trust; • etc... • …for … (see next slide)

10. What is UBIWARE ? (2) • … for the following resources, systems and components (including their groups): • data information and knowledge: data, metadata, knowledge, logic, ontologies; • software and services: software components, software agents, software and information systems, services including Web-services; • humans: users, operators, experts, administration, customers, patients, doctors, etc; • hardware: machines, devices, networks, embedded electronics, RFID; • organizations; • intangibles: human and organizational capital, innovations, property rights, trust and reputation, brand recognition, etc.; • processes: behaviors, technologies and business models; • interfaces; • intelligence: reasoning, inference, planning, learning, data-mining, knowledge discovery, etc… engines; • ecosystems: environments, smart spaces, other middleware and CSCW tools; • abstractions and mathematical models; • etc.

11. What is UBIWARE ? (3) • Due to heterogeneity of provided services and supported components, UBIWARE is based on integration of several technologies: Semantic Web, Distributed Artificial Intelligence and Agent Technologies, Ubiquitous Computing, SOA (Service-Oriented Architecture), Web X.0, P2P and related concepts. • The research and design on UBIWARE is started by Industrial Ontologies Group within UBIWARE project: “Smart Semantic Middleware for Ubiquitous Computing” (June 2007 – May 2010) funded by Tekes and industrial companies. • Project web page:http://www.cs.jyu.fi/ai/OntoGroup/UBIWARE_details.htm

12. What is UBIWARE (in short) • UBIWARE is a tool to support: • design and installation of…, • autonomic operation of… and • interoperability among… • … complex, heterogeneous, open, dynamic and self-configurable distributed industrial systems;… • … and to provide following services for system components: • adaptation; • automation; • centralized or P2P organization; • coordination, collaboration, interoperability and negotiation; • self-awareness, communication and observation; • data and process integration; • (semantic) discovery, sharing and reuse.

13. Why Semantic Web? (Ora Lassila) Semantic Web is important for UBIWARE just because UBIWARE is meant to handle also new problems, which may appear later

14. Why Agents? • Growing complexity of computer systems and networks used in industry  need for new approaches to manage and control them • IBM vision: Autonomic computing – Self-Management (includes self-configuration, self-optimization, self-protection, self-healing) • Ubiquitous computing, “Internet of Things”  huge numbers of heterogeneous devices are interconnected • “nightmare of pervasive computing” when almost impossible to centrally manage the complexity of interactions, neither even to anticipate and design it. • We believe that self-manageability of a complex system requires its components to be autonomous themselves, i.e. be realised as agents. • Agent-based approach to SE is also considered to be facilitating the design of complex systems

15. GUN Concept GUN – Global Understanding eNvironment GUN = Global Environment + Global Understanding = Proactive Self-Managed Semantic Web of Things = (we believe) = “Killer Application” for Semantic Web Technology

16. Human-to-Human Human-to-Machine Agent-to-Agent Machine-to-Human Machine-to-Machine GUN and Ubiquitous Society GUN can be considered as a kind of Ubiquitous Eco-System for Ubiquitous Society – the world in which people and other intelligent entities (ubiquitous devices, agents, etc) “live” together and have equal opportunities (specified by policies) in mutual understanding, mutual service provisioning and mutual usability.

17. S-APL Challenge 1: General Adaptation Framework RDF-Based Semantic Agent Programming Language Universal reusable semantically-configurable adapters

18. S-APL Challenge 2: General Proactivity Framework Role “Feeder” description Role “SCADA” description Role “Maintenance worker” description Universal reusable semantically-configurable behaviors

19. S-APL Challenge 3: General Networking Framework Scenario “Predictive maintenance” description Scenario “Data integration” description Universal reusable semantically-configurable scenarios for business processes

20. UBIWARE Subgoals • Core DAI platform design (UbiCore); • Policy-Based Control of MAS (PBC) • Managing Distributed Resource Histories (UbiBlog); • Self-Management, Configurability and Integration (COIN); • Smart Interfaces: Context-aware GUI for Integrated Data (4i technology); • Industrial cases and appropriate prototypes.

21. 1:Core DAI Platform Design (UbiCore) • The core platform should provide means for building systems that are flexible and consist of heterogeneous autonomous components, yet predictable in operation. • A major challenge: a semantic/ontological approach to coordination - to enable the components to communicate their intentions with respect to future activities and resource utilization and to reason about the actions, plans, and knowledge of each other, in real time.

22. S-APL S-APL UBIWARE Platform Architecture Behavior Engine .class Script: Configuration Settings Assign Settings Activity Beliefs Log Storage Script: Policy Constraints Live Activity Script: Role-Based Behavior Rules .class Activity Activity Activity Activity Activity Activity Activity Activity Everything is Belief !!!

23. Layered Agent Architecture

24. “Soul-Mind-Body-Genome-Ontonut” Agent Architecture Added 2 September 2008 by Terziyan Vagan SoftSoul “Life” Behavior HardSoul Meta-Beliefs (preferences) Shared Meta-Beliefs SoftMind RBE – Reusable Behavior Engine Shared RBEs RBE RBE RBE RBE HardMind Beliefs (facts, rules, policies, plans) Configuration(GENOME) Ontobilities Shared Beliefs SoftBody RAB – Reusable Atomic Behavior Shared RABs RAB RAB RAB RAB HardBody Hardware Shared Hardware “Visible” to other agents Environment May be an agent

25. What is Environment ? Added 2 September 2008 by Terziyan Vagan Environment • Environment of an agent is the remaining part of the UBIWARE-supported world (physical or virtual) if to exclude the agent itself (with its soul, mind, body, etc). • Main groups of entities in the environment of the agent: • Entities to which the agent provides services; • Entities from which the agent consumes services; • Conflicting entities, or other agents, which share entities of described above categories: the same service providers or service consumers; • Any combination of the above.

26. Environment: Service Consumers Added 2 September 2008 by Terziyan Vagan SC Environment: Service Consumers • Service consumers (SC) within agent environment are the entities to which the agent provides services. • Main groups of service consumers of the agent: • Entities (devices, humans, software, other agents, etc.) under the agent monitoring (supervision, control, diagnostics, maintenance, etc.); • Entities (devices, humans, software, other agents, etc.) to which the agent provides needed information (pull or push); • Entities (devices, humans, software, other agents, etc.) to which the agent provides assistance in their operation (pull or push); • Entities (devices, humans, software, other agents, etc.) to which the agent committed to serve like an instrument (“slave”) in their operation and can be anytime configured and fully controlled by them; • Any combination of the above. Monitoring Information Assistance Instrument

27. Environment: Service Providers Added 2 September 2008 by Terziyan Vagan SP Environment: Service Providers • Service providers (SP) within agent environment are the entities, which provide services to the agent. • Main groups of service providers of the agent: • Entities (humans or other agents) which monitor the behavior of the agent (supervision, policy-based control, reconfiguration, etc.); • Entities (devices, humans, software, databases, other agents, etc.), which provide needed information (pull or push) to the agent; • Entities (devices, humans, software, other agents, etc.) which provide assistance to the agent in its operation (pull or push); • Entities (devices, software, other agents, etc.) which the agent can use as instruments in service provisioning (pull) and which are fully committed for that (aka “slaves”) and are under full control by the agent while used; • Any combination of the above. Monitoring Information Assistance Instrument

28. On Ontonuts in UBIWARE:Ontonuts as agent-driven proactive service capabilities(“Ontobilities”) Added by Vagan Terziyan 17 September, 2008

29. UBIWARE Agent: Possible Future Architecture RBE – Reusable Behavior Engine SoftSoul “Life” Behavior HardSoul RAB – Reusable Atomic Behavior Meta-Beliefs (preferences) Shared Meta-Beliefs SoftMind Ontobilityis self-contained, self-described, semantically marked-up proactive agent capability (agent-driven ontonut), which can be “seen”, discovered, exchanged, composed and “executed” (internally or remotely) across the agent platform in a task-driven way and which can perform social utility-based behavior Shared RBEs RBE RBE RBE RBE HardMind Beliefs (facts, rules, policies, plans) Configuration(GENOME) Ontobilities Shared Beliefs SoftBody Shared RABs RAB RAB RAB RAB HardBody Hardware Shared Hardware Genomeis part of semantically marked-up agent configuration settings, which can serve as a tool for agent evolution: inheritance crossover and mutation “Visible” to other agents through observation Environment May be an agent

30. Ontonuts: Competence Profile of an Agent as a service provider (“what can I do” and “what can I answer”) and appropriate service plan (“how I do … or answer …”) You can ask me for … • … action • … information ontonut

31. External view to ontonuts: Shared Competence Specification You can ask me for … External Internal • I can open the door #456 • I can fly • I can use knifes • I can build house from wood • I can visualize maps • I can grant access to folder “444” • I know everything about Mary • I know everything about cats • I know what time it is now • I know all lovers of John • I know grades on chemistry of all pupils from 4-B We consider ONTONUTS to be shared S-APL specifications of these competences

32. Internal view to ontonuts: Action or Query Plans You can ask me for … External Internal • I can open the door #456 • S-APL plan of opening the door #456 • I know everything about Mary • S-APL plan of querying either own beliefs or external database about Mary We consider ONTONUTS to be also an internal plans to execute competences

33. Possible general rule of ontonut appearance You can ask me for … External Internal IF I have the plan how to perform certain complex or simple action or the plan how to answer complex or simple query AND {time-to-time execution of the plan is part of my duty according to my role (commitment) OR I am often asked by others to execute action or query according to this plan} THEN I will create ONTONUT which will make my competence on this plan explicit and visible to others

34. Example (1): Atomic Ontonut #1 I can answer any queries on mental diseases of citizens of X 007 003 Give me the list of women from X with mental diseases diagnosed after 2006 #1 I know how appropriate database is organized, I have access rights and I am able to query it SQL City X Central Hospital Relational Database

35. Example (2): Atomic Ontonut #2 I can answer any queries on loans in Nordea bank 007 005 Give me the list of Nordea clients with loans of more than 100 000 EURO #2 I know how appropriate database is organized, I have access rights and I am able to query it XQuery Nordea XML Database

36. Example (3): Complex Ontonut #3 I know how to split query to two components; I know to whom I can send component queries (I have contracts with them); and I know how to integrate outcomes of these queries I can answer any queries on mental diseases and loans of Nordea bank clients from X 001 007 Give me the list of Nordea clients from X with loans of more than 200 000 EURO and who has more than 2 mental disorders during last 5 years #3

37. Summary: Ontonut is something similar to OWL-S Semantic Web Service description (i.e. combination of Profile, Model, Grounding) OWL-S • A Service is a kind-of Resourcein the Web, i.e. some Web resources provide services. • What does the service require of the user, or other agents, and provides for them? The answer to this question is in ServiceProfile • How does it work? • The answer to this question is in ServiceModel • How is it used? • The answer to this question is in ServiceGrounding.

38. A Service as well as an Ontonut provides some Function G X Service Model Service Grounding F Service Profile Y

39. An Ontonut as well as a Service provides some Function Service Profile External Service Grounding Internal Service Model

40. Service Profile, Model and Grounding example: “Cinema Cashier” x1: movie_name; x2: time; x3: number_of_tickets; x4: seats preference; x5: money G X 1: takes x1, x2, x3, x4; 2: checks availability of x3 tickets for the x1movie, at x2 time, which suits x4 constraint ; 3: finds one_ticket_prise from the price list; 4: calculates price for x3 tickets: price = one_seet_price * x3; 5: takes x5; 6: calculates y2 ( y2 = x5 – price ); 7: gives y1, y2. F 1: cinema address; 2: cinema movie schedule; 3: cinema cash-desk location; 4: nock to the cash-desk window and, when it opens, make your order (X) Y y1: movie tickets; y2: change

41. On Ontonuts in UBIWARE:Ontonuts as distributed querying capability Added by Sergiy Nikitin 17 September, 2008

42. Agent Beliefs Original view on Ontonuts (distributed querying) Data Service Files Ontonut Bindings Ontonuts Role Script DB/KB agent-to-agent servicing adaptation of external sources

43. Excel sheet CSV file RDBMS Main components of Ontonuts architecture Business Logic Script Ontonuts bindings Agent Beliefs (S-APL code) Ontonuts triggering rule Ontonuts Role Script Query Planner Query Plan Executor SQLReader TextTableReader ExcelReader … MessageSender MessageReceiver Reusable Atomic Behaviors (Java code) QueryAnalyser QueryPlanner … Data Service

44. Ontonuts example code :DiaryCommentNut rdf:type di:DBOntoNut. :DiaryCommentNut :hasDataSource :datasourceid. :DiaryCommentNut :hasSQLQuery "SELECT CommentID, EntryID, Title FROM dbo.Comment ". :DiaryCommentNut :hasMapping { :commentID di:mapsTo "CommentID". :entryId di:mapsTo "EntryID". :commentTitle di:mapsTo "Title"}. :DiaryCommentNut :hasTransformationScript { {* :table {?rowid :row { CommentID :column ?commentid. EntryID :column ?entryid. Title :column ?title} } }=>{ :DiaryComment :instance { ?rowid rdf:type :DiaryComment. ?rowid :commentID ?commentid. ?rowid :entryId ?entryid. ?rowid :commentTitle ?title} }. }. :DiaryCommentNut :hasDataPattern { ?rowid rdf:type :DiaryComment. ?rowid :commentID ?commentid. ?rowid :entryId ?entryid. ?rowid :commentTitle ?title}.

45. Querying usecases {?diaryEvent :hasCommentText ?ctext. ?diaryEvent :hasTime ?ctime. ?ctime > ?timestart. ?ctime < ?timeend. ?alarmHistorian :hasAlarm ?alarm. ?alarm :hasTime ?atime. ?atime > ?timestart. ?atime < ?timeend. ?timestart = 2008.09.08T12:00.00. ?timeend = 2008.09.08T23:59.00. } • Event flow integration • Time-based distributed query to different sources (extract events from different systems by filtering them with the same time frame) • Additional information • Get supplementary device configuration data for a particular event-based view • Complex mining • Collect information from a set of sources, where inputs of subqueries are dependent on outputs of the preceding subqueries • Combination of all cases mentioned above {?diaryEvent :hasCommentText ?ctext. ?diaryEvent :hasTime ?ctime. ?ctime > 2008.09.08T12:00.00. ?ctime < 2008.09.08T23:59.00. ?diaryEvent :hasTag ?eventtag. ?eventtag :hasMappingTo ?node ?dpm :hasNode ?node. ?node :hasAlarmLimit ?alimit } {?dpm :hasNode ?node. ?node :hasPerfIndex ?pindex. ?pindex < 0.5 . ?node :hasMappingTo ?eventTag. ?diaryEvent :hasTag ?eventTag. ?diaryEvent :hasComment ?comment }

46. Query types • Parallel • subqueries can be executed independently from each other and results are merged • IOMIO (sequential) • Inputs are dependent on the outputs from subqueries to other sources • Hybrid (a combination of two above) • Possible cases: • Results of two or more parallel subqueries are merged and used as an input for a subsequent subquery • Result of one subquery is used as an input to a set of parallel subqueries • Result of a subquery is used as an input for both subsequent and non-susequent, parallel and non-parallel subqueries

47. Ontonuts: a mechanism for provision of dynamic information • An analog of platform-embedded constructs like: • sapl:Now sapl:is ?time (gets current system time) • But can be flexibly (re-)defined by user • fingrid:CurrentVoltage sapl:is ?voltage • metso:CurrentOilLevel sapl:is ?oillevel • innow:CurrentUsersOnline sapl:is ?usersonline • The approach simplifies the implementation of the agent’s business logic by introducing computable elements. The values of these elements are computed on-demand (only when a query appears in agent’s beliefs)

48. Ontonuts: a mechanism for provision of dynamic information(2) • When extended to more abstract level, computable values can be applied for: • counting statistics over dynamically updated data (e.g. average alarm rate per day, or number of students at the lecture now) • collecting dynamic information about others (e.g. request “what is John’s location at the moment” would look like: • “:John :currentLocation ?location”)

49. Ontonuts vs. Softbody (Environment) • An agent can seamlessly read the information from the environment in a similar manner as from the Ontonut, the observable properties in the environment can provide different kind of information, that directly goes to agent’s beliefs, however, environment represents common and most frequently used properties, whereas agent may need to perform its own specific calculations. The complexity of the calculations will be hidden behind Ontonut. End of Sergiy Nikitin chapter

50. “Soul-Mind-Body-Genome-Ontonut”: Use of S-APL provides new opportunities Added 2 September 2008 by Terziyan Vagan • Mobility of (Souls, Minds, Bodies, Genomes, Ontonuts); • Integration of (Souls, Minds, Bodies, Genomes, Ontonuts); • Semantic Search and Querying of (Souls, Minds, Bodies, Genomes, Ontonuts); • Reasoning based on (Souls, Minds, Bodies, Genomes, Ontonuts); • Configurability of (Souls, Minds, Bodies, Genomes, Ontonuts); • Inheritance, crossover, mutation, evolution, etc. of (Souls, Minds, Bodies, Genomes, Ontonuts); • Learning of (Souls, Minds, Bodies, Genomes, Ontonuts); • Compilation of Minds, Agents towards executable ones.