Mastering Intelligent Clouds

1. Mastering Intelligent Clouds Engineering Intelligent Data Processing Services in the Cloud Sergiy Nikitin, Industrial Ontologies Group, University of Jyväskylä, Finland Presented at ICINCO 2010 conference Funchal, Madeira

2. Contents • Background on Cloud Computing • Extending cloud computing stack • UBIWARE platform • Data Mining services in the Cloud • Conclusions

3. Cloud Computing: already on the market • SalesForce.com (SFDC) • NetSuite • Oracle • IBM • Microsoft • Amazon EC2 • Google • etc. • (for a complete survey see Rimal et al., 2009)

4. Cloud Computing stack Cloud computing stack Application as a Service SaaS Application (business logic) Services (Payment, Identity, Search) Solution stack (Java, PHP, Python, .NET) What add-value can we offer to the PaaS level? PaaS Structured storage (e.g. databases) Raw data storage and network OS-virtualization IaaS Virtualization Machine Hardware configuration

5. Autonomic Computing • A vision introduced by IBM in 2003 (Kephart et al.) • software components get a certain degree of self-awareness • self-manageable components, able to “run themselves” • Why? • To decrease the overall complexity of large systems • To avoid a “nightmare of ubiquitous computing” – an unprecedented level of complexity of information systems due to: • drastic growth of data volumes in information systems • heterogeneity of ubiquitous components, standards, data formats, etc.

6. UBIWARE Intelligence as a Service in the cloud Agent-driven service API Services (Payment, Identity, Search) Configuration management Data adaptation Solution stack (Java, PHP, Python, .NET) PaaS Intelligent services Solution stack Structured storage (e.g. databases) Domain models • Smoothly integrate with the infrastructure • Build stack-independent solutions • Automate reconfiguration of the solutions

7. S-APL Pool of Atomic Behaviours S-APL repository UBIWARE platform UBIWARE Agent Beliefs storage Role Script Data .class RAB RAB RAB RAB Blackboard

8. Extended API API extension: OS perspective Cloud Platform Provider Virtual machine PCA PMA SW Platform Customer applications and services PCA – Personal Customer Agent PMA – Platform Management Agent

9. Extended API Data Adaptation as a Service Cloud Platform Provider Virtual machine PMA SW Platform PCA Data Service Customer applications and services Files Adapter Agent DB/KB PCA – Personal Customer Agent PMA – Platform Management Agent

10. API Platform-driven service execution in the cloud Cloud Platform Provider Virtual machine Virtual machine Service execution environment SW Platform PCA Customer applications PMA API PCA – Personal Customer Agent PMA – Platform Management Agent

11. Agent-driven PaaS API extension Agent-driven flexible intelligent service API Agent-driven Adapters Smart data source connectivity Configurable data transformation User applications in cloud Proactive adapter management Agent-driven intelligent services Configurable model Service mobility Proactive self-management Smart cloud stack Stack control and updates Failure-prone maintenance Embedded and remote services Smart Ontology Domain models Standards & compatibility System configuration and policies

12. Intelligent services: PaaS API extension Agent-driven flexible intelligent service API User applications in cloud Agent-driven intelligent services Configurable model Service mobility Proactive self-management

13. Agent service Input Model Output Vector DM result DM model Agent-driven data mining services • Data mining applications are capabilities • Agents can wrap them as services • PMML language - a standard for DM-model representations • Data Mining Group. PMML version 4.0. URL http://www.dmg.org/pmml-v4-0.html

14. PMML*: data mining model descriptions PMML model Header Version and timestamp Model development environment information Data dictionary Definition of: variable types, valid, invalid and missing values Data Transformations Normalization, mapping and discretization Data aggregation and function calls Model Description and model specific attributes Mining schema Definition of: usage type, outlier and missing value treatment and replacement Targets Score post-processing - scaling Definition of model architecture/parameters PMML* - Predictive Model Markup Language (www.dmg.org/pmml-v3-0.html)

15. Input Model Output Vector class of V1 is: “Urgent Alarm” according to model M1 Vector to be classified: alarm message: V1={0.785, High, node_23} Paper machine alarms classifier neural network model (M1) Inputs Model Outputs Set up a model M1 Model player Model M1 assigned Paper machine alarms classifier neural network model (M1) Vector class of V1 is: “Urgent Alarm” according to model M1 Vector to be classified: alarm message: V1={0.785, High, node_23} Input Model Output Learning samples and the desired model settings Model constructor Model M1 parameters Data mining service types Fixed model service Model player service Model construction service

16. A use case for data mining service stack • A “Web of Intelligence” case: Input Model Output Distributed query planning and execution Pattern of learning data to be collected: ?V={?p1, ?p2, ?p3} A set of learning samples (vectors) 1 Learning samples and the desired model settings Model constructor Model M1 parameters 2 Set up a model M1 Model player Model M1 assigned 3 Paper machine alarms classifier neural network model (M1) Vector class of V1 is: “Urgent Alarm” according to model M1 Vector to be classified: alarm message: V1={0.785, High, node_23} 4

17. Mining method Supervised Learning Unsupervised learning Neural networks Clustering kNN Industry Electrical Engineering Process Industry Paper industry Power networks Power plant Data Mining services in UBIWARE Ontology construction Data Mining service Model construction service Computational service Fixed model service Model player service Data mining domain Core DM service ontology Problem domain

18. UBIWARE in cloud computing stack Cloud computing stack Example application UBIWARE for control and management in cloud Semantic Business Scenarios DM model wrapped as a service for paper industry Application as a Service Domain-specific components as services Applications and Software as a Service Application (business logic) Domain model (Ontology) & components DM model for paper industry Services (Payment, Identity, Search) Cross-domain Middleware components Componentization & Servicing Data Mining service player Platform as a service Solution stack (Java, PHP, Python, .NET) Cross-layer configuration & management mechanisms Connectors, Adapters Agent-driven service API Structured storage (e.g. databases) RABs, Scripts Raw data storage and network OS-virtualization Infrastructure as a service Virtualization Machine Hardware configuration Technologies in cloud

19. Conclusions • Web intelligence as a cloud service • Ubiware is a cross-cutting management and configuration glue • Advanced data adaptation mechanisms as cloud services • A competitive advantage for cloud providers • Seamless data integration for service consumption and provisioning • Autonomous agents as a Service (A4S) • Supply any resource with the “autonomous manager”

20. Thank you!