Grids@Work 2008 Sophia Antipolis – October 21 st , 2008

1. ProActive Integration with HP Software in the AGOS projectTowards the integration of business services within a grid environment supported by a virtualized infrastructure Grids@Work 2008 Sophia Antipolis – October 21st, 2008

2. Project context Partner use cases Draft architecture Products mapping HP integration ProActive features Agenda 2 12 September 2014

3. Project context Putting together experts from grids, services, resources and data to define a simple, coherent and realistic solution

4. Project context • Selected by the “Pôle de compétitivité Solutions Communicantes Sécurisées” • List of partners: HP, Oracle, Inria, Activeeon • Defined scope: integration project • Definition of an integration methodology • Definition of new software components • Integration of business and service dashboards • Integration of services with virtualized resources • Contribution to standards • Objective: “Integrate GRID technology into a business oriented SOA stack running over a virtualized infrastructure”

5. Proposed solution • A reference architecture putting together: • business intelligence • service oriented components • grid environment • virtualized systems • A library of process and service templates based on standards • A set of tools to easily build such applications • A real-time monitoring of business processes, services and resources • A methodology to migrate to the architecture

6. Applied onto water operator use case • Simulation application to better handle water distribution network and provision adequate hydraulic meshing • Focus is on software which allows the study, the planning and the control of irrigation • hydraulic simulation software for meshed networks for distribution of raw water or drinking water under pressure • full network is too large to be simulated at once; split into sub-network parts that can be studied independently • Main interest: • Parallelization of the application (SPMD approach) • Overlap multiple runs (parameters sweeping) on subnetworks • Better integrate with other applications • Better use of resources to speed up computation • Measured in terms of speed increasing of digital processing provided by the grid architecture

7. Applied onto global distribution system’s usage use case • Parsing and analysis of web session logs from target applications • Focus is on providing massive data analysis: • Customer session graph analysis or other data mining services • Know how groups of customers behave: the usage / navigation in websites • Main interest • Handle the size and the volume of data generated by core applications • Parsing of logs done massively in parallel • Results need to be stored in a distributed database • Better understand how the products are used • Help to improve the products continuously through the feedback loop • Expectations • Enable an advanced workflow beyond current standards to support processes and sub processes and/or to compose hybrid processes with services. • Enable streaming results from any parallel activity on a grid to feed a distributed database • Support operational aspects like monitoring, clustering, scalability, security, load balancing and failover thus enabling high availability SLA compliance agreements to be respected. • Enables one button push start, stop and resume on any running grid process

8. Draft architecture From building blocks to functional architecture proposal

9. Base line… Staged pyramid Adaptive infrastructure of heterogeneous resources Service delivery with quality added-value Business intelligence based on standardized processes Based on standards As defined in the related standards documents Existing implementations (proprietary / open-source) With grid middleware Grid environment: parallelism, distribution Web interfaces: thin web clients access Modeling facilities: composition, adaptive, configurable Efficient execution: design & execution patterns 9 12 September 2014

10. …of a Service Grid BPELETL SCA / GCMSOA / WS VM / XENP2P / GTK Functional Standards BusinessIntelligence BI Grid Grid Grid Grid ServiceArchitecture SA AdaptiveInfrastructure AI

11. Main building blocks Systems Virtualization Grid Utility Interface Resource Manager Tasks & Services Scheduler Enterprise Service Bus Distributed Shared Memory Database Interface Application Server Interface Web Service Interface SOA Registry / Repository SOA Governance Component Architecture Load Balancer Fault Tolerance Reconfiguration Process Business Process Execution Real-time Monitoring Tools Service Level Management Business Intelligence Reporting Tools

12. Define the business processes Processes defined in terms of business items Cope with business processes workflow Focus on business interactions between human and applications How are used the applications How are they chained in any given scenario What is expected as an output Not a programmatic task flow Do not cope with service implementation details Far from resource allocation concerns Handle specific concepts: Long transactions (suspended time, recurring patterns…) Requests approval (decision making, validation…) Human interactions (subjective perception, biased usage…) Business ecosystem (management, billing, visualization…)

13. Business Intelligence Reporting BusinessIntelligence Service Level Management Business Process • - Delivered results analysis • - Scenario validation • - Service & Service Level monitoring • - Reporting on key aspects by category • - Validation of performance gain, result precision, time to deliver… • - Comparison with reference benchmarks ServiceArchitecture AdaptiveInfrastructure

14. Calling value-added services Services stored in a registry with governance applied Manipulating proxies to available web services Services as composite components With predefined characteristics and features Interface contract Policy enforcement Delivered quality of service Mapped onto: Existing web services (black-box) Composite web services (functional composition) Skeleton web services (parallel pattern) Subordinate processes (time composition) Simple sequential services

15. Implemented as grid services Components executed on a set of resources Skeletons providing parallel patterns for efficient execution on the grid Transformed as jobs delegated to the grid scheduler Optimized active objects exposed as web services on the grid Jobs scheduled on the grid with direct access to the grid scheduler Specialized grid schedulers dedicated for specific usage

16. Service Architecture BusinessIntelligence SOA Environment Repository, Registry, Orchestration Service Composition Service Component ArchitectureParallel Programming Distributed shared memory ServiceArchitecture - Distributed parallel services accessing data in distributed shared memory - Functional composition of service components with dynamic reconfiguration - Exposed as web services in registry with a policy enforcement contract - Executed through proxies & orchestrated by a workflow engine AdaptiveInfrastructure

17. Executing on heterogeneous resources Code execution on well-known resources or over a set of dynamically discovered resources As tasks through a scheduler managing load-balancing, fault tolerance, high availability… As active objects with associated features such as migration, file transfer… As uncontrolled legacy application As tasks delegated to another grid utility environment Evolving set of resources (join / leave support) With specific or adaptive characteristics (virtualized or clustered) On a set of various operating systems Protected or not, intrusive or not

18. Adaptive Infrastructure BusinessIntelligence - Virtual resources managed dynamically- Tasks executed on the set of evolving resources- Requests of tasks execution received via the bus ServiceArchitecture Task Scheduling Resource Manager AdaptiveInfrastructure Grid Utility interface Resource Virtualization

19. Proposed architecture

20. Products integration mapping Global contribution ActiveEon products from the ProActive Suite Oracle products from the Business Services Suite HP products from the Virtualization, Quality and SOA Centers Alternatives from the open-source among others ActiveBPEL, Cockpit Sequoia Tuscany Xen VM

21. BPEL Process Manager BAC Dashboard BAC Quality Center DB + RAC J2EE AS JDeveloper DSM Coherence BIEE Oracle BPA Suite SOA Manager SOA Systinet Enterprise Service Bus Tasks Scheduler Resource Manager GCM components IC2D Skeletons Hewlett Packard Integrity VM System Insight ProActive Grid Utility Proposal Reporting BusinessIntelligence Business Process SLA Mgt BPEL Engine SOA Environment Repository, Registry, Orchestration Service Composition Service Component ArchitectureParallel Programming Distributed shared memory ServiceArchitecture Real-Time Monitoring Quality Testing Enterprise Service Bus Task Scheduling Resource Manager AdaptiveInfrastructure Grid Utility interface OS Virtualization OS, HW

22. Open Source Cockpit Active BPEL IC2D MySQL Jboss / Jonas Sequoia Apache jUDDI Eclipse RUDDI Tuscany Apache ServiceMix Tasks Scheduler Resource Manager Skeletons Timit Xen VM Gtk BusinessIntelligence BPEL Engine SOA Environment Repository, Registry, Orchestration Service Composition Service Component ArchitectureParallel Programming Distributed shared memory ServiceArchitecture Real-Time Monitoring Quality Testing Enterprise Service Bus Task Scheduling Resource Manager AdaptiveInfrastructure Grid Utility interface OS Virtualization OS, HW

23. HP Integration Points Products & Methodology

24. HP contribution • Adaptive Infrastructure lab for the project • Architecture blueprint draft • Consulting & Integration methodology • HP Software products from: • SOA Center (Service Oriented Architecture) • BSM Center (Business Service Management) • VSE Center (Virtual Server Environment) • Quality Center (Quality Test Framework) • Objective: Virtualization of services and resources of the parallel / distributed grid in a business context

25. AGOS blueprint draft

26. Products from HP SOA Center • HP Systinet • UDDI Registry • As the de-facto SOA reference repository • HP Policy Enforcement • Policy Enforcement • As the smart SOA governance runtime • HP BAC for SOA • Services Monitoring • As the integrated SOA services monitoring • Combining business, service and resource monitoring • HP Quality for SOA • Services Testing • As the best-in-class testing environment

27. Integrated specifically within AGOS Provide a library of business processes templates including both functional aspects (gather input data, run simulation, perform pre-processing, analyze output data…) and business aspects (warn user, check execution time, request for more resources…) Define the library of predefined SOA services Declare all the SOA services to be used by the business processes in the SOA registry Define and declare all SOA policy enforcements (as proxy of services) on top of existing SOA services

28. Products from HP BSM Center • HP Business Availability Center • Integrated business dashboard • Reporting for business metrics at every layer • Business processes • Services execution • Resource consumption • HP System Insight Manager • Collect raw metrics from adaptive infrastructure • Feeder of HP BAC systems for real or virtualized hardware problems

29. Integrated specifically within AGOS • Define or discover the business service models to get monitored • Get in SOA registry all the services to get metrics on the architecture components • Push the indicator values into the BAC monitoring system (source, metrics name + value) • Define the SLA based on customer’s expected results • Define the reports to deliver to the business managers

30. Products from HP VSE Center • HP VM • Virtualization of resources and operating systems • Adaptive deployment of resources with hypervisors • Support enterprise hardware and configurations • Storage Area Network • Blade Servers (Itanium, Pa-Risc, x86) • Hardware & Software High Availability • HP RDP • Resource deployment protocol • Manage VM and installed software transparently • Integration expertise with Xen environment • Integration expertise with VMWare environment • Thursday 23rd 10:30: HP VSE by Francois Donze

31. Integrated specifically within AGOS Deploy hardware (cpu, ram, disk) on demand Deploy operating system / virtual machine or dedicated application through web services Integrate with ProActive resource manager Add, get or list available nodes Define constraints of usage or quality of service Deploy within the hour Schedule the deployment of x VM Get node with x ram + y cpu

32. ProActive used features Framework in action

33. ProActive Suite

34. ProActiveAdapter BPEL Process Manager Quality Center BAC BAC Dashboard DB + RAC J2EE AS BIEE JDeveloper BPA Suite Coherence SOA Manager SOA Systinet Enterprise Service Bus Tasks Scheduler Resource Manager IC2D 7 X 8 3 6 5 2 4 1 Skeletons 9 Integrity VM System Insight Grid Utility Integration with ProActive Business Intelligence BPEL Engine SOA Environment Repository, Registry, Orchestration Service Composition Service Component ArchitectureParallel Programming Distributed shared memory Service Architecture Real-Time Monitoring Quality Testing Enterprise Service Bus Task Scheduling Resource Manager AdaptiveInfrastructure Grid Utility interface OS Virtualization OS, HW

35. Integration points with ProActive Generate SCA component as a GCM component Expose grid components in SOA Repository Provide distributed shared memory’s data to tasks & components Publish metrics from components, skeletons, tasks and resources Make tasks scheduler accessible as a service Expose virtual machines as resource nodes Expose external grid services (like scheduler as a node resource hosting a tasks scheduler) Provide IC2D aggregated metrics towards monitoring system Provide introspection agent to test a particular service, component, task or object

36. Thank youQuestions / Answers