1 / 32

European Middleware Initiative (EMI)

Learn about the European Middleware Initiative (EMI) project's objectives, administrative and technical structures, relationships, and collaborations. Discover how EMI aims to consolidate, develop, and promote middleware services for distributed computing infrastructures.

lsimon
Download Presentation

European Middleware Initiative (EMI)

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. European Middleware Initiative (EMI) Alberto Di Meglio (CERN) Project Director

  2. Outline • Project objectives • Administrative structure • Technical structure • The EMI Middleware • Relationships and Collaborations • Conclusions EMI Overview

  3. State of the Art and Objectives EMI Overview

  4. A European Vision Tomorrow Sustainability Persistence Interoperability Easier Access Today EMI Overview

  5. What is EMI? The European Middleware Initiative (EMI) project represents a close collaboration of the three major middleware providers - ARC, gLite and UNICORE, together with other software providers - to establish a sustainable model to support, harmonise and evolve the grid middleware for deployment in EGI and other distributed e-Infrastructures EMI Overview

  6. Objectives Consolidate the existing middleware distribution streamlining services and components to meet and exceed the operational requirements of EGI and other distributed computing infrastructures Develop and standardize the middleware services/functionality following the requirement of EGI and other DCIs, fully integrating virtualization, MPI, monitoring interfaces and embracing new computing models like clouds Consolidate Evolve EMI Overview

  7. Objectives Reactively and proactively maintain the middleware distribution to provide users with increasingly user-friendly, maintainable, reliable, stable, and scalable software Promote the EMI achievements in the user communities, defining and implementing standards when needed, contributing to the establishment of a sustainable model Support Promote EMI Overview

  8. Project Governance EMI Overview

  9. Partners (23) EMI Overview

  10. Administrative Structure Collaboration Board (CB) ECB Project Director Technical Director Project Executive Board (PEB) Project Technical Group (PTG) EMI Overview

  11. Project Structure Administrative and Technical Management Dissemination and Communication Quality Assurance Maintenance and Support Integration, Interoperability and Testbeds Development and Evolution EMI Overview

  12. Technical Structure EMI Overview

  13. Technical Areas Job Management Data Management Information Services Security Quality Assurance Maintenance and Support Integration, Interoperabilty and Testbeds Development and Evolution EMI Overview

  14. Product Teams • Product teams are the services implementation teams within each area responsible to deliver software releases and all associated material. • They perform the required technical tasks from design to release through implementation, testing, certification and 3rd-level support as part of one or more Work Package and according to what specified by the Project Technical Group and the QA policies. • Product Teams are flexible, in the sense that they can be formed or closed as the corresponding products are introduced or obsoleted in EMI and allow adding or removing services as needed even from external contributors • They provide a transparent and direct method to assign responsibility for a service EMI Overview

  15. Product Teams Job Management Data Management Information Services Security EMI Release x.y.z-r x.y.z-r Quality Assurance Maintenance and Support Integration, Interoperabilty and Testbeds Development and Evolution EMI Overview

  16. Software Engineering PT Release Stress tests Scalability tests Functional tests Regression tests Deployment tests Packages Unit tests Build Develop Design Quality Assurance Maintenance and Support Definition Execution Integration, Interoperabilty and Testbeds Development and Evolution EMI Overview

  17. Quality Assurance Improve Release Stress tests Scalability tests Functional tests Regression tests Deployment tests Packages Unit tests Build Develop Design ISO 12207 (Software Lifecycle Processes)ISO/IEC 14764 (Software Maintenance) ISO/IEC 29119 (Software Testing)ISO 9126(Software Quality) Quality Assurance Maintenance and Support Integration, Interoperabilty and Testbeds Development and Evolution Monitor EMI Overview

  18. Releases and Release Policies • Major releases: once or twice per year, may contain non-backward compatible changes • Minor releases: a few times per year, fully backward compatible, may contain new functionality • Revisions: every week or two weeks, only bug fixes • Emergency: as needed, only specific bug fixes, use emergency release procedures Service Level Agreements EMI Overview

  19. The Middleware EMI Overview

  20. Selection Criteria • EMI includes: • Middleware services and components from the participating providers • All services that are or will be in production by next year • EGI, PRACE operations must be fully supported • New services that are supposed to replace existing services • New services satisfying identified new user needs • EMI requirements are user-driven, satisfy the needs of the user communities (Archaeology, Astronomy, Astrophysics, Civil Protection, Computational Chemistry, Earth Sciences, Finance, Fusion, Geophysics, High Energy Physics, Life Sciences, Multimedia, Material Sciences, etc.) EMI Overview

  21. Day-0 Distribution Grid Manager A-REX libarclient ng* job CLI arc* job CLI JURA TSI XNJS UAS-Compute OGSA-BES URC UCC HILA WMS LB CREAM MPI Classic SE libarcdata2 ng* data CLI arc*data CLI Chelonia dCache StoRM UAS-Data DPM lcg_utils DPM/DICOM LFC FTS Hydra AMGA Classic Infoserver Classic Infoindex ALIS ISIS libarcclient (infosys) Grid Monitor CIS Service Registry BDII lcg-info and lcg-infosites Service Discovery API CEMon Messaging APEL HED security API arcproxy Charon Gateway XUUDB XACML Entity UVOS VOMS/VOMS-Admin voms-proxy-* ARGUS SLCS SCAS/LCAS/LCMAPS gLite-Exec (glExec) gLite-Delegation Client Shibboleth SLCS Client Gridsite CGSI_gSoap delegation-Java Trustmanager/Util-Java Proxyrenewal Information Services Job Management Data Management Security EMI Overview

  22. Harmonization Targets Data Handling Unification APIs and Clients Harmonization Standardization (PGI, SRM, SAML/XACML, US/RUS, GLUE,etc) Common Service Registry Common Security Framework EMI Overview

  23. New Development • New services or functionality are introduced as needed based on: • New user requirements • Need to evolve existing services or replace older technologies to follow the growing needs of the infrastructures • Special focus on integration of virtualization (grids on clouds), monitoring interfaces, support for easy construction of gateways • Trade between new research and stability of the infrastructure operations • Large-scale realistic testing, phase-out/migration plans, support policies EMI Overview

  24. Relationships EMI Overview

  25. EGI EGI MCB, SA2 (MU), SA3, SA4 3rd-level supportReleases Specs Requirements Repositories, tests, certification tools, etc PTG EMI Overview

  26. User Communities SSCs EGI MCB, SA2 (MU), SA3, SA4 SSCs SSCs 3rd-level support Releases Specs Requirements Repositories, tests, certification tools, etc PTG EMI Overview

  27. Collaborations SSCs EGI MCB, SA2 (MU), SA3, SA4 SSCs SSCs 3rd-level support Releases Specs Requirements Repositories, tests, certification tools, etc PTG Works with EMI Interoperability partners, commercial partners, EMI Overview

  28. Exploitation and Sustainability • Successful deployment in EGI, PRACE and other infrastructures of the EMI middleware • Collaboration with standardization bodies • Evolution of existing standards • Definition of new standards • Inclusion of middleware in OS distributions • Packaging requirements, support, conflict management • Service Level Agreements • Move towards a customer/provider model that could in time become a real commercial relationship EMI Overview

  29. Open Collaborations • EMI is not a private club • Collaborations with additional software providers are welcome and sought after • “Works with EMI” Program • Collaborating providers need to follow the same rules as the EMI partners for the software to be included in EMI releases • Openness is necessary to converge towards usable common standards and seriously interoperating solutions within and outside Europe • Industrial collaborations are welcome EMI Overview

  30. Conclusions EMI Overview

  31. Conclusions • EMI is built on the strengths of the major European middleware providers • To provide a reliable, usable, scalable, interoperable middleware distribution • It is designed to be the major middleware provider for EGI, PRACE and other DCIs • Support for reliable operations • Evolve the services as the communities grow • It’s an important step forward in the establishment of a true pan-European sustainable, persistent research infrastructure EMI Overview

  32. Thank you

More Related