1 / 33

Delivering Grid Interoperability

Delivering Grid Interoperability. Dr Alistair Dunlop, Project Manager, OMII-Europe a.dunlop@ecs.soton.ac.uk. Outline. Macro view – What is the project trying to achieve? (4) The project organisation and structure (3) The role of standards in grid interoperability (3)

mahala
Download Presentation

Delivering Grid Interoperability

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Delivering Grid Interoperability Dr Alistair Dunlop, Project Manager, OMII-Europe a.dunlop@ecs.soton.ac.uk

  2. Outline • Macro view – What is the project trying to achieve? (4) • The project organisation and structure (3) • The role of standards in grid interoperability (3) • Micro view – Standardised Components from OMII-Europe (11) • Putting OMII-Europe components together to deliver real grid interoperability (5) • Summary (3)

  3. The big picture • A definition of e-Science… • “e-Science is about global collaboration in key areas of science and the next generation infrastructure that will enable it” • ‘Next generation infrastructures’ represented by Grids • Foundation for seamless and secure collaborations over networks • National Grid infrastructures: UK NGI, D-Grid, others • New science will require the combined resources of all existing infrastructures

  4. Problem is e-Infrastructure Islands • E.g., DEISA Grid (Supercomputing/HPC community) • Non WS-based UNICORE 5: Proprietary jobs (AJO/UPL) • No Virtual Organization Membership (VOMS), Full X.509 • Suitable for parallel scientific jobs (MPI, many interactions) • E.g., EGEE Grid (mainly HEP community + others) • Non WS-based gLite: Proprietary jobs (JDL) • Proxy-based X.509 security, but proprietary VOMS support • Suitable for embarrassingly parallel jobs (few interactions) • Scientists cannot use one middleware to access both (Little adoption of standards)

  5. OMII-Europe Vision: Interoperability Highway End-users via clients& portals GOAL: Transparencyof Grids for end-users Emerging OpenStandards „Interoperability highway“based on open standards others Grid Middlewares others Grid Resources

  6. OMII-Europe in context • A guide to flagship e-infrastructure projects... • Connectivity (Geant) • Cluster grids (EGEE) • Supercomputer Grid (DEISA) • Middleware (OMII-Europe) • “Software that enables you to use and easily access these above distributed EGEE and DEISA distributed grid infrastructures” • “Together these elements form the e-infrastructures that create “Global Virtual Research Communities”...”

  7. The OMII-Europe Project • OMII-Europe stands for • Open Middleware Infrastructure Institute for Europe • It is an EU-funded project: FP6, RI • It has an initial duration of 2 years • May 2006 -> April 2008 • It has been granted a contribution of 8M € • It involves 16 partners • 8 EU • 4 USA • 4 China

  8. Partners

  9. Project Structure and Effort Allocation • Networking activities • Management, Outreach, Training • 8% Person Effort • Service Activities • Repository, QA, Support • 25% Person Effort • Joint Research Activities • Re-engineering, new services, integration, benchmarking • 67% Person Effort

  10. OMII-Europe Standards-based approach to Grid Interoperability • Adapters-based: • The ability of Grid middleware to interact via adapters that translate the specific design aspects from one domain to another • Standard-based: • the native ability of Grid middleware to interact directly via well-defined interfaces and common open standards * definition inspired by OGF GIN CG

  11. Participation in Middleware Standardisation • Most project participants involved as member/observer in many OGF WG • 11 project participant hold senior positions in • OGSA DAIS WG (Database Access and Integration Services) • OGSA RUS WG (Resource Usage Server) • OGSA BES WG (Basic Execution Service) • OGSA JSDL WG (Job Submission Description Language) • GIN CG (Grid Interoperability Now) • OGSA-AuthZ-WG (Authorization) • GLUE WG • GFSG WG (Grid File System) • RM WG (Reference Model) • OGSA Naming WG • Technical Standards Committee • GSA RG (Grid Scheduling Architecture) • GRAAP WG (Grid Research Agreement Allocation Protocol) • OGSA BYTE IO WG • OGSA D WG (Data) • OGSA DMI WG (Data Movement Interface)

  12. JRA4 SA3 SA1 JRA3 SA2 The Virtuous Cycle – Technology transfer with Grid projects and standards organisations Standards Compliance Testing and QA JRA2 New Components Standards Implementation Components JRA1 IN Globus Benchmarking Repository OUT OMII-UK Components CROWN Supported Components on Eval. Infrastructure Integrated Components

  13. Standards Components from OMII-Europe • Initial focus on providing common interfaces and integration of major Grid software infrastructures • Common interoperable services: • Database Access • Virtual Organisation Management • Accounting • Job Submission and Job Monitoring • Information modelling • Infrastructure integration • Initial gLite/UNICORE/Globus interoperability • Interoperable security framework • Access these infrastructure services through a portal

  14. Job Submission • Unify Job Submisson and Monitoring interface • Adoption of emerging OGSA-BES and JSDL standards • Alpha BES and JSDL implementations for • UNICORE 6, gLite 3.1, Globus 4, OMII-UK, CROWNgrid • Interoperability demonstrated through use of a BES compliant meta-scheduler

  15. VO Management • To provide a common Virtual Organisation (VO) management solution across middleware distributions • Extend VOMS Interface to support emerging AuthZ standard • compliance with SAML Authorisation model • Extension, not a replacement interface • Public release of VOMS integrated with UNICORE

  16. Accounting • Unify accounting information across middleware distributions • Provide standardized interfaces for accessing that information • Information standard: • Usage Record Format (URF) • Service interface standard: • Resource Usage Service (OGSA-RUS) • Alpha versions RUS • gLite (DGAS) • Globus (SGAS) • UNICORE

  17. Data Access • Port OGSA-DAI 3.0 from Globus to other middleware distributions available throughout Europe and China • UNICORE • gLite • CROWN

  18. Information Modelling • There is a lack of a common description of Grid resources suitable for discovery, monitoring and scheduling • Many descriptions exist • e.g.: GLUE Schema, NorduGrid Schema • Working on the definition of next-generation GLUE Information Model in the context of OGF GLUE WG and its implementation

  19. Portal • Deliver tools for developing Grid portals and support for key Web and Grid standards and technologies • Objectives: • Develop gateway to OMII Evaluation Infrastructure • Develop tools for portal and grid software training • Explore new approaches for grid portal development

  20. Repository of Open-Source Software • Make available software reengineered within OMII-Europe and contributed by third parties • Single services/tools & complete distributions • Provide an interface to select software from the repository based on user requirements • By capability/standards/provider/… • Support the upload, download and installation of the software • Document platform portability & pre-requisites • Verify the software through compliance & metrics tests

  21. Behind the Repository • Leverage existing infrastructure & projects • ETICS • Capture build & test configuration data for repeatability • NMI Build & Test Framework • Manage cross-platform environment for build & tests • Condor • Underlying execution infrastructure • Provides reports to be displayed within the portal • Builds: Pre-requisites & platforms • Testing: Conformance & Interoperability

  22. Tests For Standards Conformance • Job Submission and Job Monitoring • Job Submission Description Language (JSDL) • Basic Execution Service (BES) • Accounting • Usage Record (UR) • Resource Usage Service (RUS) • Database Access • WS-DAI, WS-DAIX, WS-DAIR (OGSA-DAI) • Virtual Organisation Management • Move towards SAML2

  23. Standards status: March 2008 • Accounting • SGAS/DGAS/UNICORE-RUS • OGSA-Resource Usage Service (RUS – OGF draft specification) - implies: • Usage Record Format (UR) (OGF recommendation status awaiting implementation) • Job Submission and Job monitoring • CREAM-BES, GLOBUS-BES, UNICORE-BES • OGSA-Basic Execution Service (BES) version 1.0 (OGF final specification) – implies: • Job Submission Description Language (JSDL) version 1.0 (OGF final specification) • Database Access • OGSA-DAI • WS-DAIX and WS-DAIR are being implemented in OGSA-DAI (both currently OGF candidate standards at 1.0 awaiting implementation) • Virtual Organisation Membership Service • VOMS implements the following: • Security Assertion Markup Language (SAML) Version 2.0 OASIS (March ’05) • Extensible Access Control Markup Language (XACML) version 2.0 OASIS (February ’05) • Information modelling • GLUE 2.0 • standard is currently draft, expected to go out for public comments by the end of April 2008, final version by September 2008

  24. Using OMII-Europe components to deliver grid interoperability – WISDOM scenario • Wide In Silicio Docking On Malaria (WISDOM) projects • Developing new drugs for neglected and emerging diseases with a particular focus on malaria • Accelerated Research & Development for emerging and neglected diseases • Massive computation reduces R&D costs

  25. Dr. Nicolas Jacq [16]

  26. WISDOM in the context of DEISA & EGEE • WISDOM uses EGEE for large scale in-silicio docking • Comp. method for prediction of whether one molecule will bind to another • Using AutoDock and FlexX software provided via gLite in EGEE (FlexX – licensed software on EGEE with 6k licenses) • Output is a list of best chemical compounds (potential drugs – NOT final solution) • DEISA to find best compound from potential drugs list • Fast molecular dynamics computations • Using highly scalable AMBER (Assisted Model Building with Energy Refinement) in DEISA • Goal: Accelerate drug discovery using EGEE + DEISA

  27. Interoperability Scenario: WISDOM SAML- based Attribute Authority (AA) VOMS getscentralrole & middlewareindependent

  28. Work in progress Further Interoperability Scenarios: EU - IndiaGrid

  29. What can you do Now… and Later… • Now • Most products at Beta stage becoming publicly available • They provide basic interoperability of multiple grid middleware systems focusing on job execution • Available to early adopters working with OMII-Europe partners • Spring 2008 (end of current project) • Further security integration work between different middleware platforms (SAML-VOMS, TLS (Transport level security)) • Completed QA’d services and demonstrated end-to-end solutions • Availability of GLUE 2 information model service implementations

  30. Caveat... • So you can use EGEE and DEISA seamlessly? Not quite... • OMII-Europe has provided much of the required technical interoperability, but • Needs EGEE and DEISA sites to deploy either the latest versions of the grid middleware, or to deploy the OMII-Europe services available from the repository • AND... • Currently need to apply for resources independently. Intention is that EGI will do this in the future

  31. Summary (1/2) • OMII-Europe is a 24 Month EU funded project with 16 partners to establish grid infrastructure interoperability through implementing a set of agreed open standards on all middleware platforms • OMII-Europe is implementing a number of components that will allow identically specified jobs to be run, managed and migrated to different middleware platforms • Initial versions of BES, VOMS/SAML and security service have already enabled UNICORE and gLite managed resources to be used by the same job • Users can try interoperability on the OMII-Europe evaluation infrastructure, or obtain services for installation on their own resources from the OMII-Europe repository

  32. Summary (2/2) • We anticipate OMII-Europe services to be integrated into standard middleware distributions as well as deployed on large scale e-infrastructures such as EGEE and DEISA • OMII-Europe requested continuing funding in the September EU call to support the existing services and provide further services in the areas of data and Grid management • We are interested in working with projects that have real grid interoperability issues to ensure our solutions match your needs.

  33. Further Information General information: http://omii-europe.org Repository: http://repository.omii-europe.org Support: http://support.omii-europe.org • With special thanks to the following project members for diagrams and slides: • Morris Riedel • Stephen Brewer

More Related