1 / 22

Adoption Status of the OGSA-BES interface unicore.eu

Morris Riedel, Mohammad Shahbaz Memon … and many others from the UNICORE community Jülich Supercomputing Centre (JSC). Adoption Status of the OGSA-BES interface http://www.unicore.eu. Outline. Architecture Overview OGSA-BES Adoption Overview Job Submission Description Language

micah
Download Presentation

Adoption Status of the OGSA-BES interface unicore.eu

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Morris Riedel, Mohammad Shahbaz Memon … and many others from the UNICORE communityJülich Supercomputing Centre (JSC) Adoption Status of theOGSA-BES interfacehttp://www.unicore.eu

  2. Outline • Architecture Overview • OGSA-BES Adoption Overview • Job Submission Description Language • High Performance Computing Basic Profile • OGSA-BES Adoption in Detail • OGSA-BES Limitations/Additions • Future Work for OGSA-BES • Link to other GIN session: Security Profile • References http://www.unicore.eu

  3. Architecture Overview http://www.unicore.eu

  4. Local RMS (e.g. Torque, LL, LSF, etc.) Local RMS (e.g. Torque, LL, LSF, etc.) Architecture Portal client, e.g. GridSphere command-line client Eclipse-basedclient GPEapplication client scientific clientsand applications X.509 SOAP WS-RF WS-I JSDL authentication Gateway Gateway ByteIO UNICOREWS-RFhostingenvironment ByteIO emerging standardinterfaces UNICOREWS-RFhostingenvironment UNICOREWS-RFhostingenvironment ServiceRegistry UNICORE Atomic Services OGSA-* UNICORE Atomic Services OGSA-* JSDL JSDL BES BES HPC-P HPC-P RUS RUS Grid services hosting UR UR XNJS XNJS IDB IDB job incarnation &authorization XACML XACML XACML entity XACML entity X.509 XUUDB SAML-VOMS SAML Target System Interface Target System Interface DRMAA DRMAA parallel scientific jobs of multiple end-users on target systems R. Ratering et al. [6] M. Riedel et al. [1] http://www.unicore.eu

  5. OGSA-BES Adoption Overview http://www.unicore.eu

  6. OGSA – Basic Execution Services (BES) • Three Services basicallysubstituting the proprietary services: • TargetSystemFactory (TSF) • TargetSystemService (TSS) • JobManagementService (JMS) • OGSA-BES Factory Service • CreateActivity( JSDL ) • OGSA-BES Management Service E.g. Allow new submissions or not • Use of UNICORE 6 Backend XNJS • Forwarding to TSI on resource • Developed by M.S. Memon et al. [2] http://www.unicore.eu

  7. Job Submission Description Language (JSDL) • UNICORE 5 • Proprietary description:Abstract Job Object (AJOs) • UNICORE 6 Services • Target System Service (TSS)accepts JSDLs • OGSA – Basic Execution Services accepts JSDLs • UNICORE 6 Execution Backend • XNJS backend componentnatively accept JSDLs • JSDLs are transformed into a proprietary simple protocolto submit job to RMSs (e.g. Torque) M. Riedel et al. [1] http://www.unicore.eu

  8. HPC – Basic Profile (HPC-BP) • (Participated in the Supercomputing 2007 Interoperation demonstrations) • XNJS accept JSDL extensions compliant to ones defineswithin the HPC-BP profile • Service Container provides asecurity handler to accept credentials according toWS-Security Username TokenProfile (part of HPC-BP) M.S. Memon et al. [2] http://www.unicore.eu

  9. OGSA-BES Adoption in Detail http://www.unicore.eu

  10. OGSA – BES Adoption in Detail (1) • Support of all three defined portTypes with defined faults • Configurations in uas.config configuration file of UNICORE • E.g. initial value whether accept new activities or not(property of BES-Management portType) • Support of BES State Model (and UNICORE state model) • Additional States: • BES interface implementation (developer view) • Use the xFire SOAP engine • Use XMLBeans for Java-XML bindings • No problems in creating XMLBeans from BES schemas(no schema for BES-Activity provided) • Sourcecode as open source on UNICORE@SourceForge • Execution Management provided by XNJS of UNICORE http://www.unicore.eu

  11. OGSA – BES Adoption in Detail (2) • Supports filtered activities during BES-Factory operations… • (Filtered activities are activities, which are profiled or associated with users credentials, and its visibility will be exposed to only the authorized users who created the activity) • Own filter created to support this feature • Supported Operations: GetActivityDocuments, GetActivityStatuses, TerminateActivities • UNICORE 6 supports the optional WS-RF Rendering by adding all WS-RF compliant message exchanges • Enables GetResourceProperties, UpdateResourceProperties,… • Activity resources are modeled as WS-RF resources, each BES Activity instance maintains activity document as state • Consists of JSDL sent by user, reference to BES-Factory which created that activity instance, and the current activity status http://www.unicore.eu

  12. OGSA – BES Adoption in Detail (3) • Client support in UNICORE • UNICORE Rich Client support as additional plug-in demonstrator • UNICORE Command-Line Client (UCC) Support • Interoperability Tests with othe OGSA-BES implementations in the scientific Grid landscape • Production Level for e-Infrastructure integration • Only tested with CREAM-BES (SAML-based Security), planned for gLite release – CREAM in certification process • Initial tests with Globus-BES beta (provided by KTH) • Interoperablity Testings with other HPC-BP adopters • E.g. NorduGrid ARC implementations, University of Virginia implementations,… http://www.unicore.eu

  13. OGSA-BES Limitations/Additions http://www.unicore.eu

  14. OGSA – BES Limitations / Additions (1) • Additions to the BES-Factory Attributes Document  INFOS! • GLUE resource property (for specifying admin domain info, application info, computing resource info) for more information • Additions to the BES-Activity portType • (operations are not normatively defined in specification) • To support an easy access to single resources (instead of vectors/arrays as in BES-Factory) operations added • In addition, also WS-RF compliant message exchanges work at this portType – to work with the resource model in UNICORE • No aligned security model strong enough for production • High Performance Computing Basic Profile good first step into right direction, but not for production usage in e.g.DEISA • Precise profiled attribute-based AuthZ support required today http://www.unicore.eu

  15. OGSA – BES Limitations / Additions (2) • Link to storage added in BES-Activity • (out of scope of OGSA-BES – but required in UNICORE) • Storage Endpoint to a UNICORE proprietary Storage Management Service (SMS) – Req. for BES-related activity! • SMS Endpoint added to the BES Activity resource properties • Property manages a user space mapping with individual activities • SMS can be used to access outcome of computational jobs • File Transfer for data staging issues (not precise enough) • SMS link in turn is a factory for managing individual file transfer functions such as stage-in/out of files for individual jobs • File Transfer realized via File Transfer Service (i.e. ByteIO) • Data Staging Profile might be interesting but is not complete in-line with production deployment http://www.unicore.eu

  16. OGSA – BES Limitations / Additions (3) • UNICORE 6 basic infrastructure relies on WS-RF • Any activity is an instance of the WS-RF resource model • ‘Optional WS-RF support’ in specification makes it difficult for UNICORE clients to work with other BES implementations • E.g. GetResourceProperties of CREAM-BES fails since it is not WS-RF compliant • ‘Optional WS-RF support’ requires a factory for the factory • WS-RF factory pattern difficult to implement • Because factory is not end-user specific instance http://www.unicore.eu

  17. OGSA – BES Limitations / Additions (4) • Bottom lines • BES is not isolated – it must be well embedded with other middleware services – but how to standardize/specify • Storage is tightly connected with BES • File transfer is tightly connected with BES • Security Setups are tightly connected with BES • For all these topics are not normative standards defined – only BES/JSDL • More Profiling is required http://www.unicore.eu

  18. Future Work for OGSA-BES http://www.unicore.eu

  19. Future Work for OGSA-BES • Enable more functionalities in terms of … • data-staging profile, information model GLUE hooks, … • Delegation Scenarios • How are third party transfer credentials transferred through the OGSA-BES interface • E.g. data stage-in is performed using GridFTP (requiring credentials – is not an implementation issue) • SAML delegation vs. Proxy delegation  Link to GIN Security Profile Session • More production-oriented security profiles aligned with BES • Attribute-based Authorization, where credentials transported? • E.g. WS-Security Message exchanges in SOAP header • Link to GIN Security Profile Session http://www.unicore.eu

  20. References http://www.unicore.eu

  21. References (I) [1] M. Riedel, B. Schuller, D. Mallmann, R. Menday, A. Streit, B. Tweddell, M.S. Memon, A.S. Memon, B. Demuth, Th. Lippert, D. Snelling, S. van den Berghe, V. Li, M. Drescher, A. Geiger, G. Ohme, K. Benedyczak, P. Bala, R. Ratering, A. Lukichev Web Services Interfaces and Open Standards Integration into the European UNICORE 6 Grid Middleware Proceedings of 2007 Middleware for Web Services (MWS 2007) Workshop at 11th International IEEE EDOC Conference "The Enterprise Computing Conference", 2007, Annapolis, Maryland, USA, to appear [2] M.S. Memon, A.S. Memon, M. Riedel, B. Schuller, D. Mallmann, B. Tweddell, A. Streit, S. van den Berghe, D. Snelling, V. Li, M. Marzolla, P. Andreetto Enhanced Resource Management Capabilities using Standardized Job Management and Data Access Interfaces within UNICORE Grids Proceedings of 3rd Workshop on Scheduling and Resource Management for Parallel and Distributed Systems SRMPDS 2007, ICPADS'07 - The 13th International Conference on Parallel and Distributed Systems Hsinchu, Taiwan, December, 2007, to appear [3] M. Riedel, R. Menday, A. Streit, and P. BalaA DRMAA-based Target System Interface Framework for UNICOREProceedings of Second International Workshop on Scheduling and Resource Management for Parallel and Distributed Systems (SRMPDS’06) at Twelfth International Conference on Parallel and Distributed Systems (ICPADS’06), Minneapolis, USA, IEEE Computer Society Press, pages 133 - 138 http://www.unicore.eu

  22. References (II) [4] W. Frings, M. Riedel, A. Streit, D. Mallmann, S. van den Berghe, D. Snelling, and V. LiLLview: User-Level Monitoring in Computational Grids and e-Science Infrastructures.In Proc. of German e-Science Conference 2007, Baden-Baden, Germany, Online-Publication [5] V. Venturi, M. Riedel, A.S. Memon, M.S. Memon, F. Stagni, B. Schuller, D. Mallmann, B. Tweddell, A. Gianoli, V. Ciaschini, S. van de Berghe, D. Snelling, A. StreitUsing SAML-based VOMS for Authorization within Web Services-based UNICORE Grids.Proceedings of 3rd UNICORE Summit 2007 in conjunction with EuroPar 2007, Rennes, France, to appear [6] R. Ratering, A. Lukichev, M. Riedel, D. Mallmann, A. Vanni, C. Cacciari, S. Lanzarini, P. Bala, K. Benedyczak, M. Borcz, R. Kluszcynski, and G. Ohme,GridBeans: Supporting e-Science and Grid Applications.In Proc. of the 2nd IEEE International Conference on e-Science and Grid-Computing (e-Science 2006), IEEE Computer Society Press, Amsterdam, NL, December 4-6, 2006, ISBN: 0-7695-2734-5, proceedings on CD http://www.unicore.eu

More Related