1 / 15

Overview

hide middleware heterogeneity (e.g. gLite, Globus, Unicore). JDL. RSL. Overview. job desc. JSAGA. gLite plug-ins. Globus plug-ins. hide middleware heterogeneity (e.g. gLite, Globus, Unicore). EGEE. OPlast. delegate selection & files staging. WMS. SRM. input data.

baker-york
Download Presentation

Overview

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. hide middleware heterogeneity (e.g. gLite, Globus, Unicore) JDL RSL Overview job desc. JSAGA gLite plug-ins Globus plug-ins JSAGA

  2. hide middleware heterogeneity (e.g. gLite, Globus, Unicore) EGEE OPlast delegate selection & files staging WMS SRM input data hide grid infrastructures heterogeneity (e.g. EGEE, OSG, DEISA) GridFTP LCG-CE LCG-CE WS-GRAM WS-GRAM firewall job job Overview job desc. JSAGA gLite plug-ins Globus plug-ins JDL RSL staging graph JSAGA

  3. hide middleware heterogeneity (e.g. gLite, Globus, Unicore) hide grid infrastructures heterogeneity (e.g. EGEE, OSG, DEISA) What is JSAGA ? JSAGA is… A Java implementation of the SAGA specification An API for submitting a collection of jobs to several grid infrastructures with a single job description JSAGA

  4. For end-users: ready-to-use software adapted to their needs For application developers: hide infrastructure heterogeneity For application developers: hide middleware heterogeneity For plug-in developers: As many interfaces as ways to implement the functionalities Applications Job Collection SAGA Plug-ins interfaces Global architecture JSAGA

  5. done construction planned Provided plug-ins Security X509 SSH InMemCred Globus G. Legacy G. RFC820 MyProxy VOMS Login / pwd Data catalog rns lfn srb / irods http https sftp rbyteio file zip gsiftp tar ftp mail cache srm Exec. (control) Exec. (monitor) Job control gatekeeper gLite-WMS wsgram unicore6 ssh local cream gridvm PBS remote gatekeeper gLite-LB wsgram unicore6 ssh local cream … Expression Language basic default JEP BeanShell JSDL+ext. SAGA JDL RSL-2 RSL-4 JSAGA

  6. Hide middleware heterogeneity Uniform interface (SAGA) Hide infrastructure heterogeneity Selection of the right security context Selection of the right client API configuration Transport of the job input/output data to/from worker nodes Describe the capabilities of the infrastructures you want to use VOMS VOMS Globus Describe grid infrastructures requirements CC-IN2P3 WMS gsiftp:// EGEE wsgram OpenPlast Grid World localhost JSAGA

  7. Hide middleware heterogeneity Job management technologies e.g. CREAM, WMS, SSH, GK Hide infrastructure heterogeneity Computing Elements (e.g. GK) Different grid or site policies e.g. network filtering, shared FS Different environment variables e.g. $VO_?_SW_DIR, /usr/local Different configuration attributes e.g. monitor service URL, shell path on cygwin, default SE URL Commands available on worker e.g. globus-url-copy, srmcp, Scp, wget, tar VOMS VOMS Globus Describe grid infrastructures examples gatekeeper srb:// srm:// CC-IN2P3 lfn:// WMS gsiftp:// EGEE gatekeeper wsgram OpenPlast Grid http:// tar:// gatekeeper World localhost JSAGA

  8. VOMS gatekeeper srb:// srm:// CC-IN2P3 lfn:// WMS gsiftp:// EGEE VOMS gatekeeper wsgram OpenPlast Grid http:// Globus tar:// gatekeeper World localhost plug-ins Transfer path depends on… • Using a single grid • all files can be transferred to the worker through a single node • Using several grids • need to dynamically build a transfer graph, according to… url:// job desc. JSAGA JSAGA

  9. VOMS gatekeeper srb:// srm:// CC-IN2P3 lfn:// WMS gsiftp:// EGEE VOMS gatekeeper wsgram OpenPlast Grid http:// tar:// Globus gatekeeper localhost World plug-ins Transfer path depends on… • grid or site • network filtering policy • commands available on workers • services available from workers (close Storage Element, shared FS) • supported context instances • data to stage • shared by several jobs • installed on some worker nodes • file size • required data protection level • execution service • protocols supported for staging • transfer protocol • access mode (RO, WO, RW) • third-party transfer • supported data protection level url:// job desc. JSAGA JSAGA

  10. C C' common result R1 std-error E1 R1 C' job C SMTP SRB GSIFTP GSIFTP HTTP job OPlast EGEE CA OPlast OPlast OPlast OPlast OPlast E1 job VOMS gatekeeper srb:// srm:// CC-IN2P3 lfn:// WMS gsiftp:// EGEE VOMS GSIFTP gatekeeper wsgram OpenPlast OpenPlast Grid http:// tar:// Globus gatekeeper localhost World OpenPlast Transfer path depends on… • grid or site • network filtering policy • commands available on workers • services available from workers (close Storage Element, shared FS) • supported context instances • data to stage • shared by several jobs • installed on some worker nodes • file size • required data protection level • execution service • protocols supported for staging • transfer protocol • access mode (RO, WO, RW) • third-party transfer • supported data protection level JSAGA

  11. data to stage shared by several jobs installed on some worker nodes file size required data protection level grid or site network filtering policy commands available on workers services available from workers (close Storage Element, shared FS) supported context instances E E src executable input data D1 SMTP SRB GSIFTP GSIFTP HTTP R1 C' D1 job C GSIFTP job OPlast EGEE CA OPlast OPlast OPlast OPlast OPlast TAR TAR E1 job C C' C'' common Transfer path depends on… result R1 • execution service • protocols supported for staging • transfer protocol • access mode (RO, WO, RW) • third-party transfer • supported data protection level std-error E1 E src C" E iGet JSAGA

  12. E E src executable input data D1 OPlast OPlast C C' C'' common Example of generated graph result R1 std-error E1 Data flow several protocols used, but only 3 jobs submitted on 1 grid… JSAGA

  13. Conclusion • Assets of JSAGA • does not sacrifice scalability and efficiency for abstraction • thanks to design (in particular the definition of plug-in interfaces) • thanks to caching mechanisms • does not only hide heterogeneity of middleware, but also hides heterogeneity of grid infrastructures • thanks to the description of used grid infrastructures (configuration file) • thanks to the information provided by the plug-ins • thanks to the information provided by the user in his job description http://grid.in2p3.fr/jsaga/ JSAGA

  14. Applications Related projects • JSAGA is used by… • Elis@ • a web portal for submitting jobs to industrial and research grid infrastructures • JJS (Java Job Submission) • a tool for submitting job collections to EGEE • optimized for short-life jobs (resource selection based on QoS observed while submitting jobs) • JUX (Java Universal eXplorer) • a multi-protocol file browser / JSAGA

More Related