1 / 9

coordinator: M. Sgaravatto (INFN), deputy A. Kryukov (SINP MSU), G. Shpiz (SINP MSU),

Improved LCG Job Submission Current status of T1.1 Subtask: Advanced algorithms and tools for resource allocation and job scheduling. coordinator: M. Sgaravatto (INFN), deputy A. Kryukov (SINP MSU), G. Shpiz (SINP MSU), L. Shamardin (SINP MSU).

macaulay-roberts
Download Presentation

coordinator: M. Sgaravatto (INFN), deputy A. Kryukov (SINP MSU), G. Shpiz (SINP MSU),

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Improved LCG Job SubmissionCurrent status of T1.1 Subtask:Advanced algorithms and tools for resource allocation and job scheduling coordinator: M. Sgaravatto (INFN), deputy A. Kryukov (SINP MSU), G. Shpiz (SINP MSU), L. Shamardin (SINP MSU) Presented by A. Demichev CERN-INTAS meeting, 14 March 2005, CERN

  2. T1.1 : Advanced algorithms and tools for resource allocation and job scheduling • Improved LCG Job Submission Architecture • Presented at LCG Deployment group meeting and recommended as a useful tool for LCG users • Proposed approach allows to improve RB's robustness and performance • no modifications in existing LCG components • use of the standard grid protocols only • use of the standard grid security infrastructure CERN-INTAS meeting, 14 March 2005, CERN

  3. EDG/LCG Architecture • Shortcoming: • Resource Broker mainly fulfills two tasks: • finding a computer resource satisfied job requirements • transportation of all necessary files to start a job thus Resource Broker is a bottleneck • Suggestion: Improved LCG job flow • Similar to DIRAC (LHCb) architecture but uses only standard grid (LCG) protocols CERN-INTAS meeting, 14 March 2005, CERN

  4. EDG/LCG job flow CERN-INTAS meeting, 14 March 2005, CERN

  5. EDG/LCG job flow (cont.) Typical session in CMS – CMKIN: number of jobs/per session ~ 500 size of an executable is about 20MB Looks like a DoS attack! CERN-INTAS meeting, 14 March 2005, CERN

  6. Improved LCG job flow CERN-INTAS meeting, 14 March 2005, CERN

  7. API lcg-job-submit prepare tar-file in accordance JDL transfer tar-file to SE modify original JDL submit modified JDL by edg-job-submit logging information to ~/.lcg-job-submit-log local-job-wrapper transfer tar-file from SE untar tar-file start original executable CERN-INTAS meeting, 14 March 2005, CERN

  8. API (cont.) lcg-se-clear remove tar-file from SE (for example, if job was aborted for some reasons) lcg-se-test check the possibilities to store file on the SE TRANSMIT_SE environment variable defined SE to transfer data CERN-INTAS meeting, 14 March 2005, CERN

  9. Subtask T1.1: Future plans realize job buffering to reduce wall-time of job submission; using single copy of executables for multiple submission CERN-INTAS meeting, 14 March 2005, CERN

More Related