1 / 11

Project: C3-Grid Collaborative Climate Community Data and Processing Grid

Project: C3-Grid Collaborative Climate Community Data and Processing Grid. Scheduling in C3-Grid: Requirements, Concepts, Realization Christian Grimme, Alexander Papaspyrou. Outline. Why C3-Grid? Applications Goals Architecture Applied technologies

vgilstrap
Download Presentation

Project: C3-Grid Collaborative Climate Community Data and Processing Grid

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Project: C3-GridCollaborative Climate Community Data and Processing Grid Scheduling in C3-Grid: Requirements, Concepts, Realization Christian Grimme, Alexander Papaspyrou

  2. Outline • Why C3-Grid? • Applications • Goals • Architecture • Applied technologies • Scheduling- and data management-aware workflows • Orchestration of workflow components • Implementation details

  3. Why C3-Grid? • Applications: • Earth System Science investigates anthropogenous influence on climate • Analysis of simulation data and field measurements • Huge amounts of inhomogenous data distributed over various sites • Goals: • Development of a Grid-based infrastructure for • Data discovery • Data processing • Data analysis

  4. Why C3-Grid? Simulation Data different format

  5. Architecture User User Interface API (Web Services) GUI Monitoring Job Submission Distributed Grid Infrastructure Nutzer Search Workflow Scheduler DMS (global) DIS Matchmaking ResourceInformationService Staging Data Transfer Service Harvesting Task Execution Grid Workspace SiteGrid Infrastructure File Management Data Archive DMS (local) Resource Scheduler Pre-Processing Base Data & Meta Data Data Job Meta Data ArchiveInterface AvailableResources DistributedData Archives DistributedProcessing Resources DBMS/File

  6. Applied Technologies • Gridsphere as framework for portal (UI) services • Grid core services with Globus Toolkit 4 • Web services and WSRF • GSI and Shibboleth for security • JSDL and home-grown Workflow Specification Language (WSL)

  7. Scheduling- and Data Management-aware Workflows • Jobs • Single (atomic) execution units • Execution in one step • Input data locally available • Output data stored locally • Files • Datasets in workflow, that • are created new (output) • are used for job execution (input) • possibly reusable

  8. Workflow: Restrictions Data Job Data Data Data Job Job Job Data Data … X … Data Job ? Data X Data … Allowed workflow: • Prohibited elements in a workflow: • Circles, back references • Conditionals

  9. Workflows: Orchestration Portal Selection- Wizard Data Job Data Data Data Job Job Workspace Results 1 DIS 2b 2a WSL Harvester 3 4 Meta- Data DMS Scheduler 6 8 7 5 Transfer WS GRAM HTTP Daemon 9 Workspace HH Köln Data Provider

  10. Details on Scheduling and Data Management OfferExe OfferData WS-GRAM Resource Scheduler Resource Management Exe + Data input Portal Interface Scheduler Info Service DMS MDS D Sites & Programs virtual Work- space Staging Infos & Functionality Interface to Data Provider RM RM RM Computing Resources

  11. WSL: Workflow Specification Language

More Related