1 / 4

LEAD-VGrADS Meeting

Explore the integration of LEAD and VGrADS through DAG, annotated constraints, workflow configuration, and service adaptation for efficient job execution. Understand the resource allocation, scheduling, and batch queue prediction for optimized performance.

justinhaynes
Download Presentation

LEAD-VGrADS Meeting

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. LEAD-VGrADS Meeting Day 1- PM

  2. Maybe more closely tied together DAG DAG + Annotated Constraints Performance Model Workflow Configuration Service App. Scheduler Portal LEAD Adaptation - VGrADS Trans-lation Layer (for right Now) Workflow LEAD ADAPTATION Create Services DAG + Annotated Concrete Set of Resources AND Virtual Grid LEAD ADAPTATION LEAD BPEL Workflow Engine App. Factory Launch Services Application Service (per task) Run job Virtual Grid with batch queue prediction Job notification Run workflow one step at a time Resources vgLaunch Current vgES vgStatus (from GRAM) Workflow and File Status myLEAD (subscribes to messages from the broker and knows what magic to do with input/output files and talks to RLS/DRS Event Broker Performance Model LEAD ADAPTATION Batch Queue Prediction LEAD ADAPTATION

  3. Notes/Questions for PIs • 2 logical places where LEAD interacts with VGrADS • We don’t schedule where the application services run today! Long term might (See Lavanya’s old vgDL example –last slide) • vgES needs to support GRAM, queues, status • The adaptation arrow back to LEAD will take care of non-concrete set of resources • What is our target resource set? (TeraGrid machines, other VGrADS sites?) • The resources are not completely “virtual”. Is that okay? • Data management, metadata management, replica management, providence collection • Long term implications for vg

  4. LEADSpec = LDMNode = {isLDMNode} // Note the loose coupling far WRFNode = {memory >= 500MB, cpu > 2000, diskspace > 4GB} far VizNode = {memory >=4GB, cpu > 4000} vgidLEAD = vgCreateVG(vgESsrv.renci.org, LEADspec, 1000, null); vgRoot = vgGetRoot(vgidLEAD); Provisioning 31 1 2 3 0 … 255 10 … 0 1 2 … 2 0 1 vgDL Specification for LEAD vgRoot WRFNode VizNode LDMNode WRFspec = LooseBagof<C> [1:32]; C = Clusterof<node>[4:256]; node = {node.memory > 500MB, node.cpu > 2000}; C = {C.hasSharedFileSystem = true} vgidLEAD = vgGetMyVG(); wrfNode = vgGetMyNode(); status = vgAddToVG(vgidLEAD, WRFspec, wrfNode,1000, init_WRF);

More Related