1 / 9

LQCD Workflow Project

LQCD Workflow Project. L. Piccoli October 02, 2006. Triana. Triana. public class Job extends Unit { // parameter data type definitions private String queue; private int nodes; /* * Called whenever there is data for the unit to process */

penny
Download Presentation

LQCD Workflow Project

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. LQCD Workflow Project L. Piccoli October 02, 2006

  2. Triana

  3. Triana public class Job extends Unit { // parameter data type definitions private String queue; private int nodes; /* * Called whenever there is data for the unit to process */ public void process() throws Exception { // Implementation } }

  4. Triana • Integration with existing system • Control and data dependencies represented • Substitute for current run makefile tool • Creation of toolboxes for generic jobs • Not as flexible • Workflows can be turned into command line applications • No Web Services or Grid capabilities used

  5. Triana • Requirements • Workflow representation • XML containing Triana specific tags • Interaction with external scheduler • Possible via task-by-task schedule • As dependencies are resolved, the execution request is passed to campaign scheduler • Schedule of complete workflow requires significant changes • Campaign scheduler is in charge of resolving dependencies

  6. Triana • Requirements • Scheduling (one or more workflows) • Not present • Modifications: • Submit workflows to an external scheduler • Update workflow progress based on external monitoring information • Fault Management • Not present • Triana services may be used for adding these features • Need further investigation

  7. Triana Summary

  8. System Architecture

  9. This workflow represents the dependencies found in the charm and bottom 2-pt run file (available at http://whcdf03.fnal.gov/lqcd/RunWorkflowFramework). There are pseudo dependencies in this graph (e.g. hQ*923* do not depend on hQ*127*), forcing the system to execute tasks in an order that fits better to the available hardware. This optimization is manual and requires knowledge of the system by the user. The workflow scheduler should be able to handle this situation automatically.

More Related