1 / 25

User Interaction & Workflow Management in Grid-enabled e-VLBI Experiments

This paper discusses the user interaction and workflow management in grid-enabled e-VLBI (electronic Very Long Baseline Interferometry) experiments, highlighting the benefits of distributed correlation and the use of GRID resources. It outlines the process of creating an observation workflow, submitting it for execution in the grid environment, and monitoring the experiment's progress. The paper also presents results from a correlation testbed and discusses the relationship between correlation time and chunk size.

lchildress
Download Presentation

User Interaction & Workflow Management in Grid-enabled e-VLBI Experiments

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Dominik Stokłosa Poznań Supercomputing and Networking Center, Supercomputing Department User interaction and workflow management in Grid enabled e-VLBI experiments

  2. Outline • Motivation for distributed correlation • E-VLBI System in practice • E-VLBI system behind the scene • Summary

  3. Data can be transferred over the network Each stage of the process can be speeded up GRID resources available Software correlator available First glance at distributed correlation

  4. E-VLBI System

  5. E-VLBI System – communication

  6. How to conduct an experiment Creating observation workflow using WFM application Submitting observation workflow for execution in the Grid environment Workflow execution Monitoring the VLBI experiment

  7. Observation workflow Constructed: • by VLBI operator • based on the observation schedule (VEX) • with Workflow Manager Application (WFM)

  8. Observation workflow

  9. Submitting the observation workflow

  10. Workflow execution Divided into the following phases: • Chunking data from radio telescopes • Submitting chunked data sets for correlation • Archiving the correlated data sets

  11. (1) Workflow execution: chunking

  12. (1) Workflow execution: chunking

  13. (1) Workflow execution: preparing environment

  14. (1) Workflow execution: sendingchunks

  15. (2) Workflow execution: correlation • First data chunks are available • Step, where the real correlation takes place • Computation on the Grid resources

  16. (2) Workflow execution: correlation

  17. (2) Workflow execution scheduling correlation jobs with Correlation Node ..

  18. (2) Workflow execution: correlation

  19. (3) Workflow execution: archive • First data sets have been correlated • Store correlated data with Correlated Data Service (CDS) • Clean up the Grid resources

  20. (3) Workflow execution: archive

  21. (3) Workflow execution: archive

  22. Workflow monitoring • Possibility to monitor the distributed correlation • Monitoring perspective in WFM application

  23. Fabric results • 25 minutes of data • Successful correlation with 2 clusters (PSNC) • One File Server (Jive) • Stations: Westerbork, Medicina, Torun, Cambridge

  24. Correlation testbed

  25. Correlation time vs chunk size

More Related