Supporting legacy code applications on EGEE VOs by GEMLCA and the P-GRADE portal

1. Supporting legacy code applications on EGEE VOs by GEMLCA and the P-GRADE portal P. Kacsuk*, G. Sipos* and T. Kiss** *MTA SZTAKI ** Univ. of Westminster

2. Properties of the P-GRADE Portal • What is P-GRADE portal? • General purpose, graphical, workflow-oriented Grid portal • Supports the development and execution of workflow-based Grid applications • Components of the workflows can be • Sequential jobs • Parallel jobs (MPI, PVM) • Legacy code (GEMLCA) services • Easy porting of legacy and other applications to the Grid • Enables the exploitation of two levels of parallelism • intra-job • inter-job • Enables to control and observe the execution of the workflow applications • Enables fault-tolerant workflow execution

3. Properties of the P-GRADE Portal • Enables application-specific extensions by new portlets (E-GRID, Traffic simulation Grid, etc.) • Hides the low-level Grid details • Enables the access of every important feature of the underlying Grid, e.g.: • brokering service • information service, etc. • Can serve as a bridge between different Grids to solve the Grid interoperability problem at the workflow level: • Enables easy porting of legacy and other applications between various Grids (e.g. from LCG to gLite, from GT2 to GT4, etc.) • Hides middleware changes for the end-user • Enables parallel execution of a workflow in several Grids

4. Properties of the P-GRADE Portal • Enables application-specific extensions by new portlets (E-GRID, Traffic simulation Grid, etc.) • Hides the low-level Grid details • Enables the access of every important feature of the underlying Grid, e.g.: • brokering service • information service, etc. • Can serve as a bridge between different Grids to solve the Grid interoperability problem at the workflow level: • Enables easy porting of legacy and other applications between various Grids (e.g. from LCG to gLite, from GT2 to GT4, etc.) • Hides middleware changes for the end-user • Enables parallel execution of a workflow in several Grids

5. Proxy management Grid resources management Workflow creation Job mapping to Grid resources Workflow management and execution visualization P-GRADE portal in a nutshell

6. Parallel execution inside a workflow node • Parallel execution among workflow nodes Multiple jobs can run parallel The job can be a parallel program Two levels of parallelism by the P-GRADE workflow • Semantics of the workflow enables two levels of parallelism: • The P-GRADE Portal workflow concept enables the efficient parallelization of complex problems

7. OGSA-DAI Integration with P-GRADE GridExecutionManagementforLegacyCodeApplications Tamas Kukla and Tamas Kiss Centre for Parallel Computing University of Westminster T.Kukla@student.westminster.ac.uk, kisst@wmin.ac.uk

8. OGSA-DAI portlets 4 portlets implemented • Data Service Manager • Database Browser • Query Manager • Data Manipulation

9. Data Service Manager portlet Add/remove OGSA-DAI services List available resources of a service List available tables Select and describe tables

10. Database Browser portlet Select service / resource / table Display the selected table or XML resource Order by the selected column

11. Display results Display results Query Manager portletDisplay results on screen Run query on selected database See next slide

12. Query Manager portletDeliver results to file Define query statement Define target location Data can be sliced up and compressed before delivery

13. Data Manipulation portletManipulate data of selected database Select database Define update statement Update See next slide

14. Data Manipulation portletDeliver results from file to database Select target service / resource Define target table Define files to be delivered Extracts and merges

15. A possible user scenario • The new OGSA-DAI portlet allows to browse an OGSA-DAI database and deliver query results to a set of files. • These files can be used in normal or parameter study workflows as input ports. • The workflow output can be converted and transferred back to the database. • Select the data service and resource • Run a query and deliver results to a (set of) file(s) • Run the (parameter study) workflow on the data-sets • Transfer the results back into an OGSA-DAI database

16. Step1: Select the OGSA-DAI service and DataBase resource with the DataBrowser eXist XMLDataBaseresource Storage resource (local or remote) User OGSA-DAIservice Request List of resources Request Extended PG-Portal Oracle DataBaseresource Workflow DataBrowser List of resources

17. Step1: Select the OGSA-DAI service and DataBase resource with the DataBrowser Select service and resource

18. Step2: Run a query and deliver results to a (set of) file(s) eXist XMLDataBaseresource Storage resource (local or remote) User OGSA-DAIservice Query Results Query Query Extended PG-Portal Oracle DataBaseresource Workflow DataBrowser Results Results

19. Step2: Run a query and deliver results to a (set of) file(s) Define query statement Deliver results to a set of files

20. Step3: Run (parameter study) workflow (this step does not use the OGSA-DAI portlets!) eXist XMLDataBaseresource Storage resource (local or remote) User Input data OGSA-DAIservice Create workflow Workflow output Extended PG-Portal Oracle DataBaseresource Workflow DataBrowser

21. Step4: Transfer results back into an OGSA-DAI database eXist XMLDataBaseresource Storage resource (local or remote) User Workflow output Workflow output OGSA-DAIservice Request Confirmation Confirmation Workflow output Extended PG-Portal Oracle DataBaseresource Workflow DataBrowser Confirmation

22. Step4: Transfer results back into an OGSA-DAI database Define target database Define target table Define files to be transferred and converted