1 / 34

Grid Execution Management for Legacy Code Applications

Grid Enabling Legacy Applications. Grid Execution Management for Legacy Code Applications. Tamas Kiss Centre for Parallel Computing University of Westminster London, UK. Legacy Applications. Code from the past, maintained because it works Often supports business critical functions

nita-meyer
Download Presentation

Grid Execution Management for Legacy Code Applications

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Grid Enabling Legacy Applications GridExecutionManagementforLegacyCodeApplications Tamas Kiss Centre for Parallel ComputingUniversity of WestminsterLondon, UK

  2. Legacy Applications • Code from the past, maintained because it works • Often supports business critical functions • Not Grid enabled What to do with legacy codes when utilising the Grid? • Bin them and implement Grid enabled applications • Reengineer them • Port them onto the Grid with minimum user effort

  3. GEMLCA – Grid Execution Management for Legacy Code Architecture Objectives • To deploy legacy code applications as Grid services without reengineering the original code and minimal user effort • To create complex Grid workflows where components are legacy code applications • To make these functions available from a Grid Portal GEMLCA GEMLCA PGPortal Integration

  4. GEMLCA Concept Invoke legacy code Send custom input Compute Servers Client 2: to invoke legacy code Grid service GEMLCAGrid service Return result Produce result • Resource managerdeploys: • LC • GEMLCA Client 1: to registerlegacy code as a Grid service

  5. The GEMLCA-view of a legacy code • Any code that correspond to the following model can be exposed as Grid service by GEMLCA: Legacy code Input data (files, command line params, env. vars) Output data (files) . . . Perform computation Call shared libraries Communicate over the network Query databases

  6. GEMLCA Concept Invoke legacy code Custom input Compute Servers Client 2: to invoke legacy code Grid service GEMLCAGrid service Return result Produce result • Resource managerdeploys: • LC • GEMLCA Client 1: to registerlegacy code as a Grid service Code is deployed statically Legacycode Input data is provided dynamically Access to the code is controlled by the GEMLCA Grid service

  7. Implementing the concept • The GEMLCA service can be implemented with any grid/service-oriented technology E.g: • Globus 3 or 4 currently available implementations • Jini • Web services • … • GEMLCA service could invoke legacy codes in many different ways. Current implementation: • Submit the legacy code as a batch job to a local job manager (e.g. Condor or PBS) through a Grid middleware layer (GT3/4)

  8. What’s behind the GEMLCA service… • Consequences: • Not only the legacy code and the GEMLCA service, but also a local jobmanager and a Grid middleware layer must be installed on the hosting system! • The aim of the code registration process is to tell GEMLCA how to submit the legacy code to the grid middleware layer Compute GEMLCA Service Gridmiddlewarelayer Client Servers Grid Service Client GEMLCA Job submission and file transfer services Process Legacy Local jobmanager Code Job e.g. Condor/Fork

  9. What’s the point? • Heterogeneous codes can be hidden behind the same interface (the programming interface of the GEMLCA service) • Different programs can be invoked in the same way • Extend non grid-aware programs with security infrastructure (access enabled through a Grid service) • Share your codes with your colleagues or partner institutes • Expose business logic to your employees or customers • Build customized GEMLCA clients (such as the GEMLCA P-GRADE Portal) • Compose complex processes by connecting multiple legacy code grid services together

  10. The GEMLCA P-GRADE Portal A Web-based GEMLCA client environment… University of Westminster, LondonMTA SZTAKI, Budapest

  11. The aim of the GEMLCA P-GRADE Portal • To provide graphical clients to GEMLCA with a portal-based solution • To enable the integration of legacy code grid services into workflows

  12. 3rd generation Grids: (OGSA: GT4, gLite) Web browser Grid Site 1 LC 1 Result Desktop 1 Legacy applications Grid Site 2 Web browser LC 2 Input Legacy applications Desktop N . . . The GEMLCA P-GRADE Portal GEMLCAP-GRADE Portal Server

  13. The GEMLCA-specific version of the P-GRADE Portal is different from the original P-GRADE Portal! • It contains a web page to register legacy codes as grid services • It contains a GEMLCA-specific workflow editor • Workflow components can be “legacy code grid services” (not only batch jobs) • It contains a GEMLCA-specific workflow manager subsystem • It can invoke GEMLCA services (not only submitting jobs)

  14. Legacy code registration page “GEMLCA Administration Tool” portlet

  15. Legacy code registration page

  16. GEMLCA Specific Workflow editor UoW Sztaki UoR

  17. GEMLCA workflow editor in a nutshell Workflow Creation UoW Sztaki UoR

  18. User provides the binary executable User provides input data for the executable Job produces result files Executable is already installed on a machine User provides input data for the executable Job produces result files Batch components vs. GEMLCA components in P-GRADE Portal workflows Batch component GEMLCA component Workflow components must be defined in different ways Input files represented by ports Output files represented by ports Ports guarantee compatibility  batch and GEMLCA components can mutually produce data to each other!

  19. Combine legacy codes with new codesinside the same workflow! Combining legacy and non-legacy components Code invocation Code invocation Job submission Code invocation Job submission

  20. Application example Designing Optimal Periodic Nonuniform Sampling Sequences T Factor Sequential GEMLCA 18 ~19min ~8min 20 ~3h 33min 35min 22 ~41h 53min ~7h 23min

  21. I’m a biologist not a Grid expert Sorry, no additional utilities to be deployed on core resources ??? UK NGS Leeds Oxford • Service reliability • Administration GT2 Manchester Rutherford GEMLCA and Production Grids The problem

  22. User-friendly Web interface UK NGS Leeds Oxford GT2 Manchester Portal server at UoW Rutherford The solution

  23. 2nd generation Grids Web browser Grid Site 1 Desktop 1 Grid Site 2 Web browser Desktop N Centralised GEMLCA solution – The GEMLCA repository (GEMLCA-R) P-GRADE Portal Server

  24. GEMLCA-R 3rd party service provider (UoW) GEMLCA resource (GT4 + GEMLCA classes) NGS site1 (GT2) Workflow definition NGS site2 (GT2) P-Grade Portal Central repository legacy code1 legacy code2 …. legacy coden NGS siten (GT2) user job submission

  25. Advantages of GEMLCA-R • legacy codes can be uploaded into a central repository and made available for authorised users through a Grid portal • extends the usability of production Grids as users utilise others’ legacy codes stored in the repository • No support needed at the Grid sites

  26. Westminster GEMLCA on the UK NGSThe P-GRADE NGS GEMLCA Portal • Portal Website:http://www.cpc.wmin.ac.uk/ngsportal/ • Runs both GT4 GEMLCA and GEMLCA-R

  27. GEMLCA on the WestFocus GridAlliance Grid • GT4 testbed for industry and academia • Connects two 32 machine clusters at Westminster and one at Brunel University • Runs the P-GRADE Grid portal and GEMLCA • Connected to and interoperable with the UK NGS

  28. Connecting Grids with the P-GRADE Portal and GEMLCA • Extending current second generation production Grids with GT4 resources – already in production on the UK NGS • Connecting GT2 Grids (e.g. NGS) to GT4 Grids (e.g. OpenScience Grid, WestFocus Grid) • Connecting GT2 Grids (e.g. NGS) to the LCG/gLite-based EGEE Grid

  29. UoW Portal Server P-GRADE NGS GEMLCA Portal Executable Executable User Executable Executable Executable Executable Service Invocation UoW – GT4 GEMLCA Service Extending the NGS with GT4 Resources Manchester Westminster Oxford GEMLCA Repository

  30. UoW Portal Server P-GRADE NGS GEMLCA Portal Executable Executable User WestFocus GT4 Grid Executable Leeds Executable UoW Brunel Connecting the NGS to GT4 Grids NGS / GT2 Manchester Service Invocation GEMLCA Repository

  31. UoW Portal Server P-GRADE NGS GEMLCA Portal Executable Executable Executable User EGEE/VOCE WestFocus GT4 Grid Service Invocation Leeds Poznan Executable UoW Budapest Brunel Connecting the NGS to EGEE NGS Manchester GEMLCA Repository

  32. EGEE WestFocus Grid NGS Connecting the NGS to EGEE GT2 Job submission to Poznan GEMLCA legacy code submitted to Manchester GT4 Service Invocation at UoW GT2 Job submission to Oxford

  33. Portal service for connecting all major production Grids

  34. Conclusions • GEMLCA enables the deployment of legacy code applications as Grid services without any real user effort. • GEMLCA is integrated with the P-GRADE portal to offer user-friendly development and execution environment. • The integrated GEMLCA P-GRADE solution is available for the UK NGS as a service! • www.cpc.wmin.ac.uk/ngsportal

More Related