Legacy code support for commercial production Grids

1. Legacy code support for commercial production Grids Extending GEMLCA to fulfil requirements of business grids G.Terstyanszky, T. Kiss, T. Delaitre, S. Winter School of Informatics, University of Westminster P. Kacsuk, G Kecskemeti MTA SZTAKI Laboratory of Parallel and Distributed Systems, Hungary

2. Presentation Outline • GEMLCA (Grid Execution Management for Legacy Code Applications) concept • GEMLCA requirements towards commercial production Grids • GEMLCA Extensions • Dynamic User Management • Resource & Service Monitoring • Accounting and Charging • Conclusion and future work

3. GEMLCA Concept Invoke legacy code Send custom input Compute Servers Client 2: to invoke legacy code Grid service GEMLCAresource Return result Produce result • Resource managerdeploys: • LC • GEMLCA Client 1: to registerlegacy code as a Grid service

4. The GEMLCA-view of a legacy code • Any code that correspond to the following model can be exposed as Grid service by GEMLCA: Legacy code Input data (files, command line params, env. vars) Output data (files) . . . Perform computation Call shared libraries Communicate over the network Query databases

5. What’s behind the GEMLCA resource… • Consequences: • Not only the legacy code and the GEMLCA service, but also a local job manager and a Grid middleware layer must be installed on the hosting system! • The aim of the code registration process is to tell GEMLCA how to submit the legacy code to the grid middleware layer Compute GEMLCA Service Gridmiddlewarelayer Client Servers Grid Service Client GEMLCA Job submission and file transfer services Process Legacy Local jobmanager Code Job e.g. Condor/Fork

6. 3rd generation Grids: (OGSA: GT4, gLite) Web browser Grid Site 1 LC 1 Result Desktop 1 Legacy applications Grid Site 2 Web browser LC 2 Input Legacy applications Desktop N . . . The GEMLCA P-GRADE Portal GEMLCAP-GRADE Portal Server

7. GEMLCA on the UK NGSThe NGS P-GRADE GEMLCA Portal • Runs as a service available for all NGS users • Portal Website:http://www.cpc.wmin.ac.uk/ngsportal/

8. GEMLCA and business Grids Requirements • dynamic user management  users’ accounts created automatically grid-mapfile modified automatically • monitoring of GEMLCA & Grid resources  information about previous and current state of Grid resources & services - accounting/charging of resource/service usage  resource and service usage should be metered and charged accordingly

9. Dynamic User Management • need for dynamic user management • dynamic and scalable management of several hundred or thousand users without manual interactions • dynamic user management solutions • identity mapping (i.e. mapping Grid certificates) to local accounts • providing local accounts on demand • user management implementation • Local Credential Mapping Service (LCMAPS) to map credentials to local account • Workspace Management Service (WMS) to allocate workspace (or local account) to users

10. returning a workspace Web browser P-GRADE Portal Server Desktop 1 Legacy applications + default input Web browser abstract workflow Grid Site Desktop n Dynamic User Management Service Identity Mapping (LCMAPS) Workspace Management Service account pool

11. Grid Monitoring Service • Need for Monitoring • checking availability of resources/services • generating historical availability data • classifying availability of resources/services & making this data available for brokers • Monitoring scenarios • MDS initiated regular monitoring • user initiated monitoring • Monitoring Implementation • probes are implemented by scripts and their outputs are displayed in a monitoring portlet

12. Grid Monitoring Service • Grid resource probes: • network connectivity of a remote host • access to a remote MyProxy server • file transfer using a remote host GridFTP server • Globus job submission (WS-GRAM) • availability of the local information system (MDS service) • local job manager (Condor, PBS, SGE etc.) • GEMLCA services probes: • GLCAdmin,  GLCList •  GLCProcess

13. MDS4 GMT (GEMLCA Monitoring Toolkit) Grid site with legacy code resources User Grid site with legacy code resources Grid Monitoring Service Architecture Production Grid (e.g. EGEE, NGS) 3rd party service providers Run GMT probes and collect data Portal Map execution

14. On-demand execution as a prototype Grid Monitoring Service in the P-GRADE portal Automatic execution Already works as a service on the NGS P-GRADE portal

15. MDS4 Store historical data Historical test data GMT (GEMLCA Monitoring Toolkit) Run GMT probes and collect data Prediction based on current and historical data Grid site with legacy code resources User Map execution Submit workflow Resource availability Grid site with legacy code resources Map execution Current work - Resource testing and availability prediction Production Grid (e.g. EGEE, NGS) 3rd party service providers Portal GMT classifier Grid broker (e.g. LCG broker)

16. Grid Accounting & Charging Service • Need for accounting/charging • automatic resource/service usage accounting • managing multiple charging policies & implementations • creating a database to store both accounting & charging data • Accounting/charging architecture • create a Grid Bank to manage all transactions related to accounting & charging • Grid Accounting & Charging service (GAC) based on the GGF’s RUR specification

17. Grid Accounting & Charging Service Resource Charge Record (RCR) Resource Usage Record (RUR)

18. RCR collecting usage data RUR RUR Abstract workflow Grid Accounting & Charging Service Charging Service database Grid Site1 Web browser Accounting Service P-GRADE Portal Server Desktop 1 Legacy applications + default input Broker Grid Sitem Web browser Desktop n Legacy applications + default input

19. Conclusions and Future Work • dynamic user management • installed on the GT4-based WestFocus Grid • will be used in the NGS GT4 testbed • resource/service monitoring • automatic execution installed on the NGS P-GRADE portal • On-demand execution under development & testing • accounting/charging • - under development & testing

20. Any questions? Thank you for your attention … Email: gemlca-discuss@cpc.wmin.ac.uk Websites: www.cpc.wmin.ac.uk/gemlca www.lpds.sztaki.hu/pgportal www.cpc.wmin.ac.uk/ngsportal GEMLCA/P-GRADE portal demonstrations on the GOSC booth during the AHM!