Grid testbed

1. Grid testbed Deploying and testing the Globus toolkit Andrei.Chevel@pnpi.spb.ru

2. Projects for distributed computing in science • Main goal of all projects: • To realize more or less uniform access to distributed data, computing power, other resources • GriPhyN – Grid Physics Network (www.griphyn.org) • DataGrid (grid.web.cern.ch/grid) • PPDG - Particle Physics Data Grid (‘www.cacr.caltech.edu/ppdg’ and ‘ppdg.net’) Andrei.Chevel@pnpi.spb.ru

3. General Middleware • The term middleware is used to describe separate products that serve as the glue between two applications. Middleware is sometimes called plumbing because it connects two sides of an application and passes data between them. In particular Object Request Broker is example for middleware (from ‘www.zdwebopedia.com’). • Now several slides from Globus report are to be shown (also might be seen ‘www.globus.org’). Andrei.Chevel@pnpi.spb.ru

4. The Grid:Blueprint for a New Computing InfrastructureI. Foster, C. Kesselman (Eds),Morgan Kaufmann, 1999 • Available July 1998; ISBN 1-55860-475-8 • 22 chapters by expert authors including Andrew Chien, Jack Dongarra, Tom DeFanti, Andrew Grimshaw, Roch Guerin, Ken Kennedy, Paul Messina, Cliff Neuman, Jon Postel, Larry Smarr, Rick Stevens, and many others “A source book for the history of the future” -- Vint Cerf Andrei.Chevel@pnpi.spb.ru http://www.mkp.com/grids

5. The Grid “Dependable, consistent, pervasive access to[high-end] resources” • Dependable: Can provide performance and functionality guarantees • Consistent: Uniform interfaces to a wide variety of resources • Pervasive: Ability to “plug in” from anywhere Andrei.Chevel@pnpi.spb.ru

6. Globus Project Participants • Globus Project is a large community effort • Globus Toolkit core development • Argonne, USC/ISI, NCSA, SDSC • Globus Toolkit contributors • NASA, DOE ASCI DRM (SNL, LBNL, LLNL), Raytheon, and numerous others • Collaborators • University, lab, industrial, and international partners spanning many scientific and engineering disciplines • Active in Grid Forum • http://www.gridforum.org Andrei.Chevel@pnpi.spb.ru

7. High throughput Collab. design Remote control Application Toolkit Layer Data- intensive Remote viz Information Resource mgmt . . . Grid Services Layer Security Data access Fault detection Grid Services Architecture High-energy physics data analysis Collaborative engineering On-line instrumentation Applications Regional climate studies Parameter studies Grid Fabric Layer Transport . . . Multicast Instrumentation Control interfaces QoS mechanisms Andrei.Chevel@pnpi.spb.ru

8. Globus Approach • A toolkit and collection of services addressing key technical problems • Modular “bag of services” model • Not a vertically integrated solution • General infrastructure tools (aka middleware) that can be applied to many application domains • Inter-domain issues, rather than clustering • Integration of intra-domain solutions • Distinguish between local and global services Andrei.Chevel@pnpi.spb.ru

9. Technical Focus & Approach • Enable incremental development of grid-enabled tools and applications • Model neutral: Support many programming models, languages, tools, and applications • Evolve in response to user requirements • Deploy toolkit on international-scale production grids and testbeds • Large-scale application development & testing • Information-rich environment • Basis for configuration and adaptation Andrei.Chevel@pnpi.spb.ru

10. Globus Toolkit Grid Services • Security (GSI) • Resource management (GRAM) • Information services (MDS) • Remote file management (GASS) • Communication (I/O, Nexus) • Process monitoring (HBM) Andrei.Chevel@pnpi.spb.ru

11. Globus Toolkit on ram0 • It was intalled Globus toolkit 1.1.3; • Job submission, file transfer and part of information service were tested; • globus-job*; • globus-rcp; • grid-proxy*; • grid-info (partly). • Command description might be found on ‘www.globus.org/v1.1/programs Andrei.Chevel@pnpi.spb.ru

12. Globus File Transfer • globus-rcp host-s:file-s host-d:file-d • globus-url-copy s-URL d-URL • ncftp in GSI environment • gsincftp (GSI version for ncftp) - package consists of several programs (client side) • gsiwuftp (GSI version for server wuftp); • comparison in between ftp and gsincftp is shown on the page http://home.fnal.gov/~muzaffar/Timing/#GSI-WUFTP • All testing was done with counter part cluster pcfarm.pnpi.spb.ru Andrei.Chevel@pnpi.spb.ru

13. Example of session • ram0:/Users/shevel> grid-proxy-initEnter PEM pass phrase: ......+++++..+++++ • ram0:/Users/shevel> grid-proxy-info -all • subject : /O=Grid/O=Globus/OU=i2net.sunysb.edu/CN=Andrei E. Chevel/CN=proxy • issuer : /O=Grid/O=Globus/OU=i2net.sunysb.edu/CN=Andrei E. Chevel • type : fullstrength : 512 bitstimeleft : 11:59:50 • D= /usr/local/globus/ncftp/bin • ram0:/Users/shevel> $D/gsincftpget pcfarm.pnpi.spb.ru . tickit.ps • tickit.ps: 175.10 kB 10.39kB/s • ram0:/Users/shevel> globus-job-run pcfarm.pnpi.spb.ru /bin/date -u ;echo $? • Tue Mar 27 19:24:26 UTC 2001 • 0 • ram0:/Users/shevel> globus-job-run pcfarm.pnpi.spb.ru /bin/daterrr -u ;echo $? • GRAM Job submission failed because the executable does not exist (error code 5) • 5 • ram0:/Users/shevel> globus-job-run pcfarm.pnpi.spb.ru /usr/pbs/bin/qstat • Job id Name User Time Use S Queue • ---------------- ---------------- ---------------- -------- - ----- • 801.pcfarm xs256.bat kim 30:28:37 R L • 802.pcfarm xs248.bat kim 16:17:55 R L Andrei.Chevel@pnpi.spb.ru

14. Second Example • ram0:/Users/shevel> D=/usr/local/globus/ncftp/bin • ram0:/Users/shevel> $D/gsincftpget gsiftp://pcfarm.pnpi.spb.ru/tickit.ps • ncftpget: local file appears to be the same as the remote file, no transfer necessary. • ram0:/Users/shevel> rm tickit.ps • ram0:/Users/shevel> $D/gsincftpget gsiftp://pcfarm.pnpi.spb.ru/tickit.ps • tickit.ps: 175.10 kB 9.68 kB/s Andrei.Chevel@pnpi.spb.ru

15. Main advantages (1) • General • in one toolkit (‘bag of tools’) several issues were addressed (security, proxy servers, intrgration with other existing products, using of all availabe resources, etc.) • it is freeware (GPL); • at least part of tools are working well enough; Andrei.Chevel@pnpi.spb.ru

16. Main advantages(2) • Technical • You have no needs to be logged on remote server to perform separate commands or scripts. • You have no need to confirm every transaction with remote server. • In particular you may use all Globus command inside scripts without compromise of security. Andrei.Chevel@pnpi.spb.ru

17. Globus middleware in HEP community • LHC collaborations (see ‘http://atlas.web.cern.ch/Atlas/GROUPS/SOFTWARE/OO/grid/’, ‘http://cmsdoc.cern.ch/cms/grid/’) • GDMP - Grid Data Management Pilot (‘http://cmsdoc.cern.ch/cms/grid/’) • the pilot was built on top of the Globus toolkit. Andrei.Chevel@pnpi.spb.ru

18. Conclusion • GRID technology is in wide experimenting use now (Globus toolkit as well). • I think it is the time to try to use technical toolkit which is already available from several projects (Globus, GDMP). • To permit to use Globus toolkit it might be considered nearest future scenario • to deploy the Globus toolkit on 1-2 Linux boxes at RCF (rcas????) • to create RHIC/PHENIX CA (and DN). Andrei.Chevel@pnpi.spb.ru