EU DataGrid segment in Russia. Testbed WP6.

1. EU DataGrid segment in Russia. Testbed WP6. V.Ilyin1, N. Kruglov1, A. Kryukov1, V. Korenkov2, V. Kolosov3, V. Mitsyn2, L. Shamardin1 1SINP MSU Moscow, Russia 2JINR Dubna, Russia 3IHEP Moscow, Russia

2. Russian EU-DataGrid Segment Russia participates in the following work packages of the EU-DataGrid project: • WP6 Testbed • WP8 HEP Application Building of DataGrid infrastructure in Russia started in Moscow region on the base of several institutes (SINP MSU, ITEP, JINP, IHEP and some others). Further development for other regions (St.Petersburg, Novosibirsk) and institutes is under discussion. Main goal for Russian EU-DataGrid segment is to support Tier2 center for LHC computing. Secondary goals include chemical calculations and others. Research is sponsored by CERN-INTAS-0440 grant. Lev Shamardin

3. Research directions Data transfer and traffic investigation • Investigation of local area links traffic speed and performance with different size of buffering disk server. • Practical implementation and testing and testing of high-speed site-to-site data replication through Moscow region backbone. • Practical implementation and testing of site-to-site data replication Moscow-CERN. Investigation of distributive and local data storage • The study of feasibility and robustness of disk-based, tape less data repositories of data, assembled from low cost industrial mass production components; investigation of novel storage technologies. • Investigation of the high capacity storage system implementation (tape robots) at the conditions of medium/long range domestic communications structure. Lev Shamardin

4. Research directions (cont.) GRID tools (middleware) in HEP applications • Elaboration of elements of cooperative network management for communications, connecting participating institutes. • GRID system software tools test in essentially heterogeneous local and wide-area network facilities. Research of effectiveness of CPU utilization of production farms (PC clusters) • Investigation of effectiveness of CPU utilization and other computer resources during test data production using distributed OO databases. • The study of efficiency of local schedulers of batch tasks and their scalabilities and robustness. Lev Shamardin

5. Testbed Activities Testbed 0 Globus installation GRIS-GIIS configuration Certification Authority Man power 8 FTE Testbed 1 Coordinated at the national level 4+ participating institutes Significant computational resources (160 CPUs, 7.5 TB) Wide range of used software “Complicated” network configuration Lev Shamardin

6. Testbed 1 Hardware for the testbed 1 Participating institutes: SINP, IHEP, JINR, ITEP and others. Total: 160 CPUs Pentium III 500 MHz – 1 GHz, 256 MB RAM per CPU, 6 IDE fileservers 1.2 TB each. Software environment Linux RedHat 6.2 (kernel 2.2.14), PBS, NFS & AFS, ObjectivityDB (GDMP 1.1), Globus toolkit 1.1.3 (with GRIS/GIIS patches) or newer version if available, CONDOR on several farms. Network configuration Internal FastEthernet network, 400 Mbit interfaces on fileservers. Regional and international links up to 1Gbit Lev Shamardin

7. dc=ru, o=grid Country-level GIIS lhc-fs.sinp.msu.ru:2137 dc=sinp, dc=ru, o=grid SINP MSU, Moscow dc=jinr, dc=ru, o=grid JINR, Dubna dc=srcc, dc=ru, o=grid SRCC MSU, Moscow dc=ihep, dc=ru, o=grid IHEP, Protvino CERN Top-level WP6 GIIS testbed001.cern.ch:2137 dc=itep, dc=ru, o=grid ITEP, Moscow dc=tcss, dc=ru, o=grid TCSS, Moscow dc=kiam, dc=ru, o=grid KIAM, Moscow dc=?, dc=ru, o=grid St. Petersburg Russian National GIIS • SRCC MSU, KIAM and TCSS participate only in Russian DataGrid project and are not involved in CERN projects. Lev Shamardin

8. dc=sinp, dc=ru, o=grid Institute-level GIIS lhc.sinp.msu.ru:2137 lhc10.sinp.msu.ru:2135 GRIS lhc.sinp.msu.ru:2135 GRIS PBS lhc-fs.sinp.msu.ru:2135 SINP MSU GIIS Configuration • There are both batch and interactive machines • Batch cluster uses PBS as a batch system • Job submission via globus can be done both to PBS cluster and to separate machines • There are some preliminary solutions for providing cluster status in GRIS (there is no native support for this in globus) Lev Shamardin

9. Russian DataGrid Certification Authority http://lhc.sinp.msu.ru/CA/ Lev Shamardin

10. Russian DataGrid CA (cont.) • Russian DataGrid CA issues certificates for russian institutes, labarotories and individuals participating in EU-DataGrid project. • A certificate identifies authenticy of the owner’s name, but it does not guarantee any rights. • Only the local administrator of the resource gives or cancels the permission to use the resource. • The Russian DataGrid Certification Authority is not the only CA for Russia in the EU-DataGrid project. There can be more certification authorities for different regions. Lev Shamardin

11. Russian DataGrid CA (cont.) • Russian DataGrid CA issues certificates for Russian institutes, laboratories and individuals participating in EU-DataGrid project. • A certificate identifies authenticy of the owner’s name but does not guarantee any rights. • Only the administrator of the local resource gives or cancels the permission to use the resource. • The Russian DataGrid Certification Authority is not the only CA for Russia in the EU-DataGrid project. There can be more certification authorities for different regions • Russian DataGrid CA uses modified registration authorities scheme for issuing certificates. Lev Shamardin

12. Why do we need Registration Authorities? • Certification Authorities are trusted by lots of people, so we need to grant “correct” issuing of certificates • Certification Authorities must be hack proof, so “correct” issuing of the certificates means: • Only valid persons can get the certificates. • To check that the person is valid we must contact him in hack proof way (by phone or direct contact for example). Lev Shamardin

13. Certification Authority without RA* *RA stands for Registration Authority A user submits a request to certification authority. Certification Authority administrator reads the request and phones theuser to validate that this is a normal request, not a result of mail hack oran error. If the phone validation is ok, Certification Authority administrator issues a certificate and emails it back to user Lev Shamardin

14. Why this is not the best solution • Performance problems with large amount of requests per day in future • Possible errors in user identity validation by phone, direct contact is much better • Low requests processing speed • Possible security problems with phone contacts? Lev Shamardin

15. Certification Authority with a RA • One of the ways to solve the mentioned problems is to assign “trusted” persons – Registration Authorities – in different divisions and allow them to validate the users identity. • Certification Authority administrator in this case trusts to the RA’s and does not make any validation for requests signed by RA’s. • We can also implement revocation mechanisms for RA’s so they can send trusted (no further checks by CA) certificate revocation requests to the CA. Lev Shamardin

16. CA with RA work scheme This scheme differs from the classical scheme on the second stage when we use globus certificates for signing the requests. This scheme requires some new software, but allows to use standard globus certificates for the whole process instead of using separate certificates for RA’s and RA’s owners globus certificates. A user generates a certificate request with grid-cert-request and sends it to his division administrator Division administrator validates the request and signs it with his globus certificate (digital signing software is a part of the package). After this the signed request is emailed to the Certification Authority The Certification Authority software checks the signature and trust settings for the new emailed request, issues a new certificate and emails it back to the division administrator Lev Shamardin

17. Implementation details • Globus certificates are used for digital signing email messages • The person who signed the issue request can also revoke the certificate sending a special revoke request to CA email robot. CRL is updated automatically • User can renew his certificate sending a special renew request generated with grid-cert-renew program to the CA email robot • Email robot can perform several supplementary functions except issuing and revoking certificates like checking the certificate validity, obtaining an issued certificate with a specified DN or serial number • In future the CA software will keep track of certificates expiration dates and send renew challenge text to certificate owner for challenge text renew request authorization used in globus. Lev Shamardin

18. Certificate Revocation Lists (CRLs) • CRLs are the only way to check the validity of the certificate in GSI. This means that for the best security we must update CRLs on the hosts running globus as fast as possible. • All current solutions of this problem are based on scheduled web update of the CRLs from CA sites. These solutions have some big disadvantages: • CRLs update frequency often does not match the CRLs generation frequency at CAs. This causes too frequent updates or CRL expiration problems. • Signing policies are not updated automatically now. • Frequent CRL updates can cause big load on the web servers of the CAs. Lev Shamardin

19. Requirements for “ideal” update scheme • Perhaps one of the best solution for updating the CRLs is that the update is done on all sites when the CA issues a new CRL. • There should be a way for centralized update of signing policy and CA certificates Lev Shamardin

20. Update scheme CA Testbed1 CA update root Objectclass=GlobusTestbed1CAupdate o=Grid “new CRL” request National CA update center 1 Objectclass=GlobusTestbed1CAupdate dc=ch, o=Grid National CA update center 2 Objectclass=GlobusTestbed1CAupdate dc=fr, o=Grid … Institute CA update center 1 Objectclass=GlobusTestbed1CAupdate dc=cern, dc=ch, o=Grid Institute CA update center 1 Objectclass=GlobusTestbed1CAupdate dc=urec.cnrs, dc=fr, o=Grid Institute CA update center 2 Objectclass=GlobusTestbed1CAupdate dc=in2p3, dc=fr, o=Grid Lev Shamardin

21. Benefits of this scheme • CRLs on all hosts will be always up to date. There will be no “CRL expired” problems even with short CRL expiration times (like several hours for example). • There is a possibility to add new CAs into testbed1 without any headache for site administrators. Lev Shamardin