1 / 30

Perspectives of JINR Grid-infrastructure

Perspectives of JINR Grid-infrastructure. V.V. Ivanov , Gh. Adam, V.V. Korenkov, T.A. Strizh, P.V. Zrelov Laboratory of Information Technologies Joint Institute for Nuclear Research. Participating in LHC, RHIC, TEVATRON… In future: FAIR, ILC …. LHC-2008. FAIR-2015. ILC – after 2020 (?).

fritzi
Download Presentation

Perspectives of JINR Grid-infrastructure

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Perspectives of JINR Grid-infrastructure V.V. Ivanov, Gh. Adam, V.V. Korenkov, T.A. Strizh, P.V. Zrelov Laboratory of Information Technologies Joint Institute for Nuclear Research

  2. Participating in LHC, RHIC, TEVATRON… In future: FAIR, ILC … LHC-2008 FAIR-2015 ILC – after 2020 (?) The JINR participation in the LHC experiments and in other large-scale projects asked for a substantial increase of its networking and information resources as well as the deployment of a large volume of work toward the development of the JINR Grid-segment and its integration in the Russian Grid-infrastructure RDIG (Russian Data Intensive Grid).

  3. The Laboratory of Information Technologies (LIT) is responsible for the computing infrastructure development in JINR and hosts the Central Information and Computing Complex (CICC) of the Institute. Further perspective tasks follow from the decision of the JINR Committee of the Plenipotentiary Representatives, concerning the radical improvement of the JINR computer telecommunication links with major partner laboratories in the JINR Member States during the years 2010-2015 and the development of a unified Grid-environment of the JINR Member States. This is a cornerstone task of the JINR seven-year plan on 2010–2016 within the direction ”Networks. Computing. Computational Physics”.

  4. Development of the JINR Grid-environment – 2010-2016 Three levels within the JINR Grid-environment can be distinguished: network, resource, and applied. support Algorithms and Methods support Centre of Primary Grid Services "Gridification" Virtual Organizations APPLICATIONS support, upgrade data data ATLAS ALICE CMS LHCb HONE Biomed etc. Telecommunications channels JINR-Moscow Adaptation to Grid New VO creation RESOURCE LEVEL grid grid extension JINR CICC APPLIED Computing clusters and data storage systems JINRLIB WWW/FTP Progr.Lib. NETWORK LEVEL JINR Member-states connectivity applications JINR LAN RESOURCE RBNet RUNNet RASNet RUHEP GEANT2 JINR High-speed Backbone NETWORK

  5. Cooperation Collaborators from the JINR Member States (protocols of cooperation with INRNE (Bulgaria), ArmeSFo (Armenia), FZK Karlsruhe GmbH (Germany), IHEPI TSU (Georgia), NC PHEP BSU (Belarus), KFTI NASU (Ukraine), etc.). Grants of Plenipotentiaries of the JINR Member States: Poland, Bulgaria, Slovakia, Romania. Agreement on Joint Research Programme “Meshcheryakov-Hulubei” LIT-Romania. BMBF grant “JINR Network Security, Failover Measures, Network monitoring”. CERN-JINR Cooperation Agreement on several topics, and JINR in the LCG Project (participation of JINR in the LCG project). JINR-South Africa Cooperation Agreement - project “Development of Grid segment for the LHC experiments”. Participation in joint projects: Enabling Grids for E-sciencE (EGEE -a project co-funded by the European Commission through the Sixth Framework Programme, CERN-INTAS project and "SKIF-GRID“ project of the Program of the Belarusian-Russian Union State as a part of joint propositions of NASB and Federal Agency of Science and Innovations of Russian Federation “The development and use of hardware and software in GRID-technologies and advanced supercomputer systems SKIF in 2007-2010 (SKIF-GRID)”. Joint research grants from RFBR and Ministry of Science and Education with MSU and SINP MSU, Saratov University, IM SB RAS, IMM RAS, TU Gomel (Belorussia).

  6. June 2009 - two important projects completed Two large-scale very important projects have been completed: -- JINR - Moscow 20Gbps telecommunication channel --JINR CentralInformation and Computing Complex major upgrade. Comprehensive presentation was done at the meeting of the Programme Advisory Committee for Particle Physics, on June 10. All organizations involved in the project realizations, together with the representatives of Ministry of Communications, Federal Agency for Science and Innovation, attended the Presentation. Our community colleagues and prospective users - residents of the special economical zone - also took part in the event.

  7. JINR-Moscow 20Gbps telecommunication channel Collaboration with RSCC, RIPN, MSK-IX, JET Infosystems,Nortel In perspective, the used DWDM and 10Gb Ethernet technologies allow the creation of up to 80 channels of 10 Gbps each, resulting in athroughput of up to 800 Gbps.

  8. JINR Central Information and Computing Complex In 2008, the total CICC performance was 1400 kSI2K, the disk storage capacity 100 TB At present, the CICC performance equals 2300 kSI2K and the disk storage capacity 500 TB PLAN

  9. JINR Local Area Network Backbone (LAN) • Comprises 6579 computers and nodes, • Users – 3593, • IP – 7633, • Remote VPN users (Lanpolis, Contact, TelecomMPK) - 1248; • High-speed transport (1Gbps) (Min. 100 Mbps toeach PC); • Controlled-access (Cisco PIX-525 firewall) at network entrance; • Partially isolated local traffic (8 divisions have own • subnetworks with Cisco Catalyst 3550 as gateways); • General network authorization system involves many services (AFS, batch systems, Grid, JINR LAN remote access, etc.) • Plans: • Step-by-step modernization of the JINR Backbone – transfer to 10 Gbps • Development and modernization of the control system of the JINR highway network

  10. JINR Laboratories First stage JINR Laboratories The plan for the JINR LAN development for the years 2010-2016 foresees several general purpose tasks as well as specific site-related ones. A selective enumeration points to the: • upgrade of the JINR backbone to a data transfer rate of 10 Gbps and gradual connection to it of various JINR laboratories; • increase of the performance of the LAN central telecommunication node kernel; • increase of the LAN protection at the hardware level; • wireless access to the LAN within the JINR territory; • implementation of new solutions and methods for the traffic control, information security, development of effective tools for control and the quality of service of JINR LAN; • development of the system of protection of the JINR LAN resources from non-authorized access.

  11. JINR Central Information and Computing Complex

  12. JINR CICC structural scheme

  13. LHC Computing Grid Project • The protocol between CERN, Russia and JINR on participation in LCG Project was approved in 2003. MoU on Worldwide LHC Computing Grid (WLCG) was signed by JINR in October 2007 • The tasks of the JINR in the WLCG: • WLCG-infrastructure support and development at JINR; • participation in WLCG middleware testing/evaluation, • participation in Service/Data Challenges, • grid monitoring and accounting tools development; • FTS-monitoring and testing; • participation in ARDA activities in coordination with experiments; • JINR LCG portal support and development; • HEP applications; • MCDB development; • support of JINR Member States in the WLCG activities; • User & Administrator training and education.

  14. support and development WLCG/EGEE infrastructure; participation in middleware testing/evaluation, participation in Data and Service Challenges, grid monitoring and accounting system development; FTS-monitoring and testing MCDB development; Participation in ARDA activities in coordination with experiments; HEP applications; User & Administrator Training and Education support of JINR member states in the GRID activities. JINRin the LCG/EGEE

  15. JINR WLCG infrastructure • CICC comprises: • 65 servers • 4 interactive nodes • 960 computing nodes, Xeon 5150, 8GB RAM (GEthernet) • (160 computing nodes, Xeon X5450, 16GB RAM, InfiniBand). • Site name: JINR-LCG2 • Internal CICC network – 1Gbit/sec • Operating system -Scientific Linux CERN 4.6 (transfer to Scientific Linux SL release 5.2.); • Middleware versionGLITE-3.1 • File Systems – AFS (the Andrew File System) for user Software and home directories is a world-wide distributed file system. AFS permits to share easily files in an heterogeneous distributed environment (UNIXes, NT) with a unique authentication scheme (Kerberos). • dCache- for data. • User registration system – Kerberos 5 ( AFS use Kerberos 5 for authentication)

  16. JINR WLCG infrastructure JINR provides the following services in the WLCG environment: Basic services:  - Berkley DB Information Index (top level BDII);  - site BDII;  - 2 x Computing Element (CE);  - Proxy Server (PX);  - 2 x Workload Management System (WMS);  - Logging&Bookkeeping Service (LB);  - RGMA-based monitoring system collector server (MON-box);  - LCG File Catalog (LFC);  - Storage Element (SE), dCache 400 TB, 4 x gridftp door, 14 x pool;  - 4 x User Interface (UI), installed in AFS.Special Services - VO boxes for ALICE and for CMS; ROCMON; PPS and testing infrastructure - Pre-production gLite version; Software for VOs: dCache xrootd door, AliROOT, ROOT, GEANT packages for ALICE; ATLAS packages; CMSSW packages for CMS and DaVinchi, Gauss packages for LHCb.

  17. dCache in JINR lxfs07 lxfs07 lxfs07 lxfs07 lxfs07 lxfs07 lxfs07 lxfs07 lxfs07 lxfs07 lxfs07 lxfs07 wn… lxpub01 lxfs71 rda02 Internet / GRID JINR Backbone Protocols gFTP, SRM, XROOT local lxfs07 500 TB Admin RAID DCAP PNFS WNs Doors Pools

  18. Frames for Grid cooperation of JINR in 2009 Worldwide LHC Computing Grid (WLCG); Enabling Grids for E-sciencE (EGEE); RDIG Development (Project of FASI) CERN-RFBR project “GRID Monitoring from VO perspective” BMBF grant “Development of the GRID-infrastructure and tools to provide joint investigations performed with participation of JINR and German research centers” “Development of Grid segment for the LHC experiments” was supported in frames of JINR-South Africa cooperation agreement; NATO project "DREAMS-ASIA“ (Development of gRid EnAbling technology in Medicine&Science for Central ASIA); JINR-Romania cooperation Hulubei-Meshcheryakov programme Project "SKIF-GRID" (Program of Belarussian-Russian Union State). Project GridNNN (National Nanotechnological Net) We work in close cooperation and provide support to our partners in Armenia, Belarus,Bulgaria,Czech Republic, Georgia, Germany, Poland, Romania, South Africa, Ukraine, Uzbekistan.

  19. Development of the JINR Grid-environment – 2010-2016 • Network level: • links between Moscow and Dubna on the basis of state-of-the-art technologies DWDM and 10Gb Ethernet. • JINR Local area network : • JINR High-speed backbone construction – 10Gbps • Resource level: • requirements of the LHC experiments stimulate the development of a global Grid-infrastructure, together with the resource centers of all the cooperating organizations. First of all, this is of primary concern for such large research centers as the JINR. To reach effective processing and analysis of the experimental data, further increase in the JINR CICC performance and disk space is needed.

  20. The RDIG infrastructure Now the RDIG infrastructure comprises 15 Resource Centers with > 7000 kSI2K CPU and > 1850 TB of disc storage. • RDIG Resource Centres: – ITEP – JINR-LCG2 – Kharkov-KIPT – RRC-KI – RU-Moscow-KIAM – RU-Phys-SPbSU – RU-Protvino-IHEP – RU-SPbSU – Ru-Troitsk-INR – ru-IMPB-LCG2 – ru-Moscow-FIAN – ru-Moscow-GCRAS – ru-Moscow-MEPHI – ru-PNPI-LCG2 – ru-Moscow-SINP

  21. JINR CICC RDIGmonitoring&accounting http://rocmon.jinr.ru:8080 • Monitoring – allows to keep an eye on parameters of Grid sites' operation in real time • Accounting - resources utilization on Grid sites by virtual organizations and single users • Monitored values • CPUs - total /working / down/ free / busy • Jobs - running / waiting • Storage space - used / available • Network - Available bandwidth • Accounting values • Number of submitted jobs • Used CPU time • Totally sum in seconds • Normalized (with WNs productivity) • Average time per job • Waiting time • Totally sum in seconds • Average ratio waiting/used CPU time per job • Physical memory • Average per job

  22. Production Normalised CPU time per EGEE Regions (June-August 2009)

  23. RDIG Normalised CPU time per Site and per VO (June-September 2009)

  24. Production Normalised CPU time per EGEE site for VO LHC (June – August 2009) GRID-site CPU time Num CPU time/N ------------------------------------------------------------------------ FZK-LCG 6,039,3048620 700 INFN-T1 3,493,208 2862 1220 CERN-PROD 3,309,795 6812 485 CYFRONET-LCG 3,198,111 2384 1341 PIC 3,037,977 1337 2272 JINR 2,938,039 960 3060 IN2P3-CC 2,668,914 4544 587 RAL-LCG2 2,629,889 2532 1038 UKI-GLASGOW 2,610,116 1912 1365 UKI-LT2-IC-HEP 2,533,230 960 2638 24

  25. JINR-LCG2 in RDIG JINR - 40% of total RDIG CPU time Normalised CPU time per VO at JINR site

  26. Further participation in the WLCG project • Support and development of the JINR WLCG-segment in frames of global WLCG infrastructure; • Participation in Service/Data Challenges in coordination with LHC experiments; • WLCG middleware testing/evaluation; • Grid-monitoring the LCG-infrastructure at JINR and other sites of the Russian Tier2 cluster; • Evaluation of new Grid technologies in context of their usage in WLCG; • MCDB development (data base structure; basic modules; • Participation in ARDA activities in coordination with experiments; • JINR WLCG web-portal improvement; • User’s support to stimulate their active usage of WLCG resources (courses, lectures, trainings, publication of user guides in Russian); • Support of JINR member states in WLCG activities.

  27. Applied level developments of the JINR Grid-environment(I) • The applied level covers the user applications working in a virtual organization (VO) environment which comprises both users and owners of computing resources. • In the existing Grid-systems, a VO defines a collaboration of specialists in some area, who combine their efforts to achieve a common aim. • The virtual organization is a flexible structure that can be formed dynamically and may have a limited life-time. • VOs working within the WLCG project are the VOs on the LHC experiments - ATLAS, CMS, Alice, LHCb, the first three being carried out with the noticeable and direct participation of the JINR. Nowadays, as a Grid-segment of the EGEE/RDIG, the JINR CICC supports computations of the virtual organizations registered in RDIG ( LHC experiments, BioMed, PHOTON, eEarth, Fusion, HONE, Panda. • In the future, as the interest arises at a large-scale level, VOs can be organized at JINR in the fields of nuclear physics and condensed matter physics and, most probably, in the new promising direction related to the research of the nanostructure properties.

  28. Applied level developments of the JINR Grid-environment(II) The creation of new VOs gets possible and necessary under maturation of the algorithmic approaches to the problem solution, the development of corresponding mathematical methods and tools: • methods and tools for simulation of physical processes and analysis of experimental data software and computer complexes for experimental data processing; • numerical methods, algorithms and software for modeling complex physical systems; • methods, algorithms and software of computer algebra; • new computing paradigms; • adaptation of specialized software for solving problems within the Grid-environment.

  29. Formation of the Grid environment of the JINR-Member States Creation of the Grid environment of the JINR-participating countries will allow to bring out the research underway at JINR in the field of high energy physics, nuclear physics, solid state and condensed matter physics, biophysics, biosensor (nano)technologies, etc., on a qualitatively new level. Basic directions: • Creation of a working group of the representatives of the JINR-participating countries for the development of the main directions of the activity toward forming a joint Grid environment. • Definition, together with the JINR Member States, of the basic directions of activity toward forming the joint Grid environment. • Preparation of corresponding multi-side agreements. • Development of the network level of the Grid environment of the JINR Member States. • Development of the resource level of the Grid environment of the JINR Member States. • Formation of the applied level of the Grid environment of the JINR Member States. • Preparation and implementation of joint projects.

  30. Thank you for your attention !

More Related