210 likes | 221 Views
This document provides an overview of the Kolkata Tier-2 facility, including computing and storage resources, network connectivity, job submissions and traffic load. It also highlights the availability and reliability of the site and discusses the installation and roadmap for the EOS storage system.
E N D
Kolkata Tier-2 Status and Plan Site Name :- IN-DAE-VECC-02 Gocdb Name:- IN-DAE-VECC-02VO :- ALICECity:- KOLKATACountry :- INDIA Vikas Singhal and Prasun Singh Roy VECC, Kolkata, INDIA
Kolkata On MonaLisa Vikas Singhal, VECC, INDIA
ALICE Job completed during Last 3 Years @Kolkata More than 800000 Jobs completed during last one year More than 2200000 Jobs completed during last three year. Total :- 888 Cores DELL Quad Core Blades 38 * 2 * 4 *2 = 608 (HT) HP Quad Core Blades 8 * 2 * 4 * 2 = 128 (HT) HP Dual Core Blades 38 * 4 = 156 Equivalent 9K HEPSpec 2006 Computing Resources. Storage :- 146 TB XrootD Redirector 3 Disk Servers SAN Based 78TB (IBM) + (36 + 32)TB (HP EVA) 1Gbps WAN Network speed. 10Gbps Backbone Network which is 1.15% of Total ALICE job. Vikas Singhal, VECC, INDIA
ALICE Job submitted during Last 3 Years More than 800000 Jobs completed during last one year More than 2200000 Jobs completed during last three year which is 1.15% of Total ALICE job. More than 700 jobs consistently running. We reached upto 850. Vikas Singhal, VECC, INDIA
Traffic load during Last 3 Years @Kolkata Vikas Singhal, VECC, INDIA
Kolkata Tier-2 Facility Vikas Singhal, VECC, INDIA
10 Gbps 4-10 Gbps 1 Gbps x Gbps Physical Network Connectivity upto Kolkata CERN LHCONE 10 Gbps TEIN-3 Network 4 Gbps/10 Gpbs Shared TIFR Mumbai 10 Gbps 10 Gbps (Under going) Internet 1 Gbps POP in Mumbai 10 Gbps 10 Gbps Core Network 10 Gbps SINP Kolkata POP in Kolkata 1 Gbps 1 Gbps 1 Gbps VECC Kolkata 1 Gbps Kolkata Tier-2
NetworkConnectivity NKN Router in Computer Division 1GB optical Fibre Link L3 switch in Grid Computing Lab . 1 Gbps Ethernet Link Public Network XrootD Redirector gridce02 EOS MGM & FST grid-peer Grid DNS Server XrootD Disk Servers 10Gbps Fiber Private Network HP IBM SAN Storage ALICE Worker nodes Tier-3 Nodes ALICE Centre Backbone connectivity with 10Gbps
Last Three Year Utilization, Availability and Reliability On an average site availability and reliability is more than 90% during last 3 years. Availability: Jan2013 - Dec2013 : 78.42 % Jan2014 - Dec2014 : 94.75 % Jan2015 – Dec2015 : 98% More info http://grid.tier2-kol.res.in/ Vikas Singhal, VECC, INDIA
Kolkata::SE Vikas Singhal, VECC, INDIA
Kolkata::SE Traffic Vikas Singhal, VECC, INDIA
NEW Storage Via EOS • Started long back, but due to Space and Speed problem could not configure properly. • Tried different Linux versions, Planned to deploy on CC7 such that not have to upgrade in near future. • For CC7 deploy script have to be properly checked. • Finally decided to configure on SLC 6 only.
Elements of KOLKATA EOS • SE name ALICE::Kolkata::EOS • EOS Version EOS Beryl-Aquamarine • 2 MGM(management) servers • One master MGM: eos01.tier2-kol.res.in • One slave MGM: eos02.tier2-kol.res.in • Three nos. of FST • eos01.tier2-kol.res.in • eos02.tier2-kol.res.in • eos03.tier2-kol.res.in • Usable 96TB after RAIN-6 Configuration • 4TB *12 nos of HDD in each storage. Vikas Singhal, VECC, INDIA
EOS Installation at Kolkata Vikas Singhal, VECC, INDIA
Planning and Roadmap • Last year did the Market Survey and discussed with Latchezar for the best configuration for the ALICE compute nodes:- • 2 * 12 core Intel Xeon Processors • Total 128GB of RAM ~ 5 GB/core or 2.5GB per Hyperthreaded Core • One 800 GB SSD • For better management and space requirement planed to have each server within 0.7 U • Total 48 Servers ~ 24 * 48 = 1152 Cores or 48*48 = 2304 logical Cores • Will be > 25K HEP Spec2006 . • To avoid any SPF (Single Point of Failure) so decided to have N+N power supply and N+N Cooling Fans.
Planning and Roadmap Cont.. • For the same procurement started last year. • Received 3 offers from HP, DELL and Fujitsu. • Unfortunately no one qualified for the N+N FAN Redundancy for the Dense or Blade server architecture. • Re-tendered the indent with modified specifications (removed N+N FAN Redundancy). • Added ~100TB storage via EOS under Kolkata SE. • Will start procurement for more storage approx 6*100TB servers. And will configure via EOS RAIN-6. • 10 RDHx type racks allotted in the New Computer Hall for all the new infrastructure.
Network Backbone for New Lab 10Gb backbone, 40Gb core switch connection. Both Fiber and Copper connectivity. Connection via DAC Cable. Dual path, Full Redundancy. Vikas Singhal, VECC, INDIA
Grid Computing Facility Infrastructure Total 2000 Cores of Computing. More than 400TB of storage. Have a Tier-3 Cluster named grid-peer 8 Numbers of HP BL675G7 Blade servers each with 4 * AMD Opteron Processor 6380(16 Core) & 6 Number of Dell M610 Blade servers each with 2 * Intel Quad Core E5530 Xeon 2.4GHz CPU 8MB cache and 16GB RAM. Supporting ALICE, CBM, Medical Imaging, Nuclear theory users etc. Extensively used by VECC users and PMD Collaborators. 75 TB Tape backup. 75 + active users (across India.) 45 + active users (in VECC.) Managing DNS, NFS, Monitoring, PXE etc servers.
HOLI Festival of Colors in Month of March Thank You