1 / 15

Cluster computing facility for CMS simulation work at NPD-BARC Raman Sehgal

Cluster computing facility for CMS simulation work at NPD-BARC Raman Sehgal. Outline of the talk Example of typical cluster. Cluster types. Setup up of the Cluster Components required to make up a cluster. Selection of different Components for cluster.

watson
Download Presentation

Cluster computing facility for CMS simulation work at NPD-BARC Raman Sehgal

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Cluster computing facility for CMS simulation work at NPD-BARC Raman Sehgal

  2. Outline of the talk • Example of typical cluster. • Cluster types. • Setup up of the Cluster • Components required to make up a cluster. • Selection of different Components for cluster. • Overview of selected components.

  3. NODE-01 NODE-02 NODE-03 NODE-04 NODE-05 NODE-06 NODE-07 NODE-08 FAST / GIGABIT ETHERNET SWITCH FILE SERVER NODE-09 NODE-10 NODE-11 NODE-12 NODE-13 NODE-14 NODE-15 NODE-16 CAT-5/6 CABLE A Typical Cluster

  4. Cluster Types • High Performance clusters • Parallel computers and jobs • Optimized for better job performance • Emphasis on better interconnects • High Throughput clusters • Large numbers of sequential jobs • Emphasis on better storage and I/O • Load Balancing • High Availability clusters • Clusters to provide reliable service • Web servers, database servers etc. • Emphasis on better redundancy

  5. Node1 Node2 Node3 Node 4 Node 5 Node 6 Node 7 Node 8 Node 9 Node 10 Node 11 Node 12 Node 13 Node14 Cluster Setup Cat 5/6 Cable Worker Nodes Infiniband Cable Head Node (UI,PBS,NIS) 20 TB Storage Box Private N/w To CMS Grid Infiniband Gigabit Switch Public N/w user4 user1 user2 user3 Node 15

  6. Components that make up a cluster • Nodes • Compute (Worker) nodes : For running jobs • Service nodes (Head Node) : Management, monitoring, User interface • Storage nodes : Central storage area for users and all I/P and O/P files • Network • Switches • Cables • Support equipment • Racks : House nodes and network equipment • KVM switches, Terminal servers : For console sharing • Power distribution units : Power sequencing and control • Software • Operating System • Networking protocols • Applications

  7. Selection of Node • Computing power of a cluster depends on the power of a compute node • Choice of processor (Xeon, Opteron, Itanium …), cache, frequency • Memory • Single/Dual/Quad Processor • Network ports (Single/Dual) • Expansion slots (PCI, PCIe) • Management (IPMI …) • Form Factor (Tower, Rack mountable, Chassis, Power supply) • Heat Dissipation, Cooling • OS Support Desktop node Rack Mount server

  8. Selection of network switch • Selection of a good interconnect is an important part in the design of any cluster. • Gigabit Ethernet with copper is the standard for cluster interconnects • Infiniband provides high throughput and low latency. • Infiniband is useful when there is high I/O and communication overhead. • Choice of 24 and 48 port stackable switches • Chassis (core) switches available for large configurations (upto 672 ports) Stackable Fixed Port Switch Chassis based switch

  9. Storage Box • Central storage server(s) is (are) needed to store all users’ directories and all I/P and O/P files • Should have high capacity. • Ensure reliability using RAID . • Automated backup Console Sharing • Access to each node’s console (keyboard, mouse, display) is required during installation • KVM Switches share a single Keyboard, Mouse, Video among multiple nodes

  10. Operating System • Majority of clusters nowadays run some distribution of Linux • Robust • Open source solution • Cost-effective • Support for clusters (auto installers, cluster management tools) • Widely used distributions: Redhat, SuSe and Debian • Other Linux distributions : Mandriva and Gentoo • Redhat based distributions : Scientific Linux Cern, Centos • Selection of a distribution for a cluster depends mainly on compatiability of code to be run, so for our purpose the selected distribution is Scientific Linux Cern 4.0 or higher Code to be run : CMSSW

  11. Overview of selected components 1)Worker Node and Head Node Processor: Dual Intel Quad Core Xeon speed @ 3.0 GHz or higher Harpertown series with 12 MB cache or higher with 1600 MHz FSB. Memory: 16 GB 800 MHz DDR2 memory Hard Disk: 2-3 750 GB or higher Enterprise SATA II / SAS hard disks Network: 1. Two or more PXE boot compliant on-board 1000BaseT Gigabit Ethernet Ports 2. One Infiniband 4x DDR (20 Gbps) port on PCI Express x8 or higher 3. Form Factor: 1 U Rackmountable 

  12. 2).Storage Server : 2 Number Processor: Dual Intel Quad Core Xeon @ 2.5 GHz or higher with 12 MB cache or higher with 1333 MHz FSB, 80W per processor Memory: 16 GB, 667 MHz DIMM DDR-2 Capacity : 24 x 450 GB = 10.8 TB with SAS disks of 15 K rpm. Network: ·Two PXE boot complaint on-board 1000BaseT Gigabit Ethernet Ports ·   Dual 10G Ethernet multimode fiber port on PCI Express ·   One Infiniband 4x DDR(20 Gbps) port on PCI Express Form Factor :3 / 4 U rackmountable. RAID level : Support for RAID levels 0,1,10,5,6 Server Management : Using Intelligent Platform Management Interface ( IPMI)

  13. Networking Switches 3). Infiniband Switch with following specifications: No. of Ports : 24 port InfiniBand 4X DDR (20Gbps) ports Bandwidth : 960 Gbps Chassis: 1 U Rack Mountable chassis Management protocols : SNMP, Telnet, SSH, HTTP, FTP Infiniband cables : 4xDDR infiniband CX4, 30 AWG Passive cable 4).Gigabit Ethernet Switch (a) 24 port autosensing, unmanaged Gigabit Switch (b) 24 port autosensing, managed Gigabit Switch

  14. 5). Rack Mountable KVM switch with following specifications • No. of ports: 16 ports with PS/2 and USB keyboard and mouse support • Features: Integrated with 15 inch LCD monitor, Keyboard and touch pad • Form Factor: 1U with LCD panel folded 6) Uninterruptible Power Supply (UPS) • Capacity: 6KVA • Form Factor: 3 U Rack Mountable with rail kit and internal batteries • Nominal Output Voltage: 230 V @ 50 Hz • Backup time: 30 minutes at half load with internal battery or higher

  15. THANKS

More Related