1 / 6

Virginia Tech NDSSL-HPC Proposed System Diagrams 06-11-2013

Virginia Tech NDSSL-HPC Proposed System Diagrams 06-11-2013. Diagram Page Architecture Diagram 2 Equipment Rack Placement 3 Components 4,5 Management Network 6. Racks 1-4. Rack 5. 10 GbE Network. 1GbE Management

justin-kaufman
Download Presentation

Virginia Tech NDSSL-HPC Proposed System Diagrams 06-11-2013

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Virginia Tech NDSSL-HPC Proposed System Diagrams 06-11-2013 DiagramPage Architecture Diagram 2 Equipment Rack Placement 3 Components 4,5 Management Network 6

  2. Racks 1-4 Rack 5 10 GbE Network 1GbE Management Net Switches (stacked) Racks 1-4, each with: 18 x C2112 2U Quad Node Chassis 80 nodes each with: 2 x E5-2670 CPUs 64GB Memory (4GB/core) 1 x 1TB HDD FDR IB 2 x 48-port 10GbE Switches FDR IB Fabric 432 leaf ports expandable to 648 ports Racks 6 1GbE Mgmt switch 4 x C1104-RP5 1U Xeon PhiNodes 2 x E5-2670 CPUs 64 GB Memory (4GB/core) 1 x 1TB HDD 2 x C2108-RP2 Admin nodes 2 x Intel Xeon E5 2670 2.6 GHz 128GB Mem 2x1TB HDD RAID-1 1 x PCIE-CONNX3-1P (FDR IB) 2 x 10GbE ports (SR) Console/KVM 2 x C2108 VM servers, 2 x Intel Xeon E5 2670 2.6 GHz 128GB Mem 3x1TB HDD RAID-1 1 x PCIE-CONNX3-1P (FDR IB) 2 x 10GbE ports (SR) 2 x C2108-RP2 Login nodes 2 x Intel Xeon E5 2670 2.6 GHz 128GB Mem 2x1TB HDD RAID-1 1 x PCIE-CONNX3-1P (FDR IB) 2 x 10GbE ports (SR) 1 xC2108 archive node 2 x Intel Xeon E5 2670 2.6 GHz 128GB Mem 2x1TB HDD RAID-1 1 x PCIE-CONNX3-1P (FDR IB) 2 x 10GbE ports (SR) 2 x C2108-RP2 Gateway nodes 2 x Intel Xeon E5 2670 2.6 GHz 128GB Mem 2x1TB HDD RAID-1 1 x PCIE-CONNX3-1P (FDR IB) 2 x 10GbE ports (SR) 4 x UV20 fat nodes 4 x E5-4650 2.6GHz 8 core 1.5TB memory, 1 x 1TB HDD, FDR Racks 9 1 x 1GbE Mgmt switch 4 x UV20 Oracle Servers each with 10 x 3TB HDD NFS Server 1 MIS Server with 72 3TB HDD 1 MIS JBOD Expansion with 72 3TB HDDs Racks 7,8 Cable Legend Racks 7,8 each with: 1 x 1GbE Mgmt switch 20 x C2110-RP5 2U Xeon Phi Nodes 2 x E5-2670 CPUs 64GB Memory (4GB/core) 1 x 500GB HDD 1 x 5110P Xeon Phi accelerators FDR IB 10GbE Network 1GbE Management Network FDR Infiniband Fabric NDSSL HPC System Architecture

  3. C2112-4RP4 2U Quad Node Chassis • 4 x compute nodes in 2U, each with: • - 2 x E5 CPU sockets • 16 x 1600/1333/1066/800 DIMM Slots • - 3 x 3.5” HDD per server • 1 x PCIe 3.0 x16 supportint low profile slot • 1 x FDR Infiniband port on board • - 2 x 1GbE NICs • - 2 x 1200w redundant power supplies C1104G-RP5 Accelerator node 1U 2 x E5 CPU sockets 8 x 1600/1333/1066/800 DIMM Slots 4 x 2.5” HDD 3 x PCIe 3.0 x16 supporting 3 GPGPU or Xeon Phi cards 1 x PCIe 3.0 x8 slot 2 x 1GbE NICs 2 x 1800w Platinum level redundant power supplies • C2108 • - 2U dual-socket E5-2600 series server • - HPC management, head node, I/O nodes • - Up to 8 x 3.5” hot-swap drives • - Up to five expansion PCIe slots • - 1+1 Redundant AC Power Supplies • UV20 2U Quad Socket Server • - 4 socket E5-4600 series server • - 48 DIMM slots (1.5TB mem with 32GB DIMMs) • - Up to 8 x 2.5” HDDs • 8 PCIe slots • Redundant power NDSSL HPC Components

  4. SGI IS2112 2U 12-drive JBOD • 12 3.5-inch HDD • SAS Interfaces SGI Modular InfiniteStorage Server • Storage Server Options • Integrated into 4U chassis • 1 or 2 motherboards (Sandy bridge Platform) • 2 socket LGA ES-2600 (E5-2600 Series) • Each chassis supports SATA/SAS/SSD drive types: • Up to 72 x 3.5” drives • Up to 144 x 2.5” drives • Dual GbE onboard. Optional 2 port 10GbE, 2 port GbE, or 4 port 8Gb FC PCIe cards • JBOD Option • Up to 81 SATA/SAS/SSD drives (162 w/2.5” SFF) • Up to 8 SAS connections NDSSL HPC Components

  5. Rack 1 Rack 2 Rack 3 Rack 4 Rack 5 Rack 6 Rack 7 Rack 8 Rack 9 Racks 1-4 each with: 19 x C2112 Quad-node compute chassis (total: 304 nodes) Rack 5: IB switches, 10GbE switches Rack 6: 4 x C2112 Quad compute chassis (16 nodes) 2 x admin nodes, 2 x VM servers, 2 x login nodes, 1 x archive node 4 x UV20 fat nodes Rack 7-8: each with: 40 C1104-RP5 1U nodes with Xeon Phi ( total: 80 Xeon Phi nodes) Rack 9: 4 x UV20 Oracle servers with local storage NAS Storage: 1 x MIS server with 72 x 3TB HDD + 1 x MIS JBOD expansion with 72 x 3TB HDDs Total: 144 x 3TB HDDs = 14 x 8+2 LUNs + 4 hot spares = 319.2TB usable capacity NDSSL HPC Equipment Rack Placement

  6. 1G sw1 1G sw1 1G sw1 1G sw1 1G sw2 1G sw2 1G sw2 1G sw2 Bond0 Active-Backup Admin Nodes 10G Nodes x 8 10G Nodes x 8 10G Sw1 10G Sw2 1G sw1 1G sw1 1G sw1 1G sw1 Rack 1 Rack 2 Rack 3 Rack 4 Rack 6 Rack 7 Rack 8 Rack 9 Rack 5 not shown 10G switches are in rack 5 NDSSL HPC Management Network

More Related