1 / 7

Virginia Tech NDSSL-HPC Proposed System Diagrams 06-25-2013

Virginia Tech NDSSL-HPC Proposed System Diagrams 06-25-2013. Diagram Page Architecture Diagram 2 VM Server Storage Diagram 3 Components 4,5 Equipment Rack Placement 6 Management Network 7. 10 GbE Network.

inez-leblanc
Download Presentation

Virginia Tech NDSSL-HPC Proposed System Diagrams 06-25-2013

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Virginia Tech NDSSL-HPC Proposed System Diagrams 06-25-2013 DiagramPage Architecture Diagram 2 VM Server Storage Diagram 3 Components 4,5 Equipment Rack Placement 6 Management Network 7

  2. 10 GbE Network 1GbE Management Net Switches (stacked) 16 x C2112 2U Quad Node Chassis 80 nodes each with: 2 x E5-2670 CPUs 64GB Memory (4GB/core) 1 x 1TB HDD FDR IB 2 x 48-port 10GbE Switches FDR IB Fabric Mellanox director chassis switch 432 leaf ports expandable to 648 ports 1GbE Mgmt switch 60 x C1104-RP5 1U Xeon PhiNodes 2 x E5-2670 CPUs 64 GB Memory (4GB/core) 1 x 1TB HDD 2 x C2108-RP2 Admin nodes 2 x Intel Xeon E5 2670 2.6 GHz 128GB Mem 2x1TB HDD RAID-1 1 x PCIE-CONNX3-1P (FDR IB) 2 x 10GbE ports (SR) Console/KVM 4 x C2108 VM servers, 2 x Intel Xeon E5 2670 2.6 GHz 128GB Mem 3x1TB HDD RAID-1 1 x PCIE-CONNX3-1P (FDR IB) 2 x 10GbE ports (SR) IS5000 RAID w/24 HDD 4 x C2108-RP2 Login nodes 2 x Intel Xeon E5 2670 2.6 GHz 128GB Mem 2x1TB HDD RAID-1 1 x PCIE-CONNX3-1P (FDR IB) 2 x 10GbE ports (SR) 1 xC2108 archive node 2 x Intel Xeon E5 2670 2.6 GHz 128GB Mem 2x1TB HDD RAID-1 1 x PCIE-CONNX3-1P (FDR IB) 2 x 10GbE ports (SR) 2 x C2108-RP2 Gateway nodes 2 x Intel Xeon E5 2670 2.6 GHz 128GB Mem 2x1TB HDD RAID-1 1 x PCIE-CONNX3-1P (FDR IB) 2 x 10GbE ports (SR) 4 x UV20 fat nodes 4 x E5-4650 2.6GHz 8 core 1.5TB memory, 1 x 1TB HDD, FDR 1 x 1GbE Mgmt switch 4 x UV20 Oracle Servers each with: 4 x E5-4650 8 core 2.6GHz 16 x 16GB DIMMs (256GB) 2 x 1TB HDD RAID-1, with BBU 1 x FDR IB 2 x 10GbE RAID controller IS2112 JBOD with: 10 x 3TB HDD 8+2 LUN = 24TB Attached storage NFS Server 1 MIS Server with 72 3TB HDD 1 MIS JBOD Expansion with 72 3TB HDDs Cable Legend 1 x 1GbE Mgmt switch 20 x C2110-RP5 2U Xeon Phi Nodes 2 x E5-2670 CPUs 64GB Memory (4GB/core) 1 x 500GB HDD 1 x 5110P Xeon Phi accelerators FDR IB 10GbE Network 1GbE Management Network FDR Infiniband Fabric NDSSL HPC System Architecture

  3. Brocade 24-port FC8 Switch IS5000 RAID Systems With 24 x 2.5-inch HDD C2108 VM Servers . . . . . . . . . . . . NDSSL HPC VM Server Storage Architecture

  4. C2112-4RP4 2U Quad Node Chassis • 4 x compute nodes in 2U, each with: • - 2 x E5 CPU sockets • 16 x 1600/1333/1066/800 DIMM Slots • - 3 x 3.5” HDD per server • 1 x PCIe 3.0 x16 supportint low profile slot • 1 x FDR Infiniband port on board • - 2 x 1GbE NICs • - 2 x 1200w redundant power supplies C1104G-RP5 Accelerator node 1U 2 x E5 CPU sockets 8 x 1600/1333/1066/800 DIMM Slots 4 x 2.5” HDD 3 x PCIe 3.0 x16 supporting 3 GPGPU or Xeon Phi cards 1 x PCIe 3.0 x8 slot 2 x 1GbE NICs 2 x 1800w Platinum level redundant power supplies • C2108 • - 2U dual-socket E5-2600 series server • - HPC management, head node, I/O nodes • - Up to 8 x 3.5” hot-swap drives • - Up to five expansion PCIe slots • - 1+1 Redundant AC Power Supplies • UV20 2U Quad Socket Server • - 4 socket E5-4600 series server • - 48 DIMM slots (1.5TB mem with 32GB DIMMs) • - Up to 8 x 2.5” HDDs • 8 PCIe slots • Redundant power NDSSL HPC Components

  5. SGI IS2112 2U 12-drive JBOD • 12 3.5-inch HDD • SAS Interfaces SGI Modular InfiniteStorage Server • Storage Server Options • Integrated into 4U chassis • 1 or 2 motherboards (Sandy bridge Platform) • 2 socket LGA ES-2600 (E5-2600 Series) • Each chassis supports SATA/SAS/SSD drive types: • Up to 72 x 3.5” drives • Up to 144 x 2.5” drives • Dual GbE onboard. Optional 2 port 10GbE, 2 port GbE, or 4 port 8Gb FC PCIe cards • JBOD Option • Up to 81 SATA/SAS/SSD drives (162 w/2.5” SFF) • Up to 8 SAS connections IS5000 RAID System • 24 x 2.5-inch HDD with Dual Active RAID controllers • FC interfaces • Expandable to 192 drives NDSSL HPC Components

  6. Rack 7 Rack 9 Rack 8 Rack 1 Rack 2 Rack 3 Rack 4 Rack 5 Rack 6 Rack 10 Rack 11 Rack 12 Rack 13 Racks 1-6 each with : 16 x C2112 Quad-node compute chassis (total:384 nodes) Rack 7 : 12 x C2112 Quad-node compute chassis (total: 48 nodes) Rack 8: IB switches, 10GbE switches, 4 x C2112 Quad-node chassis (total: 16 nodes) Rack 9: 1 x archive 2 x gateway, 2 x VM servers, 2 x admin 1 x console 4 x login nodes, 4 x VM servers each w/ IS5000 w/24 HDD Rack 10: 8 x C2112 Quad-node chassis (total: 32 nodes) 4 x UV20 Oracle servers with local storage NAS Storage: 1 x MIS server with 72 x 3TB HDD + 1 x MIS JBOD expansion with 72 x 3TB HDDs Total: 144 x 3TB HDDs = 14 x 8+2 LUNs + 4 hot spares = 319.2TB usable capacity Rack 11: 15 x C2112 Quad-node chassis (total: 60 nodes) = 540 compute Rack 12-13: each with: 30 C1104-RP5 1U nodes with Xeon Phi ( total: 60 Phi nodes) = 60 Phi = 600 Nodes NDSSL HPC Equipment Rack Placement

  7. 1G sw1 1G sw1 1G sw1 1G sw1 1G sw2 1G sw2 1G sw2 1G sw2 Bond0 Active-Backup Admin Nodes 10G Nodes x 8 10G Nodes x 8 10G Sw1 10G Sw2 1G sw1 1G sw1 1G sw1 1G sw1 Rack 1 Rack 2 Rack 3 Rack 4 Rack 6 Rack 7 Rack 8 Rack 9 . . . NDSSL HPC Management Network

More Related