1 / 10

NCCS Network Roadmap

NCCS Network Roadmap. Josh Lothian High Performance Computing Operations National Center for Computational Sciences. NCCS network roadmap. Summary: Hybrid Ethernet/Infiniband (IB) network to provide both high-speed wide-area connectivity and uber-speed local-area data movement. 2007

komar
Download Presentation

NCCS Network Roadmap

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. NCCS Network Roadmap Josh Lothian High Performance Computing OperationsNational Center for Computational Sciences

  2. NCCS network roadmap Summary: Hybrid Ethernet/Infiniband (IB) network to provide both high-speed wide-area connectivity and uber-speed local-area data movement • 2007 • Scaled Ethernet to meet wide-area needs and current day local-area data movement • Developed wire-speed, low-latency perimeter security to fully utilize 10Gb production and research WAN connections • Began IB LAN deployment • 2008 • Continue building out IB LAN infrastructure to satisfy the file system needs of the Baker system • Test Lustre on IB/WAN for possible deployment • Increase security on 10Gb WAN connections

  3. NCCS network roadmap summary Ethernet core scaled to match wide-area connectivity and archive Infiniband core scaled to match central file system and data transfer Lustre Baker Gateway Ethernet [O(10Gb/s)] Infiniband [O(100 Gb/s)] Jaguar High-Performance Storage System (HPSS)‏ Viz

  4. NCCS WAN overview Ciena CoreStream Ciena CoreDirector TeraGrid Juniper T320 DOE UltraScience/ NSF CHEETAH Force10 E600 Juniper T640 CCS Cisco 6509 NSTG CCS OC-192 to Internet 2 OC-192 to Esnet Internal 10Gb Link Internal 1Gb Link OC-192 to TeraGrid 2 x OC-192 to USN OC-192 to CHEETAH

  5. Jaguar NCCS Network 2007 Infiniband Ethernet Voltaire ISR2012 32 DDR 32 Spider30 20 DDR 48 48 Spider10 48 6-12 DDR 64 DDR 48 Cisco 7024D 48 Force10 E1200 Viz 48 32 DDR 32 4 HPSS 6-12 DDR 4 48 24 SDR Voltaire ISR9288 4 Devel 48 87 SDR 48 E2E 48 24-48 SDR 20

  6. NCCS IB network 2008 24 SDR Voltaire ISR9288 Devel 87 SDR Jaguar E2E 48-96 SDR (16-32 SDR/link)‏ 64 DDR 20 DDR 20 DDR Viz Spider10 32 DDR Voltaire ISR2012 50 DDR HPSS Spider30 50 DDR Cisco 7024D 32 DDR 300 DDR (50 DDR/link)‏ Baker Spider240

  7. 2007 milestones April 2007—Completed 10 G WAN upgrade June 2007—Completed VL/QOS testing November 2007—Complete fiber IB cable testing December 2007—IBoIP/SDP decision point

  8. 2008 milestones January—4x QDR testing February—Deploy flow-based perimeter security February—Complete 4X QDR testing March—Enable routing of XT3 HSN to production network June—Deploy redundant links between IB switches

  9. 4x Infiniband SDR OC-192 SONET Lustre End-to-End IB over WAN testing Ciena CD-CI (SNV)‏ • Placed 2 x Obsidiam Longbow devices between Voltaire 9024 and Voltaire 9288 • Provisioned loopback circuits of various lengths on the DOE UltraScience Network and ran test • RDMA Test Results: Local: 7.5 Gbps (Longbow to Longbow) ORNL ↔ ORNL (0.2 mile): 7.5 Gbps ORNL ↔ Chicago (1400 miles): 7.46 Gbps ORNL ↔ Seattle (6600 miles): 7.23 Gbps ORNL ↔ Sunnyvale (8600 miles): 7.2 Gbps DOE UltraScience Network Obsidian Longbow Obsidian Longbow Ciena CD-CI (ORNL)‏ Voltaire 9288 Voltaire 9024 20 81

  10. Contact Josh Lothian High Performance Computing Operations National Center for Computational Sciences (865) 241-5563 lothian@ornl.gov 10 Lothian_Networking_SC07

More Related