1 / 21

Scientific Networking: The Cause of and Solution to All Problems

April 14 th 2011 - Workshop on High Performance Applications of Cloud and Grid Tools Jason Zurawski, Research Liaison. Scientific Networking: The Cause of and Solution to All Problems. And Now for Something Completely Different.

lita
Download Presentation

Scientific Networking: The Cause of and Solution to All Problems

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. April 14th 2011 - Workshop on High Performance Applications of Cloud and Grid Tools Jason Zurawski, Research Liaison Scientific Networking: The Cause of and Solution to All Problems

  2. And Now for Something Completely Different • Topics so far on the core design and operation of Grid/Cloud infrastructures • Fertile area for work • Lots of advancement – being driven by scientific needs (e.g. Physics, Biology, Climate, etc.) • Achilles Heal of Grid/Cloud computing = infrastructure that links the components • Distributed CPU, Disk, and Users • Earlier efforts to improve the overall performance (e.g. Logistical Networking) • Role of Networking • “Under the hood”. Should enable science, but stay out of the way • Lots of advancement, highlight 2 efforts today: • DYNES – Dynamic Networking to end sites • LHCONE – Dedicated resources for data movement

  3. DYNES • Data movement to support science: • Increasing in size (100s of TBs in the LHC World) • Becoming more frequent (multiple times per day) • Reaching more consumers (VO size stands to increase) • Time sensitivity (data may grow “stale” if not processed immediately) • Traditional networking: • R&E or Commodity “IP” connectivity is subject to other users • Supporting large sporadic flows is challenging for the engineers, and frustrating for the scientists

  4. DYNES • Solution • Dedicated bandwidth (over the entire end to end path) to move scientific data • Invoke this “on demand” instead of relying on permanent capacity (cost, complexity) • Exists in harmony with traditional IP networking • Connect to facilities that scientists need to access • Integration with data movement applications • Invoke the connectivity when the need it, based on network conditions • Prior Work • “Dynamic Circuit” Networking – creation of Layer 2 point to point VLANs • Transit the Campus, Regional, and Backbone R&E networks • Software to manage the scheduling and negotiation of resources

  5. DYNES • NSF Funded “Cyber-Instrument” • Internet2/Caltech/University of Michigan/Vanderbilt University • Provide equipment and software to extend the Internet2 ION service into Campus and Regional networks • Build using the OSCARS IDC software (based on work in OGF NSI Working Group) • perfSONAR Monitoring (based on work in the OGF NM, NMC, and NML Working Groups) • FDT (Fast Data Transfer) data movement

  6. DYNES • Deployment Targets: • 25 End Sites • 8 Regional Networks • Collaboration with like minded efforts (DoE ESCPS) • Plans to consider provisional applications (send email to dynes-questions@internet2.edu if you are interested) • Supporting all science - early focus on Physics (LHC) sites

  7. DYNES Infrastructure Overview

  8. DYNES Standard Equipment • Inter-domain Controller (IDC) Server and Software • IDC creates virtual LANs (VLANs) dynamically between the FDT server, local campus, and wide area network • Dell R410 (1U) Server • Fast Data Transfer (FDT) server • Fast Data Transfer (FDT) server connects to the disk array via the SAS controller and runs the FDT software • Dell R510 (2U) Server • DYNES Ethernet switch options (emerging): • Dell PC6248 (48 1GE ports, 4 10GE capable ports (SFP+, CX4 or optical) • Dell PC8024F (24 10GE SFP+ ports, 4 “combo” ports supporting CX4 or optical)

  9. DYNES Data Flow Overview

  10. DYNES Current Status • 4 Project Phases • Phase 1: Planning (Completed in Feb 2011) • Phase 2: Initial Deployment (Feb 2011 through July 2011) • Phase 3: Full Deployment (July 2011 through Sept 2011) • Phase 4: Testing and Evaluation (Oct 2011 through August 2012) • A draft DYNES Program Plan document is available with additional details on the project plan and schedule: • http://www.internet2.edu/dynes • Questions can be sent to the mailing list: • dynes-questions@internet2.edu

  11. Inductive Step • Campus connectivity is just one part of a solution • Campus has been the traditional bottleneck • Using a traffic engineering solution like DYNES will connect sites on a national level in a point to point fashion • What about transit to non-DYNES sites? What about other countries? • Resources on a national and international level • Investment in networking is still strong • Backbone capacity upgrades coupled with availability of new sites (U.S. UCAN)

  12. Inductive Step • Scientific networking needs to be pervasive • Availability where the science is, e.g. “everywhere” • Linking the resources that require this capability • Clusters and Supercomputers • Data stores • Scientific Instruments (Telescopes, Colliders). • LHC Community: • Pro-active in terms of network preparedness • Designing next generation connectivity options to meet the needs of the VO as a whole • Sensitive to funding, but always wanting the best for the community to support scientific activity for the next 10+ years

  13. LHC Open Network Environment (LHCONE) • The goal of LHCONE is to provide a collection of access locations that are effectively entry points into a network that is private to the LHC • It is anticipated that LHCONE access locations will be provided in countries / regions in a number and location so as to best address the issue of ease of access • In the US, LHCONE access locations might be co-located with the existing R&E exchange points and/or national backbone nodes • A similar situation exists in Europe and Southeast Asia.

  14. LHCONE – North America • Proposed installation of two nodes to provide immediate service • Chicago • New York • Interconnected via Internet2 IP Network • Generally has 9 Gbps of available capacity for initial best-effort traffic use • Potential to provide a dedicated backbone circuit to provide 10G of capacity just for LHCONE (or shared with other scientific VOs) • It is certain that this bandwidth will grow as the Internet2 network upgrades its backbone links to 100 Gbps in 2011.

  15. Sample Architecture and Connectivity

  16. LHCONE Access Methodology • Designed to be “come as you are” • Network connectivity is expensive, budgets are tight • Funding opportunities can accommodate increased connectivity in the future • Short term is to offer several methods • There will be three primary methods of connection to the LHCONE-NA architecture. • Direct Connection to LHCONE-NA Nodes • Layer2 Connectivity via Internet2 Network (e.g. ION) • Layer3 Connectivity via Internet2 Network

  17. Direct Connection to LHCONE-NA Nodes • Normally an expensive option, but one that provides the greatest access • Physical connection from end site to connection point • Initially Chicago and New York, others over time • 10GE anticipated • Mimics the current Tier1 to Tier2 connectivity via static circuits

  18. Layer2 Connectivity via Internet2 Network • Two basic approaches discussed • Static connectivity into Internet2 at some other location (e.g. not in Chicago or New York) • Facilitates end sites with this network option already in place • Dynamic connectivity via the ION service • Inexpensive way to manage traffic through existing network connections • Takes advantage of newly deployed infrastructure for DYNES

  19. Layer3 Connectivity via Internet2 Network • Option that will appeal to many Tier3 facilities without dedicated connections for science traffic • Cost effective • Additional hardware is not needed • In most cases, R&E IP access is sufficient (e.g. 10G or less) • Use the R&E connectivity of their institution • Best effort in terms of bandwidth • Harder to manage traffic flows

  20. Conclusions/Next Steps • DYNES is in deployment, demonstrations at major conferences expected (SC11) • LHCONE Demonstration in Summer 2011 • http://lhcone.net • LHCONE NA meeting scheduled for May 2011 in Washington DC (participation welcome) • Future Work • LHCONE is just the beginning • Opportunity to provide a nationwide “science focused” infrastructure for all VOs • Dedicated Bandwidth • Cutting edge technology (Open Flow, etc.) • Integration with International efforts • Open Access and Open Standards

  21. Scientific Networking: The Cause of and Solution to All Problems April 18th 2011, Workshop on High Performance Applications of Cloud and Grid Tools Jason Zurawski, Research Liaison For more information, visit http://www.internet2.edu

More Related