760 likes | 782 Views
Explore Internet2's advancements in network architecture and monitoring tools for performance optimization. Learn about the global research network's initiatives in supporting network-based science and achieving end-to-end performance.
E N D
Network Developments and Network Monitoring in Internet2 Eric Boyd Director of Performance Architecture and Technologies Internet2
Overview • Internet2 Network • Performance Middleware: Supporting Network-Based Science • Internet2 Network Observatory
An Asset for the Community Internet2 Network: An Asset for the Community Universities Universities Researchers Researchers Regional Networks Regional Networks K-12 K-12 Industry Industry International International
Internet2 Network • Hybrid optical and IP network • Dynamic and static wavelength services • Fiber, equipment dedicated to Internet2; Level 3 maintains network and service level • Platform supports production services and experimental projects
Internet2 Network - Layer 1 Internet2 Network Optical Switching Node Level3 Regen Site Internet2 Redundant Drop/Add Site ESnet Drop/Add Site
Internet2 Network Deployment Internet2 Network Optical Switching Node Level3 Regen Site Internet2 Redundant Drop/Add Site ESnet Drop/Add Site
Internet2 Network Deployment Internet2 Network Optical Switching Node Level3 Regen Site Internet2 Redundant Drop/Add Site ESnet Drop/Add Site
Internet2 Network Deployment Internet2 Network Optical Switching Node Level3 Regen Site Internet2 Redundant Drop/Add Site ESnet Drop/Add Site
A New Wrinkle • Internet2 exploring a merger with National Lambda Rail (NLR) • Goal: Consolidate national higher education and research networking organizations • Technical team is exploring what the merged technical infrastructure will look like
Overview • Internet2 Network • Performance Middleware: Supporting Network-Based Science • Internet2 Network Observatory
Network-Based Science • Science is a global community • Networks links scientists • Collaborative research occurs across network boundaries • For the scientist, the value of the network is the achieved network performance • Scientists should not have to focus on the network; good end-to-end performance should be a given
Large Hadron Collider • International Physics facility located in CERN Switzerland • Major US involvement • 2 major US data repositories (PetaBytes/year) • 17 US Institutions provide data analysis and storage • 68 Universities and National Laboratories with scientists looking at the data • Dedicated transatlantic networks connect US to CERN • Advanced network services required over existing campus, connector/regional, and national networks
Cyberinfrastructure Cyberinfrastructure Middleware Performance Policy Security End to End
Achieving Good End-to-End Performance • Internet2 consists of: • Campuses • Corporations • Regional networks • Internet2 backbone network • Our members care about connecting with: • Other members • Government labs & networks • International partners • The Internet2 community cares about making all of this work
No other complaints Everything is AOK Talk to the other guys System Administrator LAN Administrator LAN Administrator System Administrator Campus Networking Campus Networking Backbone Gigapop Gigapop Identifying the Problem Hey, this is not working right! Others are getting in ok Not our problem Applications Developer Applications Developer The computer Is working OK Looks fine All the lights are green How do you solve a problem along a path? We don’t see anything wrong The network is lightly loaded
Status Quo • Performance is excellent across backbone networks • Performance is a problem end-to-end • Problems are concentrated towards the edge and in network transitions • We need to: • Diagnose: Understand limits of performance • Address: Work with members and application communities to address those performance issues
Vision: Performance Information is … • Available • People can find it (Discovery) • “Community of trust” allows access across administrative domain boundaries (AA) • Ubiquitous • Widely deployed (Paths of interest covered) • Reliable (Consistently configured correctly) • Valuable • Actionable (Analysis suggests course of action) • Automatable (Applications act on data)
Goal: No more mystery … • Increase network awareness • Set user expectations accurately • Reduce diagnostic costs • Performance problems noticed early • Performance problems addressed efficiently • Network engineers can see & act outside their turf • Transform application design • Incorporate network intuition into application behavior
Strategy: Build & Empower the Community Decouple the Problem Space: • Analysis and Visualization • Performance Data Sharing • Performance Data Generation Grow the Footprint: • Clean APIs and protocols between each layer • Widespread deployment of measurement infrastructure • Widespread deployment of common performance measurement tools
Tactics: Leverage position • Internet2 is leveraged to help provide diagnostic information for “US backbone” portion of problem • Create *some* diagnostic tools (BWCTL, NDT, OWAMP) • Make network data as public as is reasonable • Work on efforts to more widely make performance data available (perfSONAR) • Contribute to ‘base’ perfSONAR development (partnership with ESnet, Europe, and Brazil) • Contribute to standards for performance information sharing (Open Grid Forum Network Measurement Working Group) • Integrate ‘our’ diagnostic tools as ‘good’ example of perfSONAR services
From the scientist’s perspective On behalf of the scientist, network engineer or application can easily/automatically: • Discover additional monitoring resources • Authenticate locally • Authorized to use remote network resources to a limited extent • Acquire performance monitoring data from remote sites via standard protocol • Innovate where needed • Customize the analysis and visualization
Internet2 End-to-End Performance Initiative (E2Epi) • Includes: • Internet2 staff • Internet2 members • Federal partners • International partners • Building: • Performance monitoring tools • Performance middleware frameworks • Performance improvement tools
Support for E2Epi • Funded out of network revenues • Partnerships • Leveraging GÉANT2, ESnet, and RNP resources through consortium leadership • Grants • NSF Apps - Targeted Assistance and Instrumentation for Internet2 Applications • NSF SGER - Leveraging Internet2 Facilities for the Network Research Community • NSF SGER2 - Network Measurement for International Connections • NSF BTG - Bridging the Gap: End-to-End Networking for Landmark Applications • NLM Pilot - User Experience with the High Performance Internet Infrastructure: Critical Incidents of Success and Failure • NLM NDT - Enhancing the Web 100-based Network Diagnostic Tool
Current Activities • Analysis/Diagnostic tools • Performance tools • Software distributions to enable partner network organizations to participate • Google Summer of Code • New network deployment of measurement infrastructure on new observatory
Software Distributions • NPToolkit (Network Performance Toolkit) • Will include much of the following eventually • NDT (avail now) • OWAMP (avail now) • BWCTL/Thrulay (May) • AMI (Fall?) • Regular testing and collection for OWAMP/BWCTL • perfSONAR-PS (Earliest - Fall) • SNMP collection/archive • AMI archive • TopoS and L2Status • perfSONAR UI’s (Earliest - Winter)
Google Summer of Code 5 Projects • NDT enhancements • Phoebus protocol enhancements • Chrolog (user-space timestamp) • OWAMP (Java Client) • perfSONAR/cacti interface
OWAMP (3.0c) • One-way latencies • Full support of RFC 4656 • Deployment Status • Abilene: all remaining nms4 hosts • New network newy and chic (nms-rlat) • Software available at: http://e2epi.internet2.edu/owamp/
BWCTL (1.2b) • Throughput Test Controller • Pending Software release • Additional throughput tools • Iperf/thrulay/nuttcp • More tolerant of questionable clocks • Deployment Status • Abilene: open TCP testing • New network - awaiting new software release
What is perfSONAR? • Performance Middleware • perfSONAR is an international consortium in which Internet2 and GÉANT2 are founders and leading participants • perfSONAR is a set of protocol standards for interoperability between measurement and monitoring systems • perfSONAR is a set of open source web services that can be mixed-and-matched and extended to create a performance monitoring framework
perfSONAR Design Goals • Standards-based • Modular • Decentralized • Locally controlled • Open Source • Extensible • Applicable to multiple generations of network monitoring systems • Grows “beyond our control” • Customized for individual science disciplines
perfSONAR Integrates • Network measurement tools • Network measurement archives • Discovery • Authentication and authorization • Data manipulation • Resource protection • Topology
perfSONAR is a joint effort: ESnet GÉANT2 JRA1 Internet2 RNP ESnet includes: ESnet/LBL staff Fermilab Internet2 includes: University of Delaware Georgia Tech SLAC Internet2 staff GÉANT2 JRA1 includes: Arnes Belnet Carnet Cesnet CYNet DANTE DFN FCCN GRNet GARR ISTF PSNC Nordunet (Uninett) Renater RedIRIS Surfnet SWITCH perfSONAR Credits
R&E Networks Internet2 ESnet GÉANT2 European NRENs RNP Application Communities LHC Roll-out to other application communities in late 2007 Distributed Development Individual projects (10 before first release) write components that integrate into the overall framework Individual communities (5 before first release) write their own analysis and visualization software perfSONAR Adoption
perfSONAR-PS* • perfSONAR (Perl Services) Why? • Adoption of Java Services difficult • Many network administrators don’t do Java, but are fluent in Perl) • Services more directly targeted at the data available from Internet2 observatory deployment.
Overview • Internet2 Network • Performance Middleware: Supporting Network-Based Science • Internet2 Network Observatory
History and Motivation • Original Abilene racks included measurement devices • Included a single (somewhat large) PC • Early OWAMP, Surveyor measurements • Optical splitters at some locations • Motivation was primarily operations, monitoring, and management - understanding the network and how well it operates • Data was collected and maintained whenever possible • Primarily a NOC function • Available to other network operators to understand the network • It became apparent that the datasets were valuable as a network research tool
The Abilene Upgrade Network Rick Summerhill
Upgrade of the Abilene Observatory • An important decision was made during the Abilene upgrade process (Juniper T-640 routers and OC-192c) • Two racks, one of which was dedicated to measurement • Potential for research community to collocate equipment • Two components to the Observatory • Collocation - network research groups are able to collocate equipment in the Abilene router nodes • Measurement - data is collected by the NOC, the Ohio ITEC, and Internet2, and made available to the research community
An Abilene router node Power (48VDC) Power Measurement Machines (nms) Eth. Switch Out-of-band Space for Collocation! (M-5) Measurement(Observatory) Rack T-640
Houston Router Node - In this picture: Measurement machines Collocated PlanetLab machines Dedicated servers at each node
Example Research Projects • Collocation projects • PlanetLab – Nodes installed in all Abilene Router Nodes. See http://www.planet-lab.org • The Passive Measurement and Analysis Project (PMA) - The Router clamp. See http://pma.nlanr.net • Projects using collected datasets. See http://abilene.internet2.edu/observatory/research-projects.html • “Modular Strategies for Internetwork Monitoring” • “Algorithms for Network Capacity Planning and Optimal Routing Based on Time-Varying Traffic Matrices” • “Spatio-Temporal Network Analysis” • “Assessing the Presence and Incidence of Alpha Flows in Backbone Networks”
The New Internet2 Network • Expanded Layer 1, 2 and 3 Facilities • Includes SONET and Wave equipment • Includes Ethernet Services • Greater IP Services • Requires a new type of Observatory
The New Internet2 Observatory • Seek Input from the Community, both Engineers and Network Researchers • Current thinking is to support three types of services • Measurement (as before) • Collocation (as before) • Experimental Servers to support specific projects - for example, Phoebus (this is new) • Support different types of nodes: • Optical Nodes • Router Nodes
Existing Observatory Capabilities • One way latency, jitter, loss • IPv4 and IPv6 (“owamp”) • Regular TCP/UDP throughput tests – ~1 Gbps • IPv4 and IPv6; On-demand available (“bwctl”) • SNMP • Octets, packets, errors; collected 1/min • Flow data • Addresses anonymized by 0-ing the low order 11 bits • Routing updates • Both IGP and BGP - Measurement device participates in both • Router configuration • Visible Backbone – Collect 1/hr from all routers • Dynamic updates • Syslog; also alarm generation (~nagios); polling via router proxy