250 likes | 382 Views
Status Report on US networks at the Turn of the Century. Les Cottrell – SLAC & Stanford U. www.slac.stanford.edu/grp/scs/net/talk/us-net-status-2000.htm Presented at CHEP00, Padua Italy, February 9, 2000
E N D
Status Report on US networks at the Turn of the Century Les Cottrell – SLAC & Stanford U. www.slac.stanford.edu/grp/scs/net/talk/us-net-status-2000.htmPresented at CHEP00, Padua Italy, February 9, 2000 Partially funded by DOE/MICS Field Work Proposal on Internet End-to-end Performance Monitoring (IEPM), also supported by IUPAP
Overview • U.S. Networks: • Internet 2 • Federal networks, in particular ESnet • Commercial • Performance seen from U.S.: • Compare Internet2 vs. ESnet vs Commercial • Performance to Europe and rest of world • Trends • Summary
US National Networks • Internet 2 - universities • Abilene & vBNS backbones • Federal backbone networks • ESnet (DoE), DREN (DoD), NREN (NSF), NSI (NASA) ... • Commercial Internet Service Providers (ISPs) • Interconnection points (MAEs, NAPs, NGIXs, & colocation points …)
Internet 2 • A project by consortium of universities (UCAID) to: • foster development of advanced internet applications; • foster development internet technology itself; • provide a high performance network for general research. • Not a Government project; no direct Federal subsidy • Not a network itself: • The NSF-funded vBNS evolved into a ‘pre’ Internet-2 backbone; • Abilene is the UCAID-sponsored Internet-2 backbone.
Internet 2 Membership • Internet 2 • 170 universities • ~ 10-12 non-university members including CERN • other networks can connect as affiliates; • U.S. National Labs are not members of UCAID: • National Lab Internet-2 participation assumed thru ESnet • vBNS • 101 institutions connected • 22 peer networks • Abilene • 96 participants, 75 connected • 15 peer networks
Internet 2 AUP • Internet 2 Acceptable Use Policy (AUP) prohibits transit traffic between affiliates: • ESnet site with Internet-2 member site is ‘OK’ to use I2; • ESnet site to other affiliated network site via Internet-2 backbone is NOT permitted. • Internet 2 AUP also prohibits “commodity” internet traffic! • Traffic to another Internet-2 site routed via Internet-2 backbone; • Traffic to a non-Internet-2 site must be routed via commercial ISP; • Requires universities to have a separate Internet connection for commodity traffic.
Abilene San Fran. Seattle UCDavis CalRen UCSF UCB Stanford UCSC ESnet • 5500 miles deployed, backbone operates at speeds up 2.4Gbps. Interconnections at 155 & 622Mbps • Universities connect to GigaPoPs, GigaPoPs connect to backbone • Peer with: APAN/Transpac, CA*net-2, DANTE, DFN, DREN, ESnet, ILAN, INFN, JANET, NACSIS, NORDunet, NISN, NREN, RENATER, SingAREN, SURFnet, vBNS … • Red=N.America, Blue=Europe, Green=Asia
vBNS • ATM-based OC12 (622Mbps) backbone, with 2.4 Gbps IP-over-SONET segments on parts of the backbone: • 45Mbps is the minimum connection line speed; • 98 connections as of 6/15, with 7 more pending; • Run by MCI; subsidized out of NSF NGI funds; • NSF vBNS contract expires April 1st, 2000; • June 99 MCI announces vBNS+ a 5 year agreement with EDUCAUSE • 1750 higher education institutions members • vBNS+ no restrictive NSF imposed AUP, • Connection speeds from 1.5Mbps - 2.5Gbps • peering: with vBNS requires NSF authorization, none at moment with Abilene, will offer to FEDnet and International nets
ESnet • DOE Energy Sciences Research Network • Connects DOE ER labs and universities with major DoE funded projects • ~ 50 sites • Mainly 155 Mbps backbone with some 622Mbps links • Peers with other major networks • 13 Internet Interconnect points • Peering exchange at MAE-West, MAE-East, Sprint NAP, Ameritech NAP, PacBell NAP • International connections • CERN, DFN, INFN, JAERI, KEK, Moscow, NIFS
ESnet • 100% growth/year since 1990 • New contract, Sprint did not bid • New contract with Qwest announced January 4, 2000 • 1 year transition, 2 concurrent contracts for coming year means funding tight • Same supplier as US-CERN link (KPN-Qwest) • Initial deployment ATM based • ESnet3 backbone to be Tbit/sec by 2003-2005 • 5 major hubs see next transparency ...
ESnet-3 Initial Configuration DFN INFN DANTE Canada France CERN KEK/China OC3? Japan/Russia JAnet SURFnet NORDUnet Abilene SEA PNNL ANL CHI FNAL CHI-NAP PPPL BNL LBNL NYC AMES TELEHOUSE NERSC 60-HUD SNLL MIT LLNL SNV FULL-MESHED ATM CORE FIX-W SLAC JLAB PB-NAP DC MAE-E GTN INEEL JGI DC Offices MAE-W ORNL SNLA YUCCA-MT ATL ALB PANTEX LANL ORN OC48-ATM SRS OC12-ATM ASIG GA OC3-ATM (SDSC) T3-ATM (SAIC) GA OC48-SONET (BECHTEL) OC12-SONET OC3-SONET T3 Courtesy of Jim Leighton/ESnet
Testbeds for U.S. NRENs • “Foster Development of Internet Technology” • IPv6 = Next Generation Internet • volunteers see warrenm@slac.stanford.edu • QoS - VoIP, multimedia and data transfer • Computing & Data grids • Collaboratories - video, virtual reality, electronic notebooks, multicast .. • Middleware - PKI, directories ...
Commercial Internet • Important to HENP • ~ 18% of US HEP universities still rely solely on commercial ISPs for internet access: • Internet-2 participation, even subsidized, isn’t cheap… • Critical information needed from commercial sites • Quality of US commercial Internet Service is improved: • Commercial ISPs have been keeping their backbone capacity in line with (or ahead of…) demand; • Network Access Point (NAP) congestion is down. • ISPs match research networks technologically: • ISPs are ahead in rollout of high bandwidth links; • ISPs are pursuing Quality of Service solutions
Commercial traffic: Ames Internet Exchange (AIX)https://anala.caida.org/AIX/ • TCP 90%, UDP 10%, Web ~ 55%, FTP ~ 5%, mail ~ 3% • Game traffic represents noticeable portion • Quake & Starcraft account for 5% in summer, 2-3% term-time • Fairly constant (0.5%) • Real audio declining (factor 2 in 6 months, now 1%) • IPSEC traffic, small (< 0.2%) but growing (factor 3 in 6 months) • Spikes in ICMP (security scans?)
Performance PingER • Measurements from • 28 monitors in 15 countries • Over 500 remote hosts • 72 countries (covers all 56 PDG booklet countries) • Over 1200 monitor-remote site pairs • Over 50% of HENP collaborator sites are explicitly monitored as remote sites by PingER project • Atlas (37%), BaBar (68%), Belle (23%), CDF (73%), CMS (31%), D0 (60%), LEP (44%), Zeus (35%), PPDG (100%), RHIC(64%)
How are the U.S. Nets doing? In general performance is good (i.e. <= 1%). Edu (vBNS/Abilene) is catching up with ESnet XIWT (70% .com) 3-5 times worse than ESnet | I2
Europe seen from U.S. Monitor site Beacon site (~10% sites) HENP country Not HENP Not HENP & not monitored 200 ms 650ms 1% loss 7% loss 10% loss
Asia seen from U.S. 10% loss 3.6% loss 0.1% loss 250ms 640 ms 450 ms
Latin America, Africa & Australasia 4% Loss 170 ms 220 ms 700ms 2% Loss 350 ms
Bulk transfer - Performance Trends Bandwidth TCP < 1460/(RTT * sqrt(loss)) Note: E. Europe NOT catching up
Summary • US HEP research network research environment is improving… • The Internet-2 project is already having positive results for collaborative research: • More research universities have high bandwidth, low latency access to major U.S. research facilities; • Mission-specific research networks have something to direct improving university access efforts at…; • Testbed projects emerging for new network technologies. • International performance from US to sites outside W. Europe, Japan, Korea is generally poor to bad
More Information • ESnet home page • http://www.es.net/ • Internet 2 home page • http://www.internet2.edu/ • vBNS+ home page • http://www.vbns.net/vBNS+/index.html • IEPM/PingER home site • http://www-iepm.slac.stanford.edu/ • ICFA-SCIC Homepage • http://www.hep.net/ICFA/index.html
Peering PingER • Not always optimal • paths may go through congested exchange points -increased loss • paths may be very indirect (e.g. KEK to SLAC was via NY) - adds 80 msec to RTT Performance 28 monitors in 15 countries Over 500 remote hosts Over 1200 pairs 72 countries Over 50% HENP sites are monitored directly