1 / 21

ESnet Update Winter 2008 Joint Techs Workshop

ESnet Update Winter 2008 Joint Techs Workshop. Joe Burrescia ESnet General Manager. Energy Sciences Network Lawrence Berkeley National Laboratory. January 21, 2008. Networking for the Future of Science. ESnet 3 with Sites and Peers (Early 2007). SINet (Japan) Russia (BINP). CERN

magee
Download Presentation

ESnet Update Winter 2008 Joint Techs Workshop

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. ESnet Update Winter 2008 Joint Techs Workshop Joe Burrescia ESnet General Manager Energy Sciences Network Lawrence Berkeley National Laboratory January 21, 2008 Networking for the Future of Science

  2. ESnet 3 with Sites and Peers (Early 2007) SINet (Japan) Russia (BINP) CERN (USLHCnetDOE+CERN funded) GÉANT - France, Germany, Italy, UK, etc PNNL SEA NERSC SLAC BNL ANL MIT LIGO INEEL LLNL LBNL MAN LANAbilene Starlight SNLL CHI-SL TWC JGI OSC GTNNNSA Lab DC Offices CHI ATL PPPL JLAB FNAL AMES Equinix Equinix ORNL SRS SNV SDN LANL SNLA DC DOE-ALB NASAAmes PANTEX NOAA ORAU OSTI ARM NSF/IRNCfunded YUCCA MT BECHTEL-NV GA Abilene Abilene Abilene SDSC MAXGPoP KCP Allied Signal UNM R&Enetworks AMPATH(S. America) AMPATH(S. America) NREL SNV Japan (SINet) Australia (AARNet) Canada (CA*net4 Taiwan (TANet2) Singaren ESnet Science Data Network (SDN) core CA*net4 France GLORIAD (Russia, China)Korea (Kreonet2 MREN Netherlands StarTapTaiwan (TANet2, ASCC) PNWGPoP/PAcificWave AU NYC ESnet IP core: Packet over SONET Optical Ring and Hubs MAE-E SNV PAIX-PA Equinix, etc. AU ALB 42 end user sites ELP Office Of Science Sponsored (22) International (high speed) 10 Gb/s SDN core 10G/s IP core 2.5 Gb/s IP core MAN rings (≥ 10 G/s) Lab supplied links OC12 ATM (622 Mb/s) OC12 / GigEthernet OC3 (155 Mb/s) 45 Mb/s and less NNSA Sponsored (12) Joint Sponsored (3) Other Sponsored (NSF LIGO, NOAA) Laboratory Sponsored (6) Specific R&E network peers Other R&E peering points commercial peering points ESnet core hubs IP Abilene high-speed peering points with Internet2/Abilene

  3. ESnet 3 Backbone as of January 1, 2007 10 Gb/s SDN core (NLR) 10/2.5 Gb/s IP core (QWEST) MAN rings (≥ 10 G/s) Lab supplied links Future ESnet Hub ESnet Hub Seattle New York City Sunnyvale Chicago Washington DC San Diego Albuquerque Atlanta El Paso

  4. ESnet 4 Backbone as of April 15, 2007 10 Gb/s SDN core (NLR) 10/2.5 Gb/s IP core (QWEST) 10 Gb/s IP core (Level3) 10 Gb/s SDN core (Level3) MAN rings (≥ 10 G/s) Lab supplied links Future ESnet Hub ESnet Hub Seattle Boston New York City Clev. Sunnyvale Chicago Washington DC San Diego Albuquerque Atlanta El Paso

  5. ESnet 4 Backbone as of May 15, 2007 10 Gb/s SDN core (NLR) 10/2.5 Gb/s IP core (QWEST) 10 Gb/s IP core (Level3) 10 Gb/s SDN core (Level3) MAN rings (≥ 10 G/s) Lab supplied links Future ESnet Hub ESnet Hub Seattle Boston Boston New York City Clev. Clev. Sunnyvale Chicago Washington DC SNV San Diego Albuquerque Atlanta El Paso

  6. ESnet 4 Backbone as of June 20, 2007 10 Gb/s SDN core (NLR) 10/2.5 Gb/s IP core (QWEST) 10 Gb/s IP core (Level3) 10 Gb/s SDN core (Level3) MAN rings (≥ 10 G/s) Lab supplied links Future ESnet Hub ESnet Hub Seattle Boston Boston New York City Clev. Sunnyvale Denver Chicago Washington DC Kansas City San Diego Albuquerque Atlanta El Paso Houston

  7. ESnet 4 Backbone August 1, 2007 (Last JT meeting at FNAL) 10 Gb/s SDN core (NLR) 10/2.5 Gb/s IP core (QWEST) 10 Gb/s IP core (Level3) 10 Gb/s SDN core (Level3) MAN rings (≥ 10 G/s) Lab supplied links Future ESnet Hub ESnet Hub Seattle Boston Boston New York City Clev. Clev. Sunnyvale Denver Chicago Washington DC Kansas City Los Angeles San Diego Albuquerque Atlanta El Paso Houston Houston

  8. ESnet 4 Backbone September 30, 2007 10 Gb/s SDN core (NLR) 10/2.5 Gb/s IP core (QWEST) 10 Gb/s IP core (Level3) 10 Gb/s SDN core (Level3) MAN rings (≥ 10 G/s) Lab supplied links Future ESnet Hub ESnet Hub Seattle Boston Boston Boise New York City Clev. Clev. Sunnyvale Denver Chicago Washington DC Kansas City Los Angeles San Diego Albuquerque Atlanta El Paso Houston Houston

  9. ESnet 4 Backbone December 2007 10 Gb/s SDN core (NLR) 2.5 Gb/s IP Tail (QWEST) 10 Gb/s IP core (Level3) 10 Gb/s SDN core (Level3) MAN rings (≥ 10 G/s) Lab supplied links Future ESnet Hub ESnet Hub Seattle Boston Boston Boise New York City Clev. Clev. Denver Sunnyvale Chicago Washington DC Kansas City Los Angeles Nashville San Diego Albuquerque Atlanta El Paso Houston Houston

  10. ESnet 4 Backbone December, 2008 DC 10 Gb/s SDN core (NLR) 10/2.5 Gb/s IP core (QWEST) 10 Gb/s IP core (Level3) 10 Gb/s SDN core (Level3) MAN rings (≥ 10 G/s) Lab supplied links Future ESnet Hub ESnet Hub Seattle Boston X2 New York City Clev. X2 X2 X2 Sunnyvale Denver Chicago Washington DC X2 X2 Kansas City X2 Los Angeles Nashville San Diego Albuquerque Atlanta El Paso Houston Houston

  11. ESnet Provides Global High-Speed Internet Connectivity for DOE Facilities and Collaborators (12/2007) SINet (Japan) Russia (BINP) CERN (USLHCnet:DOE+CERN funded) GÉANT - France, Germany, Italy, UK, etc PacWave StarlightUSLHCNetNLR PacWave Internet2NYSERNetMAN LAN BNL IARC Abilene DOE GTNNNSA Abilene KAREN / REANNZ Internet2 SINGAREN ODN Japan Telecom America CHIC DENV SUNN SUNN JLAB PPPL Equinix Equinix SRS SNV1 LANL SNLA DOE-ALB WASH NASAAmes PANTEX MAXGPoP NLR ORAU NOAA OSTI ARM YUCCA MT NSF/IRNCfunded USHLCNet to GÉANT BECHTEL-NV GA Allied Signal KCP NASH ALBU R&Enetworks AMPATH(S. America) ATLA UNM AMPATH(S. America) ELPA NREL Japan (SINet) Australia (AARNet) Canada (CA*net4 Taiwan (TANet2) Singaren KAREN/REANNZ ODN Japan Telecom America NLR-Packetnet Abilene/I2 CA*net4 France GLORIAD (Russia, China)Korea (Kreonet2 MREN StarTapTaiwan (TANet2, ASCC) SEA AU PNNL CHI-SL LIGO MIT/PSFC NEWY Lab DC Offices Salt Lake INL FNAL LVK JGI ANL LLNL SNLL LBNL DOE NETL NERSC AMES SLAC PAIX-PA Equinix, etc. ORNL SDSC AU ~45 end user sites International (1-10 Gb/s) 10 Gb/s SDN core (I2, NLR) 10Gb/s IP core MAN rings (≥ 10 Gb/s) Lab supplied links OC12 / GigEthernet OC3 (155 Mb/s) 45 Mb/s and less Office Of Science Sponsored (22) NNSA Sponsored (13+) Joint Sponsored (3) Other Sponsored (NSF LIGO, NOAA) Laboratory Sponsored (6) Specific R&E network peers commercial peering points Geography isonly representational Other R&E peering points ESnet core hubs

  12. ESnet4 1625 miles / 2545 km Production IP core (10Gbps) SDN core (20-30-40-50 Gbps) MANs (20-60 Gbps) or backbone loops for site access International connections Primary DOE Labs High speed cross-connectswith Ineternet2/Abilene Possible hubs 2700 miles / 4300 km Core networks 50-60 Gbps by 2009-2010 (10Gb/s circuits),500-600 Gbps by 2011-2012 (100 Gb/s circuits) Canada (CANARIE) CERN(30+ Gbps) Canada (CANARIE) Asia-Pacific Asia Pacific CERN(30+ Gbps) GLORIAD(Russia and China) USLHCNet Europe (GEANT) Asia-Pacific Science Data Network Core Seattle Boston Chicago IP Core Boise Australia New York Kansas City Cleveland Denver Washington DC Sunnyvale Atlanta Tulsa LA Albuquerque Australia South America (AMPATH) San Diego Houston South America (AMPATH) IP core hubs Jacksonville SDN hubs Core network fiber path is~ 14,000 miles / 24,000 km

  13. A Tail of Two ESnet4 Hubs MX960 Switch 6509 Switch T320 Routers T320 Router Sunnyvale Ca Hub Chicago Hub ESnet’s SDN backbone is implemented with Layer2 switches; Cisco 6509s and Juniper MX960s each present their own unique challenges.

  14. ESnet 4 Factoids as of January 21, 2008 • ESnet4 installation to date: • 32 new 10Gb/s backbone circuits • Over 3 times the number from last JT meeting • 20,284 10Gb/s backbone Route Miles • More than doubled from last JT meeting • 10 new hubs • Since last meeting • Seattle • Sunnyvale • Nashville • 7 new routers 4 new switches • Chicago MAN now connected to Level3 POP • 2 x 10GE to ANL • 2 x 10GE to FNAL • 3 x 10GE to Starlight

  15. ESnet Traffic Continues to Exceed 2 PetaBytes/Month Overall traffic tracks the very large science use of the network 2.7 PBytes in July 2007 1 PBytes in April 2006 ESnet traffic historically has increased 10x every 47 months

  16. When A Few Large Data Sources/Sinks Dominate Trafficit is Not Surprising that Overall Network Usage Follows thePatterns of the Very Large Users - This Trend Will Reverse in the Next Few Weeks as the Next Round of LHC Data Challenges Kicks Off

  17. ESnet Continues to be Highly Reliable; Even During the Transition “4 nines” (>99.95%) “5 nines” (>99.995%) “3 nines” (>99.5%) Dually connected sites Note: These availability measures are only for ESnet infrastructure, they do not include site-related problems. Some sites, e.g. PNNL and LANL, provide circuits from the site to an ESnet hub, and therefore the ESnet-site demarc is at the ESnet hub (there is no ESnet equipment at the site. In this case, circuit outages between the ESnet equipment and the site are considered site issues and are not included in the ESnet availability metric.

  18. OSCARS Overview On-demand Secure Circuits and Advance Reservation System OSCARS Guaranteed Bandwidth Virtual Circuit Services • Path Computation • Topology • Reachability • Contraints • Scheduling • AAA • Availability • Provisioning • Signalling • Security • Resiliency/Redundancy

  19. OSCARS Status Update • ESnet Centric Deployment • Prototype layer 3 (IP) guaranteed bandwidth virtual circuit service deployed in ESnet (1Q05) • Prototype layer 2 (Ethernet VLAN) virtual circuit service deployed in ESnet (3Q07) • Inter-Domain Collaborative Efforts • Terapaths (BNL) • Inter-domain interoperability for layer 3 virtual circuits demonstrated (3Q06) • Inter-domain interoperability for layer 2 virtual circuits demonstrated at SC07 (4Q07) • LambdaStation (FNAL) • Inter-domain interoperability for layer 2 virtual circuits demonstrated at SC07 (4Q07) • HOPI/DRAGON • Inter-domain exchange of control messages demonstrated (1Q07) • Integration of OSCARS and DRAGON has been successful (1Q07) • DICE • First draft of topology exchange schema has been formalized (in collaboration with NMWG) (2Q07), interoperability test demonstrated 3Q07 • Initial implementation of reservation and signaling messages demonstrated at SC07 (4Q07) • UVA • Integration of Token based authorization in OSCARS under testing • Nortel • Topology exchange demonstrated successfully 3Q07 • Inter-domain interoperability for layer 2 virtual circuits demonstrated at SC07 (4Q07)

  20. Network Measurement Update • ESnet • About 1/3 of the 10GE bandwidth test platforms & 1/2 of the latency test platforms for ESnet 4 have been deployed. • 10GE test systems are being used extensively for acceptance testing and debugging • Structured & ad-hoc external testing capabilities have not been enabled yet. • Clocking issues at a couple POPS are not resolved. • Work is progressing on revamping the ESnet statistics collection, management & publication systems • ESxSNMP & TSDB & PerfSONAR Measurement Archive (MA) • PerfSONAR TS & OSCARS Topology DB • NetInfo being restructured to be PerfSONAR based • LHC and PerfSONAR • PerfSONAR based network measurement solutions for the Tier 1/Tier 2 community are nearing completion. • A proposal from DANTE to deploy a perfSONAR based network measurement service across the LHCOPN at all Tier1 sites is being evaluated by the Tier 1 centers

  21. End

More Related