140 likes | 150 Views
R. Kevin Oberman ESnet Network Engineer Lawrence Berkeley National Laboratory. ESnet Update ESnet/Internet2 Joint Techs Albuquerque, New Mexico February 6, 2005. Outline. Physical Connectivity What’s new BAMAN in production LIMAN nearing production CHIMAN under construction
E N D
R. Kevin Oberman ESnet Network EngineerLawrence Berkeley National Laboratory ESnet UpdateESnet/Internet2 Joint Techs Albuquerque, New Mexico February 6, 2005
Outline • Physical Connectivity • What’s new • BAMAN in production • LIMAN nearing production • CHIMAN under construction • New Circuits • Traffic Driving: • Interconnectivity • Architecture
ESnet SF Bay AreaMAN Ring NLR to Seattle IP core to Chicago (Qwest) • 2 λs (2 X 10 Gb/s channels) in a ring configuration, and delivered as 10 GigEther circuits • - 10-50X current site bandwidth • Dual site connection (independent “east” and “west” connections) to each site • Used as a 10 Gb/s production IP ring and2 X 10 Gb/s paths (for circuit services) to each site • Project completed 9/2005, on time and on budget 10 Gb/soptical channels Joint Genome Institute LBNL λ1 production IP λ2 SDN/circuits NERSC λ3 future λ4 future SF Bay Area LLNL SNLL SLAC ESnet MAN ring (Qwest circuits) Qwest /ESnet hub Level 3hub DOE Ultra Science Net(research net) ESnet hubs and sites NLR to San Diego NASAAmes IP core to El Paso
Long Island MAN Initial Configuration T320 CERN Chicago(Qwest hub) Starlight Europe BrookhavenNational Lab, Upton, NY AbileneNYSERNetSINet (Japan) CANARIE (Canada) HEANet (Ireland)Qatar GEANT 10GE circuit ESnet boarder demarcation device MAN LAN USLHCnet LI MAN DWDM Cisco 6509 ESnetIP core 1 GE or 10 GE 32 AoA, NYC BNL IP gateway ESnet IP core(Qwest) 10 Gb/s circuits International LHCnet circuits (proposed)production IP core SDN/provisioned virtual circuits Washington (Qwest hub)
Chicago MAN T320 T320 USLHCnet 4x10G Starlight 455 N. City Front 3x10G ESnet IP SDN Fermilab Argonne
New 10G Circuits ESnet is nearing completion of a process to turn up 44 new 10G circuits in less than 12 months including: MANs 16 circuits composing the San Francisco Bay Area MAN 2 circuits comprising the Long Island MAN (New York to Brookhaven) 15 circuits comprising the Chicago MAN (FNAL, ANL and StarLight) NLR 3 circuits: San Diego to Sunnyvale, Sunnyvale to Seattle, hub to hub in Sunnyvale Sites 1 circuit JLAB (via, MAX/NGIX-E, MATP, E-LITE) 1 circuit between NYC and DC to reach GEANT 1 circuit ORNL – Starlight - CHI-HUB(NBC) Peering 1 circuit GEANT 1 circuit SNV – NASA-AMES
Other Circuits Completed: OC48 ALB-ELP DS3 IARC-SNV (New Site) OC3 ALB-KCP In the works: OC3 ALB-HUB-PANTEX <-- In build-out
The Office of Science U.S. Community* Institutions supported by SC Major User Facilities DOE Specific-Mission Laboratories DOE Program-Dedicated Laboratories DOE Multiprogram Laboratories Pacific Northwest National Laboratory Idaho National Laboratory Ames Laboratory Argonne National Laboratory Brookhaven National Laboratory Fermi National Accelerator Laboratory Lawrence Berkeley National Laboratory Stanford Linear Accelerator Center Princeton Plasma Physics Laboratory Lawrence Livermore & Sandia National Laboratories Thomas Jefferson National Accelerator Facility General Atomics Oak Ridge National Laboratory Los Alamos National Laboratory Sandia National Laboratory National Renewable Energy Laboratory SC Program sites
ESnet’s Physical Connectivity (Spring 2006)* SINet (Japan) Russia (BINP) GÉANT - France, Germany, Italy, UK, etc PNNL SEA SLAC NERSC BNL ANL MIT INEEL LIGO LBNL LLNL MAN LANAbilene SNLL CHI-SL LVK JGI Starlight OSC GTNNNSA LBNL DC LLNL/LANL DC Offices ORAU DC Chi NAP PPPL AMES FNAL JLAB ORNL SRS SNLA LANL DC USN USN DOE-ALB KCP-ALB NASAAmes PANTEX ORAU NOAA OSTI ARM IARC YUCCA MT BECHTEL-NV GA Abilene Abilene Abilene Abilene MAXGPoP KCP SoXGPoP NREL SNV Japan (SINet) Australia (AARNet) Canada (CA*net4 Taiwan (TANet2) Singaren CA*net4 France GLORIAD (Russia, China)Korea (Kreonet2 MREN Netherlands StarTapTaiwan (TANet2) UltraLight PNWGPoP/PAcificWave CERN (USLHCnetCERN+DOE funded) NLR Supplied Circuits AU ESnet IP core MAN rings NYC MAE-E SNV CHI Equinix PAIX-PA Equinix, etc. SNV SDN ATL SDSC AU ALB AMPATH UNM 42 end user sites AMPATH (S. America) ELP Office Of Science Sponsored (22) NNSA Sponsored (12) International (high speed) Lab Supplied 10 Gb/s SDN core 10G/s IP core 2.5 Gb/s IP core MAN rings (≥ 10 G/s) OC12 / GigEthernet OC3 (155 Mb/s) 45 Mb/s and less Joint Sponsored (4) Other Sponsored (NSF LIGO, NOAA) ESnet IP core: Packet over SONET Optical Ring and Hubs Laboratory Sponsored (6) commercial and R&E peering points ESnet core hubs IP high-speed peering points with Internet2/Abilene
Network Utilization ESnet Accepted Traffic (Bytes)Jan, 1990 to Dec., 2005 ESnet is Currently Transporting About 750 Terabytes/mo.and this volume is increasing exponentially TBytes/Month
Network Utilization ESnet traffic has increased by about 10X every 46 months, on average, since 1990 Dec., 2001 42 months Jul., 1998 TBytes/Month 57 months Oct., 1993 Aug., 1990 39 months
OSCARS: Guaranteed Bandwidth VC Service For SC Science • ESnet On-demand Secured Circuits and Advanced Reservation System (OSCARS) • To ensure compatibility, the design and implementation is done in collaboration with the other major science R&E networks and end sites • Internet2: Bandwidth Reservation for User Work (BRUW) • Development of common code base • GEANT: Bandwidth on Demand (GN2-JRA3), Performance and Allocated Capacity for End-users (SA3-PACE) and Advance Multi-domain Provisioning System (AMPS) • Extends to NRENs • BNL: TeraPaths - A QoS Enabled Collaborative Data Sharing Infrastructure for Peta-scale Computing Research • GA: Network Quality of Service for Magnetic Fusion Research • SLAC: Internet End-to-end Performance Monitoring (IEPM) • USN: Experimental Ultra-Scale Network Testbed for Large-Scale Science • In its current phase this effort is being funded as a research project by the Office of Science, Mathematical, Information, and Computational Sciences (MICS) Network R&D Program • A prototype service has been deployed as a proof of concept • To date more then 20 accounts have been created for beta users, collaborators, and developers • More then 100 reservation requests have been processed
Observation • In 4 years, we can expect a 10x increase in traffic over current levels without the addition of production LHC traffic • Nominal average load on busiest backbone links is ~1.5Gbps today • In 4 years that figure will be ~15Gbps if current trends continue • Measurements of this kind are science-agnostic • It doesn’t matter who the users are, the traffic load on the backbone is increasing • Bandwidth trends drive requirement for a new network architecture
ESnet Target Architecture: IP Core+Science Data Network Core+Metro Area Rings Europe (GEANT) CERN Canada (CANARIE) Canada (CANARIE) Asia-Pacific CERN Europe (GEANT) Seattle SDN Core Chicago Australia New York Denver IP Core Sunnyvale Washington, DC MetropolitanArea Rings Atlanta Loop off Backbone LA Albuquerque Aus. South America (AMPATH) San Diego IP core hubs 10-50 Gb/s circuits Production IP core Science Data Network core Metropolitan Area Networks International connections South America (AMPATH) SDN hubs Primary DOE Labs possible hubs