1 / 22

US LHCNet Update

US LHCNet Update. Dan Nae California Institute of Technology dan.nae@cern.ch. LHC OPN Meeting Munich, April 2007. What is the US LHCNet. A transatlantic network designed to support the LHC and the U.S. HEP community

orpah
Download Presentation

US LHCNet Update

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. US LHCNet Update Dan Nae California Institute of Technology dan.nae@cern.ch LHC OPN MeetingMunich, April 2007

  2. What is the US LHCNet • A transatlantic network designed to support the LHC and the U.S. HEP community • Funded by the US DoE and CERN and managed by Caltech in collaboration with CERN • Evolved from a network between US and CERN which dates back to 1985 • Our mission is to deliver a reliable network service to support the upcoming LHC experiments at CERN • Designed to support the LHC three-tiered model and to deliver data directly to/from the US Tier 1’s (FNAL and BNL) LHC OPN Meeting, Munich

  3. Circuit Status NYC 111 8th Bellport NYC-MANLAN Bude Whitesands AC-2 AMS-SARA VSNL North VSNL Frankfurt Highbridge GVA-CERN NY60 Hudson Wal, NJ CHI-Starlight VSNL South Paris Global Crossing Qwest Pottington (UK) Colt London Atlantic Ocean GEANT • Unprotected circuits (lower cost) • Service availability from provider’s offers: • Colt Target Service Availability is 99.5% • Global Crossing guarantees Wave Availability at 98% • Canarie and GEANT: No Service Level Agreement (SLA) LCG Availability requirement: 99.95% LHC OPN Meeting, Munich

  4. Circuit Status (cont.) • 6 x 10 Gbps SONET/10GbE WAN-PHY. • Amsterdam PoP installed in January. • Last circuit (Geneva-Amsterdam) delivered at the end of last month. • Not as diverse as one might expect. • We’ve never had an isolated node so far. • It happened more than once that the circuits went down in pairs (GC NYC-CHI and NYC-AMS, Qwest NYC-CHI and CHI-GVA). • Rumors of a new RFP in May-June? LHC OPN Meeting, Munich

  5. Circuit Availability (March) Proposal to produce a monthly report LHC OPN Meeting, Munich

  6. “Old” Network Map (March ’07) LHC OPN Meeting, Munich

  7. Equipment Status • Next generation network based on Ciena CoreDirector/CI – VCAT/LCAS/GFP capabilities. • Four nodes, one at each PoP to complement the existing Force10 boxes. • Number of links on each box expected to increase each year; the upgrade plan has been discussed with Ciena up to ~2010. • Deployment has already started (the New York and Chicago nodes were installed, tested and commissioned by the end of last month). • The Geneva node is physically installed, will be tested and most likely accepted next week. • The fourth node (Amsterdam) expected to be operational in May. LHC OPN Meeting, Munich

  8. Past and Present Issues • At least one Stratum 1 PRS is recommended for normal operation (still to be installed) • Second DC power feed in Chicago is still to be installed (today in non-redundant mode) • One ESLM card (10GbE) locked up during the migration of the the NYC-CHI GC circuit (fixed by hardware reset) • DC rectifier at CERN not fully populated (but enough for the moment) • CERN commissioning has been delayed due to complicated site access procedures • We need more operational experience with the boxes • Currently only one circuit (GC NYC-CHI) runs over the new equipment (migrated on March 29th) LHC OPN Meeting, Munich

  9. Equipment Status Today • Three nodes delivered (GVA, CHI, NYC) • Two nodes installed (NYC, CHI) • One circuit migrated (GC NYC-CHI) LHC OPN Meeting, Munich

  10. Proposed Migration (May) • Two complementing networks: • first network is essentially last year’s US LHCNet – stable, reliable, dedicated 10Gbps T0-T1 (and T1-T1) • the second is based on the new equipment – dynamic or static point-to-point VCs between any of the four nodes • the “new network” can be reconfigured to carry T0-T1 traffic in case of problems • Equipment diversity (more robust) LHC OPN Meeting, Munich

  11. GEANT-ESNET Peering over US LHCNet • The idea came up last year during the meeting at FNAL • US LHCNet would transparently carry a L1/L2 Virtual Circuit between NYC and Amsterdam • GEANT and ESNET peer over this VC • The bandwidth of the circuit is still to be discussed • An MOU between ESNET and US LHCNet was signed (with DOE’s blessing) • ESNET will install a new 10 GbE connection to the US LHCNet CD/CI in NYC for this purpose • A new 10 GbE connection in Amsterdam between US LHCNet and GEANT is required LHC OPN Meeting, Munich

  12. Proposed Migration (cont.) • Additional links are needed to make full use of the new infrastructure: • New BNL/ESNET link in NYC • New FNAL/ESNET link in Chicago (or existing to be moved to the CD/CI) • New LCG – CD/CI link(s) in Geneva • New GEANT & SARA links in Amsterdam (T1-T1 and peerings) LHC OPN Meeting, Munich

  13. Network Forecast - 2007 4 x 10GBE 2 x 10GBE Geneva 1 x 10G SDH Amsterdam 1 x 10G SDH 1 x 10G SDH 1 x 10G SDH Chicago 2 x 10G SDH New York 1 x 10G SDH 1 x 10G SDH 4 x 10GBE 4 x 10GBE ESnet SDN Mostly sunny, scattered clouds LHC OPN Meeting, Munich

  14. Network Forecast - 2008 6 x 10GBE 2 x 10GBE Geneva 2 x 10G SDH Amsterdam 1 x 10G SDH 2 x 10G SDH 1 x 10G SDH Chicago 2 x 10G SDH New York 1 x 10G SDH 1 x 10G SDH 4 x 10GBE 4 x 10GBE 1 x 10G SDH ESnet SDN 1 x 10G SDH Clouds gathering… I2 NewNet LHC OPN Meeting, Munich

  15. Network Forecast - 2009 8 x 10GBE 2 x 10GBE Geneva 3 x 10G SDH Amsterdam 2 x 10G SDH 2 x 10G SDH 2 x 10G SDH Chicago New York 3 x 10G SDH 2 x 10G SDH 2 x 10G SDH 1 x 10G SDH 1 x 10G SDH 1 x 10G SDH 1 x 10G SDH 4 x 10GBE 1 x 10G SDH 4 x 10GBE ESnet SDN 1 x 10G SDH Hybrid Network Overcast ;-) I2 NewNet LHC OPN Meeting, Munich

  16. Network Forecast - 2010 10 x 10GBE 2 x 10GBE 4 x 10G SDH Geneva Amsterdam 3 x 10G SDH 2 x 10G SDH 3 x 10G SDH New York Chicago 4 x 10G SDH 3 x 10G SDH 3 x 10G SDH 1 x 10G SDH 1x 10G SDH 1x 10G SDH 4 x 10GBE 1x 10G SDH 1x 10G SDH 4 x 10GBE 1x 10G SDH ESnet SDN Severe chances of high-speed data transfers Hybrid Network I2 NewNet LHC OPN Meeting, Munich

  17. Ciena CD/CI • VCAT/LCAS/GFP support – allows for provisioning of dynamic circuits • Difficult to manage with current network tools (no SNMP for example) • Integrated control plane functionality (still to be explored) • Need to be integrated with our provisioning/monitoring tools (MonALISA) • Also selected by Internet2 for their NewNet installation (dynamic provisioning with DRAGON software) • Inter-domain integration for dynamic circuits is desirable in the future LHC OPN Meeting, Munich

  18. Ciena CD/CI (cont.) • Bandwidth granularity of STS-1 (51Mbps) • Bandwidth adjustments without interrupting the circuit (very important for long-lived transatlantic data transfers) • OSRP + Mesh restoration allows for easy provisioning and re-routing of circuits over alternate paths • Fast circuit restoration (Ciena claims 50ms) • Associated circuit priorities (lower priority circuits can be preempted during restoration) • Need to be integrated with perfSONAR monitoring (help needed) LHC OPN Meeting, Munich

  19. Other Technical Issues • The CD/CI management/monitoring is done only via TL1 or GUI (NodeManager) • Circuit status is not reflected in the terminating 10 GbE interface status (workaround – UDLD, FAFD) • We need to understand how to interact with the CD/CI control plane • We need an API to work with for automated operations; apparently there is a CORBA interface but we don’t have any specifications for it yet. LHC OPN Meeting, Munich

  20. US LHCNet Network Monitoring • Various monitoring tools are deployed • Spectrum – used by the CERN operators (first line of support) • MonALISA – used by the US LHCNet team to get a global view of all the services • Various open-source toos (Nagios, Cricket, rancid, RRD) - used by the US LHCNet NOC for configuration tracking, logs and alarms • perfSONAR – used by the E2ECU Current perfSONAR installation extracts data from the Spectrum monitoring service at CERN; we have plans to migrate this to a stand-alone setup (at least for the CD/CIs) LHC OPN Meeting, Munich

  21. US LHCNet Team • Harvey Newman – Project PI • Dan Nae – Network Engineer – tech team lead • Ramiro Voicu – Software Engineer • Two people left – Sylvain Ravot (October, 2006) and Yang Xia (March, 2007) • Artur Barczyk - new Network Engineer, started on March 1st, 2007, based at CERN • Tony Cheng – new Network Engineer, started April 15th, based in Pasadena, CA LHC OPN Meeting, Munich

  22. Questions? • Thank You! LHC OPN Meeting, Munich

More Related