210 likes | 336 Views
NL-T1 Expectations , findings , and innovation. Bas Kreukniet, Sr Network Specialist at SURFsara. Geneva Workshop 10 Februari 2014. Outline. Expectations from NL-T1 grid administrators Findings while connecting to the LHCONE Innovation : Ethernet OAM and NSI.
E N D
NL-T1 Expectations, findings, andinnovation Bas Kreukniet, Sr Network Specialist at SURFsara Geneva Workshop 10 Februari 2014
Outline • Expectationsfrom NL-T1 gridadministrators • Findingswhileconnectingto the LHCONE • Innovation: Ethernet OAM and NSI
Expectations from NL-T1 administrators • Advicefromgrid administrators NL-T1: • Bulk data withsimpleapplications (grid-FTP). “Keep itsimple” • Network provisioningfromapplication is considered “complex” • Don’t rush tomerge LHCOPN withLHCONE
BGP routing starts at the connected site. • BGP routing for T1’s: • Not only a “NREN thing” – it already starts at your organisation! • Focus on some BGP topics for connecting sites to LHC networks
LHCONE connectivity LHCONE.inet.0: 133 destinations, 133 routes (133 active, 0 holddown, 0 hidden) + = Active Route, - = Last Active, * = Both 128.142.0.0/16 *[BGP/170] 1w2d 04:07:47, MED 120, localpref 100 AS path: 20965 20641 513 I > to 62.40.126.161 via xe-4/0/1.2012
Preferred route: LHCOPN, LHCONE or Internet • primary: direct T1–T1 over LHCOPN • secondary: T1–T1 over LHCOPN via another T1 • tertiary: LHCONE • quaternary: Internet
Preferred route: route (a)symmetry ISP-A 10 Gb/s 10 Gb/s site 1 site 2 1 Gb/s 100 Gb/s ISP-B • If everyone makes this choice, all connections are symmetric. • But: sites may have different preferences:
Tie-breaker between LHCOPN and LHCONE • CNAF routers at NL-T1 • LHCOPN • 131.154.128.0/17 *[BGP/170] 14:58:18, localpref 100 • AS path: 34878 137 I • LHCONE • 131.154.128.0/17 *[BGP/170] 1w2d 04:23:40, MED 120, localpref 100 • AS path: 20965 137 I
LHCOPN / LHCONE route preference • BNL routes at NL-T1 • LHCOPN • 130.199.185.0/24 *[BGP/170] 2w5d 09:35:43, MED 10, localpref 100 • AS path: 513 43 I • > to 192.16.166.73 via xe-1/1/0.0 • [BGP/170] 3d 01:15:22, MED 51, localpref 100 • AS path: 39590 513 43 I • > to 109.105.124.17 via xe-2/1/0.0 • LHCONE • 130.199.185.0/24 *[BGP/170] 1w2d 04:27:43, MED 120, localpref 100 • AS path: 20965 293 43 I • > to 62.40.126.161 via xe-4/0/1.2012
Problems and concerns regarding BGP • A site advertising his routes has no control who to send a route to. At best they can give hints with BGP communities. • The site receiving a route decides which route to accept and how to accept.
BGP in LHC networks • Only As is not enough. More info needed. community for origin of a route T1/T2? • Or even better: • -site connected to LHCOPN-site connected to LHCONE • - Site connected to both: • Type A “prefer LHCONE for this route” • Type B “prefer LHCOPN for this route” • - Specials: dedicated link between two (T1) sites. (“VPN”or “private link”)
Route Preference Solutions • Idea: Tag routes with two types of communities: • One for origin or source • One for destinations • See also: BGP hinting by Martin Sweeny (Indiana U) • BGP Always-compare-MED always on. We sometimes add metrics on incoming routes.
Operational Issues • We received routes over LHCfrom a site,but traffic we send is blackholed by that site • The site was still reachable over the Internet. • This happened to us twice recently: on LHCOPN (accidental route redistribution) and LHCONE (incoming IP filter). • Configuration errors will be made (we’re also just human) • LHCOPN Link NL-T1 – TRIUMF link is still unstable • 31 outages last 4 months • Monitoring remains important!
LHCONE and LHCOPN layer 2 monitoring • Ethernet OAM monitoring or Layer 2 monitoring • NIKHEF, TRIUMF toparticipate as Measurement Points • Lookingfor T1’s and T2 toparticipate • Advantages: • Layer2 keep-alive, ping andtraceroute • Interdomain, intervendor solution • L2 devicescanbe made visible • unidirectional fibercuts canbesignalled
LHCONE Innovation • NL-T1 likestopartcipate in NSI for LHCONE • NSI experiencessofar: • Use-case: Life ScienceGrid (LSG) in Holland makesuse of NSI sinceautumn 2013. • Freek contributedto standard • Sander wrote NSI clientandimplementeditfor “Cloud Bypassing” in the Life ScienceGrid.
Cloud Bypassing on Life Science Grid (LSG) Compute clusters at ±10 locations in the Netherlands
Cloud Bypassing on Life Science Grid (LSG) Offloading campus networks (some only have 1 Gb/s Internet)
LHCONE Innovation: Offloading is cheaper Internet full routing (incl backup): € 8k – €10k per month for 10 Gb/s LHCONE or dynamic lightpath: € 2k – 3k per month for 10 Gb/s
Questions? Erik Ruiter Bas Kreukniet DiederikVandevenne Sander Boele FarhadDavani Freek Dijkstra