700 likes | 893 Views
Janet Update GridPP Collaboration meeting Sept 2012. David Salmon. Topics. Janet6 Janet & e -Infrastructure Issues & futures. Janet6 Neil Shewry Rob Evans. SuperJANET4/5. Getting us out of the cycle of networks that were bursting at the seams at the end of their life
E N D
Janet UpdateGridPP Collaboration meetingSept 2012 David Salmon
Topics • Janet6 • Janet & e-Infrastructure • Issues & futures....
SuperJANET4/5 • Getting us out of the cycle of networks that were bursting at the seams at the end of their life • Dedicated fibre backbone, allowed us to grow. • Subject to contract • SJ5 was originally due to finish October 2011 • Extended to October 2013 • Still no congestion • Originally 10Gbit/s, but have upgraded through 40Gbit/s to 100Gbit/s
SuperJANET5 • Greater visibility of the optical layer than previous networks • Managed by Verizon Business • Janet NOC had read-only access to (most) transmission equipment • Regular meetings with Verizon on implications of capacity requests • Good communications between Janet’s senior engineers and Verizon’s. • 40G and 100G trials before deployment • However, still some distance away from the ‘laser-face’ • Deployment issues that were contractual rather than technical or operational
Janet6: Building up ideas • Thinking & reflection as SJ5 progressed • Greater visibility of optical layer • New features of transmission systems • ROADMs: Colourless; Directionless; Contentionless (more later) • Direct experience of transmission equipment • TVN • Developments in Janet and other R&E networks • Use of point-to-point circuits • Dynamic provisioning
Requirements Gathering Off-net • Anytime, anywhere access • Internationalisation of education Management of costs • Funding environment • Costs of change Bandwidth, bandwidth, bandwidth... Flexibility Agility Cost control Service delivery Delivery of third party services Cloud services Reliability and resilience Partnerships Public/public and public/private Information assurance
Requirements Gathering High reliability by minimising single points of failure; flexibly coping with major breaks; assisting in resilience to customer organisations Reliability Ability to increase bandwidth at controllable cost Scalability With agility End-to end Across JANET Protection of interests of teaching and learning and research sectors Separability Responsiveness to additional network service requirements Flexibility Support for the provision of wide range of services: e.g. High information assurance; Unified comms Functionality
Procurement Strategy Options appraisal Bespoke arrangement • Procure dark fibre infrastructure • Procure optical transmission equipment • Management in-house by the Janet NOC competitive dialogue procedure
Procurement Strategy Fibre Procurement Lot 1: Fibre Infrastructure in the UK Options: Scotland, Wales, Ireland, Aurora, London, Research Lot 2: Fibre Infrastructure in Ireland Framework Agreement(s) for 4 years • Initial order from 5+2 years to 10+5 years
Procurement Strategy Transmission equipment procurement • Supply, design, delivery, installation, commissioning, maintenance, training, specialist technical support • Framework Agreement for 7 years • Initial order 3 + 2 + 2 years
Fibre Procurement Prequalification phase Launched October 2011 13 responses Shortlist 6 bidders
Fibre Procurement UK Government approval to proceed – Cabinet Office July 2012 Contract Signed July 2012
Transmission Equipment Procurement Prequalification phase Launched November 2011 17 responses Shortlist 6 bidders
Transmission Equipment Procurement Contract signed last week September 2012
From requirements to services • Capacity • Exponential growth continues • ‘Commodity’ IP networking • R&E requirements • Must be able to scale the network • Not just more channels, but faster channels over the network’s life • 100Gbit/s to start • 400Gbit/s? • 1Tbit/s?
From requirements to services • Ubiquitous connectivity • Access to Janet from anywhere • Not all in the scope of Janet6 • Other activities in the company • Feeds the capacity requirements • Feeds the external connection requirements • Feeds the reliability requirements
From requirements to services • Distributed campuses • Inter-site, intra-organisation connections • VPNs • Separacy • Centralised firewall
From requirements to services • Outsourced services • Compute, data storage, applications • Low latency for interactive services • High bandwidth for bulk data services • Reliability, reliability, reliability.
From requirements to services • Research • National and global collaboration • Interoperation with GEANT and other R&E networks worldwide • LHC, SKA, ITER, Bioinformatics, Climate… • ‘Data deluge’ • Services • Dynamic provisioning of layer 2 circuits
Janet6: Core Fibre and Backbone Resilience • Similar topology to SJ5. • Two north-south paths • New east-west path from Birmingham to Nottingham • Erdington to Lowdham • No backbone PoP at Reading • Still have a regional PoP • STFC RAL as a backbone PoP • New PoP in Acton, West London
Janet6: Regional fibre • Illustrative only • The details are subject to change • Similar to SJ5 • ‘Collector arcs’ • Spurs • Regional networks connect to two different routers • Over 70 optical nodes • Amplifier sites, add/drop sites, and router sites
Lighting the fibre • DWDM transmission equipment • Still some way to go in the procurement • Theoretically 80 channels of 100Gbit/s per channel • Roadmap for 400Gbit/s and 1Tbit/s with better spectral density • ROADMs • ‘Native’ capacity on the core to be 100Gbit/s • 10Gbit/s multiplexed over that where needed • Muxponders or OTN switching • Coherent technology required for 100Gbit/s doesn’t sit well with dispersion compensation required for 10Gbit/s • RNEP fibre may still carry 10G native
10G Circuits: muxponders or switched? Muxponders Switching
ROADMs • Will be more reconfigurable than SJ5 • One of the collector arcs in Scotland required 13 engineers to be in dotted along it at the same time to commission • Get away from the ‘air gap’ between RNEP fibre and backbone fibre that exists on SJ5. • Different transmission systems on collector arcs and backbone • Requires two client cards and a patch cable to bridge a circuit from one to the other
Building the network and the NOC • What will this give us? • Janet NOC will have a view from the fibre up the stack to the routers • We will see when fibres/lasers/receivers degrade • Lower signal levels • Increase in error rate • Forward Error Correction (FEC) working harder • We will see where faults are • Fewer administrative boundaries to cross • Directly translate the community’s requirements into engineering
IP service • As you were • Finish moving away from SDH access towards ethernet • 802.1ad bonding for 20Gbit/s, 30Gbit/s etc. • 100GE if you can afford the optics (have seen a dramatic fall) • Still IPv4, IPv6, unicast, multicast • In London, retaining Telehouse and Telecity Harbour Exchange as major points of external connection • Continuing to expand Telecity Manchester • Content providers and content delivery networks • ‘IX Manchester’ operated by LINX
VPNs • VPNs over primary access • Inter-site connectivity • Low bandwidth layer 2 circuits • Other services? • Statically configured VLANs (site/RN boundary, perhaps RN/core) • BGP signalled VPNs (RN/core, perhaps some sites) • Bandwidth management between IP and VPNs • These are layer 2/3 VPNs • Not IPsec • If you want to be sure it is encrypted, best to do that yourself • Would you trust this man with your encryption keys?
Lightpath service • Good question • Ethernet over MPLS network will remain • Upgrade the MX960s to 100GE interconnects • Move 10Gbit/s circuits from dedicated wavelengths to EoMPLS • Investigating OTN switching as part of transmission procurement • May be too early to do much • …especially as we’ll also have MX960 platform • Don’t need guaranteed bandwidth? • See VPN service
Migration • 66.666666666666667% (ish) chance we’ll be moving core PoPs • Moving routers from PoP to PoP overnight would be ‘challenging’ • Requires all regional networks connected to a PoP to move simultaneously (no link between old PoP & new PoP) • Can’t move anything until all the regional network fibres to a PoP are ready • One delay holds up everything • Remember the newts! • Need to upgrade the switching fabric anyway • T640 to T1600 • T1600 to T4000 • Investigating options & costs
Migration • Aiming to have the network installed and ready by the end of March 2013 • Migration between March and August • Post-Clearing until end of October is contingency only • End of October is a hard cut-off.
Janet6 summary • Extensive fibre footprint • Direct management of optical transmission • On-schedule • Very capable strategic foundation – 10 years+
Topics • e-Infrastructure funding • What has Janet been doing ? • Emerging Issues
Janet – e-Infrastructure networking • £26M With HEFCE Breakdown • £10M Contribution to Janet6 backbone • £12M Network provision within “classic” Janet community, including key locations which may need additional fibre access • £4M “Industry connection”
e-Infrastructure programme support • CTO Bob Day on ELC • Paul Lewis secondment to BIS programmeOffice • 8 months • STFC Michael Wilson secondment • just started • funded from Janet’s e-Infrastructure allocation
e-Infrastructure Strategic Sites • Potential fibre / capacity requirements • Work with organisations to understand requirements • Build cases for provision • If approved, integrate provision with deployment of Janet6
Patterns of use • Data movement – working patterns • How much – data volume ? • Beware Bytes vs. bits ! • How quickly ? • How often ? • Where to ? • National & International • Network engineers need bit-rates ! • Working with research communities to understand emerging/evolving requirements
Norwich Bioinformatics Cluster • The Genome Analysis Centre - TGAC • The Sainsbury Laboratory • John Innes Centre • Institute for Food Research • University of East Anglia • Norwich Research Park
Hinxton Genome Campus • European Molecular Biology Laboratory - European Bioinformatics Institute • The Sanger Centre • Babraham Institute
Exeter & locale • Weather & Climate research • Monsoon HPC – NERC & M.O. • Met Office • University of Exeter
Broaden scope of Janet’s fibre infrastructure • Complementary & integrated with Janet6
e-Infrastructure projects funded • HPC • 5 regional consortia • National facility – Hector / Archer • Dirac • Particle physics • NERC • STFC • RAL • Daresbury • will engage to discuss requirements & evolution