National Backbone Extension: Rolling out Multisite Gigabit connections over Fibre Optics

1. National Backbone Extension: Rolling out Multisite Gigabit connections over Fibre Optics Brian Boyle & Mike Norris

2. Topics • NBE – in context • Aims of the project • What has been achieved • Under the bonnet • The bigger picture • Summary • Next steps • Discussion 2

3. NBE – in context • Building on the NBN • New requirements – hybrid networking • New paradigm of service and support • Proposal and resources • Support from stakeholders and clients 3

4. Bandwidth Use

5. HPC facility ICHEC traffic ICHEC latency

6. General Internet Daily traffic Yearly traffic

7. NBE – the project • Project plan and management • Inventory of sites • Criteria, priorities • Surveys and schedules • Procurement, multivendor scenario 7

8. Project organisation

9. Project goals • A network for connecting clients, with the following features • High bandwidth • Multi-service • Resilient • Scaleable • Flexible • Secure 9

10. Goals – Multi-service • Service orientated connectivity, e.g: • Private secure connection per Institute to An Chéim • Interconnectivity with the Government VPN • Not just focused on internet reachability • Capable of delivering access to multiple services using the same infrastructure • Advanced networking protocols supported – IPv6, multicast 10

11. Goals - Resilience • Seek to build in circuit, equipment, path, and routing diversity • Equivalent service levels on primary and secondary • Using multiple ducts, termination points, equipment • Using a robust routing technology (BGP) • Can make use of existing connectivity via other service providers 11

12. Goals - Scalability • Can deliver multiple 1 Gigabit/s circuits • Can operate at line speed end to end • Capable of offering even higher speeds subject to requirements (10 Gbps, higher) • 10 Gbit/s access for clients being explored • Technology refresh available for 40 Gbit/s… • …and similar for 100 Gbps in development 12

13. Goals - Flexibility • Not subject to telco business case scenarios • Not tied by contract to any particular network implementation technology or model • Can use a mix of fibre and traditional managed Ethernet circuits • Fully compatible with existing networks (no big bang changes) • Capable of delivering wavelengths to clients if the need is proven 13

14. Client perspective • Client priorities are key throughout • Aim is to keep change to client sites minimal • HEAnet installs managed routers and switches in addition to current network equipment • When ready, services can be tested and migrated individually • No Big Bang change • No change without explicit client agreement 14

15. Typical installation • Research of available providers (fibre, managed circuits), usually via Tender process • Client site survey by provider with client • Paths identified, leading to “way leave” contracts • Civil works commence • Fibres are pulled into building, and subsequently spliced • Routers and switches are delivered, configured by HEAnet • Services can be tested and used by client 15

16. New link services • Point-to-point Ethernet links (p2p) • 1 Gbit/s, Ethernet framed • Ethernet Port Service • Ethernet VLAN Service • No aggregation • Automatic monitoring/graphing 16

17. Ethernet Port and VLAN services

18. Some examples • Services- connectivity- institutional access and security- p2p • Applications- Oireachtas- ICHEC- An Chéim- UCLP • Client examples- DCU 18

19. LAN Extensions – Ryan Academy • Centralise services, both voice and data • Servers centrally located at DCU • IP Telephony Extension to Nortel Meridian • Seamless telephony integration • Security cameras and DVR remote view and monitoring • Buildings Management System • High-End Video Conferencing > 2 Meg/sec • Reliability • No onsite technical support ( low maintenance ) 19

20. LAN Extensions – Mater Dei Institute • Sister College of DCU • All Servers consolidated in DCU • Previous difficulties with server location in MD ( Dust and no Aircon ) • Minimal on-site technical support ( mainly helpdesk 2/3 days per week ) • No requirement for Sysadmin • Reliability of service 20

21. Under the bonnet • DWDM, CWDM • Ethernet switches • MPLS • New routing strategies • Out of band access for network management • CPE, routers • Outsourcing of L2 ops management • Provisioning tool • Contributions, SLA 21

22. Multiple sources • We make use of many fibre providers to connect at the lowest cost for our requirements • E-net • Dublin City Council • ESBT • BT Ireland • NTL • Aurora 22

23. Fibre network end 2006 – approx 2300km of fibre

24. MAN design

25. Transmission technologies • We use a heterogeneous mix of technologies to achieve our goals: • CWDM in metropolitan areas gives us multiplexed fibre services for short hops • DWDM between regions to give the same for longer hops • MPLS network to give multi-service offering • Multiple physical connections to add resiliency 25

26. HEAnet backbone network topology

27. How does it do it? • Circuits are managed using an automated provisioning system • Based on client requirements, the system is used to configure routers, service monitoring portal • Aim to have client use the tool directly, with administrator later checking/approving circuit change requests 27

28. Network provisioning tool

29. The bigger picture • NIBEST – NW link with NIRAN • UKERNA DF – SuperJANET 5 links • GÉANT – hybrid networking • Transit – preparing for the next step • INEX – peerings and more 29

30. INEX traffic growth • HEAnet-INEX (IPv4) • Total INEX

34. Summary of changes

35. Benefits • Improved bandwidth • Guaranteed uncontended bandwidth service levels • Reduced cost • Flexibility • Improved access to HEAnet services 35

36. Outline of services

37. Phase 2 progress, 2nd Nov Confirmed 7 Cancelled 4 In progress 17 Completed 13 Total number of links 43 37

38. Dark fibre statistics • No. of DF providers 5 • Aggregate length of DF pair 2,300 km • No. of primary client sites with DF links 18 (of 40) • Total sites with DF connections 31 38

39. Next steps • New model, service levels • Phase 3 • Client service reviews • Review of research infrastructures • Integrated e-infrastructures 39

40. Questions?

41. Project history • In 2004, we commenced implementation of dark fibre point to point links • Followed by additional of CWDM and DWDM equipment to increase service possibilities • Commenced work on an automated provisioning tool to create and manage configurations • Quickly grew into a heterogeneous network combining a mix of technologies and providers 41

42. Scalable capacity • Backbone – 10 Gbit/s • Can add more 10 Gbit/s using existing infrastructure • Client connections at 1 Gbit/s 42

43. What has been achieved • High levels of service to clients- high bandwidth- resilience – path, equipment, service • Scalability now and in the future • Underpins e-infrastructure • Complex and flexible • Rich mix of services • Secure 43