EARNEST: The Future of Routing & Networking Technology HEAnet National Networking Conference 2007 15 November 2007,

1. EARNEST: The Future of Routing & Networking TechnologyHEAnet National Networking Conference 200715 November 2007, Kilkenny, Ireland

2. EARNEST Background • EC-funded GN2 project (35 NRENs including HEAnet ): • Build and operate GÉANT2, the pan-European research and education backbone. • Joint Research Activities (JRA 1-5) to investigate and develop network enhancements. • Service Activities (SA 1-6) to procure network and connect NRENs. • Networking Activities (NA 1-7) to provide user support, dissemination, addressing digital divide, coordination activities, conferences and workshops, and undertake foresight study. • GN2-NA4 = Education And Research Networking Evolution STudy (EARNEST)

3. EARNEST Background • Aims to identify trends, developments, and to make recommendations for future research and education networks. • Seven sub-studies: • Organisational and Governance issues • Economic issues (move to dark fibre, and provision of new services) • Researchers’ needs (what type of network and services are required?) • Other users’ needs (e.g. schools, healthcare, arts & humanities) • Geographic issues (examining and quantifying digital divide) • Campus issues (infrastructure, services, expertise and collaboration) • Technical issues (transmission, control plane & routing, network virtualisation, operations and performance, middleware) • All sub-study reports plus final conclusions available shortly. • http://www.terena.org/activities/earnest/

4. Technical Study Areas • Transmission Technologies • Fibre provisioning, transmission protocols, equipment evolution. • Control Plane & Routing Technologies • Switching & routing developments (optical & IP), IPvX, multicasting. • Operations and Performance • QoS vs overprovisioning, end-to-end performance, network management (optical & IP), network monitoring, PERT. • Middleware • AAIs, identity management, federations, mobility. • Network Virtualisation • Customer-enabled networks utilising virtual routers, lightpaths, VPNs, VLLs controlled by UCLP, DRAC etc..

5. Methodology & Caveats • Technical panel with expertise in specific areas advised on important or emerging technologies. • Interviews with key personnel from 11 vendors, 3 research institutes, and a number of NRENs. • Technological briefings and research papers also used. • Primary goal was to investigate technologies applicable to NRENs, although attempts to address other types of network as well. • R&E networks often have different requirements to telco and ISP sectors, and usually have fewer legacy issues.

6. Transmission Technology Findings

7. Ethernet or SDH? • No obvious path for SDH beyond OC-768 (40 Gbps), and likely to become legacy technology in coming years. • All manufacturers developing 40 and/or 100 Gigabit Ethernet because of cost advantages, and because packet-based services are increasingly prevalent. • Vendors don’t wish to repeat experience of having to support different variants of 10 GE (i.e. LAN-PHY, WAN-PHY). • Was initially expected that 100 GE would be next standard, but this is proving to be technically difficult.

8. Ethernet or SDH? • 100 GE implementations not expected before 2010, and likely later. • Initially likely to be 4 x 25 Gbps and restricted to short-haul applications. • Full serial implementations not expected until at least 2012. • 40 GE may be interim solution as implementations possible by 2009 • Expected to be 40% the cost of OC-768. • Supposedly intended for data centre applications, but some vendors talking about WAN capabilities (80 km before amplification/2000 km before regeneration).

9. Ethernet Enhancements • Ethernet scalability initially addressed with IEEE 802.1Q and 802.1ad. • PBB (IEEE 802.1ah) aims to greatly increase number of customer networks, and defines protocols for connecting provider-bridged networks. • Carrier-grade OAM&P and virtual circuit functionality is also currently being added: • PBBTE (802.1Qay) will support point-to-point circuits over Ethernet. • CFM (802.1ag) will support hop-by-hop detection, isolation of connectivity problems • Shortest-Path Bridging (IEEE 802.1aq) being developed as alternative to Spanning Tree for loop-free forwarding. http://www.terena.org/activities/ngn-ws/ws1/docs/061107-jacobs-TERENA-NGN-WS-01.pdf

10. DWDM Systems • Trade-off between number of wavelengths, faster line rates and longer reaches due to CD, PMD, XPM and FWM. • New modulation techniques (e.g. DP-QPSK) are becoming practical and promise longer reaches at 40 Gbps+ speeds, whilst minimising need for EDCM. • Most manufacturers focusing on 50 GHz spacing for DWDM channels (i.e. ~80 channels per fibre). This has been found to provide optimal performance with respect to faster line rates and longer reaches. • Tunable lasers, VOAs, EDCMs, multi-degree ROADM technology, and PIC-based OEOs promise easier-to-facilitate (and potentially cheaper) DWDM systems. Also make meshed optical networks possible. • Passive Optical Networks (PONs) being trialled.

11. DWDM Systems • Questions to ponder: • There was a lot of hype about DWDM five years ago, but actually how important is this to NRENs? • Dark fibre is increasingly available to NRENs, but few fully exploit DWDM possibilities. • Why is the take-up of DWDM by NRENs so slow? • Is being ‘faster’ or ‘fatter’ more important to NRENs?

12. Control Plane & Routing Findings

13. IP Routing • Routing scalability becoming problematic (again). • Huge rise in number of hosts, fragmentation of service provider hierarchy, increase in multihoming, and amount of traffic. • Global routing table now >230,000 entries, which generates around 400,000 BGP updates per day. • Concern that growth is starting to outstrip router chipset and memory developments, but more specifically the cost of provisioning these. • IPv6 doesn’t help as end-users unwilling to use provider-assigned addresses, or renumber when changing service providers.

14. IP Routing • Not immediate cause for concern, but IAB/IETF looking for efficiencies. • Multihoming and traffic engineering should be possible. • Addresses should be provider-independent • Work with IPv6, and ideally IPv4. • Proposals based on splitting IP addresses into identifiers and locators. • End hosts would have unique identifier regardless of location (EID) • Locator used for intermediate routing, but is dynamically allocated in accordance with network location (RLOC) • Locator would be provider dependent and allow for better aggregation.

15. IP Routing • How to do it? • NAT sort of achieves the same thing, but uses private IP addresses which introduce other problems. • e-FIT would use EIDs within user networks, and encapsulate packets using RLOCs in transit networks. Routing separated between networks. • LISP also uses encapsulation, but this happens at edge routers. Is transparent to BGP. • Six/One based on 8+8/GSE and shim6. Lowest 64-bits are unique identifer, but top 64-bits written by edge router (although hosts can suggest route). • EID-to-RLOC mapping: APT (all nodes hold limited database with default routes), NERD (all nodes hold complete database), and LISP-CONS (distributed query) • What part of the network should have global overview? • How to determine default routes? • How dynamic can/should mapping be? • Security?

16. IPv6 • Core IPv6 specifications and related protocols largely completed some years ago. • Most NRENs already support IPv6 in dual-stack systems, but also tend to have more IPv4 address space. • Some router and user equipment still has limited support. • Still limited support in most campuses. • New predictions suggest IPv4 address space could be exhausted in 3-5 years. • Regional Internet Registries discussing rationing measures.

17. IP Multicasting • Never really taken off in past 20 years, but IPTV in context of ‘triple play’ increasing interest amongst service providers. • Increased availability to end-users may make it easier to deploy across Internet. • Inter-domain multicast routing still complex, although SSM may improve situation. • Automatic Multicast Tunnelling (ATM) allows hosts to find convenient multicast relay if native multicast not available. • Many peer-to-peer applications already multicast at application layer.

18. Dynamic-control of hybrid networks • Lightpaths are still largely manually configured. • Optical and IP domains still managed separately. • GMPLS offers possibility of integrating IP routing and WDM control planes (amongst other things). • In development for long time, but only just starting to be deployed using vendor-specific solutions. • Still signalling and interoperability issues to resolve, especially between domains. • Peer or overlay model? • Probably necessary for fully exploiting hybrid networks, but introduces more complexity.

19. Network Virtualisation Findings

20. Network Virtualisation • Virtualisation concepts starting to be used across all networking layers. • Basic virtualisation already implemented in certain modern routers to enable upgrades and troubleshooting of specific interfaces, and programmable features. • NRENs (e.g. CANARIE, CESNET) pioneered customer-empowered network concept, where resources on NREN-provisioned infrastructure can be managed by customers to build logical networks. • Deployment of UCLP, DRAC and similar technologies are first step towards full network virtualisation. • Need for technology agnostic infrastructure, although most users still want IP connectivity as part of service.

21. Network Virtualisation • MANTICORE and FEDERICA projects aim to develop network virtualisation to allow disruptive technologies to be tested over production infrastructure. • US-based GENI initiative extends concept to wireless and sensor networks as well. • EARNEST study revealed there was little knowledge in wider R&E community about virtualisation initiatives, but lot of potential interest. • TERENA NGN Workshop (06/11/07) had session on network virtualisation/customer-empowered networks. • Generated much discussion. • Support for information exchange and coordination activity (e.g. task force). • Need a better term to describe all this though!

22. Operations & Performance Findings

23. Layer 0-2 Management • NRENs have traditionally only managed Layer 3 and above, so have limited experience at the optical level (WDM systems and/or SDH). • Limited tools for managing Network Layers 0-2, and expensive. • Although some R&E developments such as TL1 Toolkit and NDL. • Management of Layers 0-2 is currently labour intensive and relies heavily on documentation. • NRENs have not really made extensive use of WDM systems to-date, and the management of much so-called dark fibre is often outsourced. • Is this something to investigate further?

24. Overprovisioning vs QoS • Core networks likely to continue to be overprovisioned as bandwidth is (relatively) cheap. • Some edge networks do need to undertake traffic engineering though, so QoS transparency should be supported. • Increasing availability of dark fibre allows R&E networks to operate hybrid networks, enabling dedicated links to be provisioned for demanding customers using C/DWDM. • Should encourage innovation through network neutrality, subject to traffic engineering requirements.

25. End-to-End Connectivity • Most end-to-end performance issues are due to problems at customer sites. • Middleboxes such firewalls, NATs, rate shapers, caches and other ‘black box’ solutions are responsible for many of these problems. • This is due to instrinic architecture, misconfigurations, or simply intentional behaviour. • They encourage workarounds that circumvent what the box is trying to achieve in the first place. • Consider improving network transparency, either through protocol support, or moving functionality closer to end-hosts. • Filtering and firewalling should also be weighed against reduction in innovation capabilities within research environment. • Buggy or sub-optimally tuned software also responsible for some problems (e.g. TCP stacks for large file transfers). • Consider evolution of PERT concept.

26. Middleware Findings

27. Identity Federations • Identity federations are solution for supporting user access to remote services. • Most NRENs have identity federation or are establishing one. Others should plan to do so within next couple of years. • NRENs are natural candidates for supporting technical organisation within their countries, as well as representing national federations. • User-centric identity (e.g. OpenId) management also growing, and abstract identity framework also being worked on. NRENs should monitor developments. • Already integrations of identity federation and OpenId

28. Interoperability • Inter-operability of identity federation happening: • SAML 2.0 is today choice for exchanging identity data for web-based applications. • All the identity federations technologies are SAML2.0-compatible or they migrating to be SAML2.0-compatible. • Schemas such as eduPerson or SCHAC becoming more important to facilitate inter-operability. • In order to be able to handle different AAIs it is recommended that NRENs support multiple trust infrastructures: • X.509 certificates used quite a lot. • SAML signed tokens, coming up. • It is recommended that NRENs try to minimise number necessary (e.g. by reusing existing PKIs). • Still open issue: No well established standard for communicating identity data to applications. • NRENs should be proactive about this (possible task force?)

29. Further Information • EARNEST Reports • http://www.terena.org/activities/earnest/publications.html (Draft Technical Report available, minus some updates) • TERENA NGN Workshop • http://www.terena.org/activities/ngn-ws/ws1/ • Thanks to: Alcatel-Lucent, Calient, Cisco, DTU-COM, DANTE, Extreme Networks, Force10, i2CAT, IBM, Juniper, Liberty Alliance, MERLIN Project, Nortel, Sun Microsystems & SxIP plus the Advisory Panellists