Deploying IPv6 Across the Internet2 Infrastructure

1. Deploying IPv6 Across the Internet2 Infrastructure Rick Summerhill Associate Director, Backbone Network Infrastructure, Internet2 North American IPv6 Global Summit San Diego, CA 24 June 2003

2. Outline • General Internet2 Infrastructure • Internet2 Goals • Abilene Partners • Abilene Backbone • Connectors • Peers • Focus • IPv6 Deployment • Goals • History • Backbone • Measurement • Support

3. Internet2 Goals • Create a leading edge network capability for the national research community • Enable revolutionary Internet applications • Ensure the rapid transfer of new network services and applications to the broader Internet community.

4. Partners • Internet2 • Cisco Systems • Indiana University • Juniper Networks • Nortel Networks • Qwest Communications • North Carolina, Ohio, San Diego ITECs

5. Abilene Backbone • Abilene backbone – OC-192c over unprotected DWDM waves with SONET framing • In final stages of an upgrade to OC-192c • Often easier to deploy advanced services on a backbone network than at the edges • Multicast • IPv6 • Topology

6. Abilene Backbone(Late Summer 2003)

7. Abilene Backbone(Early Summer 2003)

8. Abilene scale(April 2003) • 48 direct connections (OC-3c  10-Gbps) • 2 10-Gbps connections (both 10-Gigabit Ethernet) • 6 OC-48c • 1 Gigabit Ethernet • 23 connections at OC-12c (622 Mbps) or higher • 221 participants – universities and labs • All 50 states, District of Columbia, & Puerto Rico • Recently: Oak Ridge National Laboratory • Expanded access • 85 sponsored participants • 28 state education networks

9. 09 January 2002 Last updated: 17 January 2003 Abilene Federal/Research Peering (Late Spring 2003)

10. 09 January 2002 Abilene International Peering(Late Spring 2003)

11. Internet2 Infrastructure • The Full Internet2 infrastructure is diverse and complex • Backbone is relatively simple • Management provided by Indiana Global NOC • Testing by Internet2 Test and Evaluation Centers (ITECs) • Connectors often exhibit a complicated hierarchy • Some research institutions connected directly • Some are connected through regional networks, state networks, and some have complex campus networks • Land Grant institutions often have county extension offices • Diversity/Complexity increases as one gets closer to the edges of the network • Influences the way IPv6 is implemented • Consider the classic IPv6 addressing/routing plan, with potentially multiple connections, in this diverse infrastructure

12. Internet2 Infrastructure Simple to Complex Hierarchies within the Internet2 Infrastructures DNS and multi-homing issues within these types of hierarchies

13. Abilene Focus Areas - 2003 • Advanced Services • IPv6 and Multicast (and IPv6 Multicast) • All the following include both IPv4 and IPv6 – the common bearer service for Abilene is both IPv4 and IPv6 • Facilitating end-to-end performance • Supporting network research – Abilene Observatory • Experimenting with MPLS/VPN on backbone • Supporting large MTUs • Security and the REN-ISAC

14. Internet2 IPv6 Goals • Support and encourage development of advanced applications using IPv6 • Create a national infrastructure to support IPv6 • Implement IPv6 on Abilene Backbone • Encourage deployment of IPv6 throughout the Internet2 infrastructure • Support end-2-end transparency for IPv6 advanced applications • Important issue for high performance applications • High performance applications often have trouble with NATs • Provide a more robust infrastructure to provide security • Educating the Internet2 IPv6 user base • Support interconnectivity and transit during the initial stages of IPv6 deployment

15. Abilene IPv6 History • Substantial input from the Internet2 IPv6 working group • Initial Tunnel Approach • At the beginning stages, code was not available for backbone routers • Cisco donated four 7200 routers for a tunnel deployment, originally located in gigapops • Tunnels across the IPv4 backbone, and to IPv4 connectors • Gigapops used tunneled connections to this “tunnel backbone”, and connected universities also using tunnels • Routing – BGP and RIP-NG (later ISIS) • Relatively simple process – tunnel interfaces with v6 addresses

16. Abilene IPv6 History

17. Abilene IPv6 History • Transition to native dual stack backbone • Found we couldn’t do high-bandwidth applications across the tunnel backbone. • DVTS video application from Japan to DC stressed tunnel approach • Major concern was the effect of IPv6 on IPv4 performance • IPv6 configured on backbone routers – originally Cisco GSRs, now Juniper T-640s • Tunnel backbone and dual stack backbone connected together – Internal BGP and IS-IS. • Connectors transitioned to native dual stack mode • Tunnel backbone gradually replaced and reduced – now down to one router to support legacy tunnel connectors. Router is located at Indiana NOC.

18. Abilene IPv6 Backbone • Currently a native IPv6 Dual Stack implementation • Fully deployed on all Abilene Routers • Routing is BGP and ISIS (for both IPv4 and IPv6) • Peering and connectivity does not fall under the Abilene CoU • Is there interest in Abilene IPv6 connectivity? • Legacy tunnel connections still supported on single router at Indianapolis • Successful Tests: • 8 Gbps across backbone • IPv6 only and mixed IPv6/IPv4

19. Abilene IPv6 Backbone • Addressing Plan • Currently have /32, originally was a /35 • Allocate a /40 to a gigapop or a /48 to a university • Some gigapops have 12 member universities and had to allocate their /40 to those universities plus state networks • Universities immediately felt constrained by this • Recall that some universities have locations in potentially 200 counties within a state (i.e. Land Grant Institutions) • WiscREN and Pittsburgh gigapops have obtained their own space • Some universities attempting to obtain space – could satisfy current ARIN guidelines

20. Abilene IPv6 Backbone • Current IPv6 Connectors • Front Range Gigapop • Great Plains Network • Indiana Gigapop • Intermountain Gigapop • MAGPI • MREN • Merit • Mid-Atlantic Crossroads • NYSERNet • North Texas Gigapop • Northern Crossroads • Northern Lights • OneNet • Oregon Gigapop • Pittsburgh Gigapop • SDSC/UCSD • University of Memphis • WiscREN • Current IPv6 Peers • ASNet • CUDI • CA*net • ESnet • GEANT • HARnet • HEAnet • Hurricane Electric • Kreonet • NORDUnet • Surfnet • WIDE • Other Peers • DTF • vBNS • Exchange Points: • 6Tap (StarTap) • StarLight • PacWave

21. Abilene IPv6 Backbone • Current Connectors/Peers • 18 native connections • 17 native peers • IPv6 Multicast • Testing at North Carolina ITEC • Internet2 consensus is that SSM is the appropriate direction for multicast under IPv6 • Need layer 2 devices to do the correct thing • DNS • Production type server for reverse lookups at NOC – points to gigapop servers for details • Connectors/members handle there own forward lookups • Simple implementations right now

22. Abilene IPv6 Backbone • Will provide an IPv6 Measurement Infrastructure • Attempt to perform measurements using IPv4 and IPv6 • Need for MIBs for basic measurements via SNMP • Types of data collected • One way latency tests • Throughput measurements • Netflow measurements • Routing • End-2-end performance testing

23. IPv6 in Gigapops, Campus • Implementing IPv6 in gigaPoPs is similar to implementing on backbone, although there are different approaches • Most are native connections now • Some tunnels persist back to the campus or even to individual departments • Campus implementations vary, but are gradually progressing, and are migrating deeper into the campus environments • Problems include: • Older equipment in locations where there is reluctance to upgrade because of a production environment • DNS – often done on separate servers • Deploying other applications • Fear of deploying IPv6 servers in dual stack mode • Naming issues

24. Internet2 Support for IPv6 • Tutorials • Two day workshops, hands-on experience • Descriptions and planning guides • http://ipv6.internet2.edu/workshops/index.shtml • Alternate discussion/lecture with hands-on work • Slides are available • http://ipv6.internet2.edu/fiu/presentations/ • Very popular events

25. Internet2 Support for IPv6 • Topics • Addressing • Allocation Schemes • Router Configuration • Basic Functionality • Multi-homing • Multi-homing Lab • Provider Independent Addressing • Provider Independent Addressing Lab • Under the Hood • Stateless Autoconfiguration • Neighbor Solicitation • Transition and Tunnels • DNS • Unix Hosts • Microsoft Windows • DVTS • ISIS • GigaPoP Implementations

26. Internet2 IPv6 Deployment Issues • Addressing and routing – multi-homing • Abilene currently has a /32, allocates /40 prefixes to large connectors, /48 prefixes to universities • The classic model for IPv6 is PA addressing, to contain the size/stability of the global routing table – potentially very large if using PI addressing. • Recall the complicated hierarchy within our infrastructure • Within the next 6 months, we expect member institutions to inherit multiple prefixes, potentially from 2 or more research networks and 2 or more commodity networks • Policy requirements complicate the multi-homing problem

27. Internet2 IPv6 Deployment Issues – The Multi-homing Problem

28. Internet2 IPv6 Deployment Issues – The Multi-homing Problem • How should we deal with this problem? We have to make this work. • Should we allow others to punch holes in the Internet2 prefix? • Should we punch holes in the prefixes of other research networks? • Should we encourage all connectors to obtain their own space? • At some point in the hierarchy, this solution doesn’t work • How to deal with multiple addresses is very important to our deployment • Does the punching of holes in PA space make sense? • Would it make more sense to allocate some PI space, make it work for now, allowing time for development of a solution to the multi-homing problem? Use some scheme, possibly one geographically based, to allow for a few years of development?

29. Internet2 IPv6 Deployment Issues • Routing databases – are these useful? • Should we deploy an IPv6 RADB database? • Experimental at first, production later?

30. Internet2 IPv6 Deployment Issues • References • http://www.internet2.edu • http://abilene.internet2.edu • http://ipv6.internet2.edu • abilene@internet2.edu • Questions?