Internet2 Update R/D and Infrastructure

1. Internet2 UpdateR/D and Infrastructure Guy AlmesInternet2 Project <almes@internet2.edu> NANOG MeetingDearborn — 9 June 1998

2. Outline of the Talk • Technical Working Groups • The Challenge of Delay-Bandwidth Products • Abilene Project Update

3. Applications and Engineering Applications Motivate Enables Engineering

4. Comments on Apps and Plumbing • Advanced applications transform high-speed plumbing into value • Advanced plumbing enables advanced applications • Profligate use of bandwidth, per se, does not make an application ‘advanced’ • Megalomaniac plumbing, per se, does not make the plumbing ‘advanced’

5. IPv6 Measurement Multicast Network Management Network Storage Quality of Service Routing Security Topology Technical Working Groups

6. Chair: Dale Finkelson, Univ Nebraska <dmf@unl.edu> Membership: Total 12; 9 .edu, 3 .com, 1 .gov Focus: Explore the rôle that IPv6 might play in the Internet2 project Work with those interested in IPv6 to build IPv6 testbeds across the Internet2 structure, including vBNS and Abilene IPv6

7. Measurement • Chair: David Wasley, Univ California <david.wasley@ucop.edu> • Focus: • Places to measure: • at campuses, at gigaPoPs, within interconnect(s) • Things to measure • traffic utilization • performance: delay and packet loss • traffic characterization

8. One example measurement technology • IETF IPPM WG defining one-way delay • Take all delay to be due to: • Propagation • Transmission • Queuing • Variation in delay suggests congestion

14. Multicast • Chair: vacant [Dave Meyer, Univ Oregon still serving] • Nearing completion of naming a successor • Membership: Total 3; 3 .edu • Focus: Make native IP multicast scalable and operationally effective

15. Network Management • Chair: Mark Johnson, MCNC <mj@ncren.net> • Membership: Total 4; 3 .edu, 1 .com • Focus: • Common trouble ticket system • How can all our interconnects and gigaPoPs and universities appear to be a seamless whole?

16. Network Storage • Chair: Micah Beck, Univ Tennessee <mbeck@utk.edu> • Membership: Total 13; 9 .edu, 4 .com • Focus: Distributed Storage Infrastructure for Internet2 • Replication • Physical proximity • Transparency

17. Quality of Service • Chair: Ben Teitelbaum, Internet2 staff <ben@internet2.edu> • Membership: Total 36; 17 .edu, 19 .com • Focus: Multi-network IP based QoS • Relevant to advanced applications • Interoperability: carriers and kit • Scalable • Administratable and Measurable • Hosts, campus/gigaPoP/Interconnect routers/switches

18. Quality of Service Sketch A B • Does the approach support advanced applications? • Are there implementations that work? Only one? • If cloud ‘A’ and cloud ‘B’ both implement QoS, does the combined A+B catenation implement QoS?

19. Results to date: Requirements document Series of technical recommendations First Internet2 Joint Applications/ Engineering QoS WorkshopSanta Clara, CaliforniaMay 21-22, 1998Hosted by Bay Networks QoS, continued

20. Routing • Chair: Steve Corbato, Univ Washington <corbato@cac.washington.edu> • Membership: Total 48; 32 .edu, 16 .com • Focus: Internal and External routing • Critical issues • gigaPoP internal routing design • Explicit routing requirement (the “fish problem”) • Met at UCSD in January (21 attendees) • gigaPoP external routing recommendations • Subscribers (Internet2 campuses) • National interconnects (vBNS, Abilene, and NGI networks)

21. Security • Chair: Peter Berger, Carniege Mellon Univ <peterb@hoopoe.psc.edu> • Membership: Total 13; 13 .edu • Focus: • Authentication • Application to QoS • Application to Digital Libraries

22. Topology • Chair: Paul Love, Internet2 staff <epl@internet2.edu> • Membership: Total 16; 13 .edu, 2 .com, 1 .gov • Focus: Topology of Internet2 • Internal Internet2 Connections • Internet2 with other Advanced Research Networks

23. Summary • Internet2’s WGs focus on project’s needs • Complement IETF WGs • Membership by invitation - welcome participation by Internet2 corporate members

24. Large Delay-Bandwidth Products • As the product of delay and bandwidth grows: • The number of unacknowledged packets grows • It becomes more difficult to sustain a steady stream of data from end to end • Several consequences: • Need for direct physical paths • Tradeoff between buffering and variation in delay

25. A pessimistic result from Mathis et al. • Mathis, Semke, Mahdavi, and Ott, "The Macroscopic Behavior of the TCP Congestion Avoidance Algorithm", Computer Communication Review, July 1997. • www.psc.edu/networking/papers/model_abstract.html • BW  C * packet-size / (delay *  packet-loss)

26. Consider the implications for the internationalhigh-performance Internet • BW  packet-size • BW  1 / delay • BW  1 /  packet-loss

27. Example: Delay BW  C / delay delay due to distance original raw bandwidth

28. Example: Delay with fatter pipe BW  C / delay delay due to distance more raw bandwidth

29. Example: Packet Loss similar phenomenon, but … to double bandwidth, you must cut packet loss by four

30. Abilene Update UCAID Project Addresses infrastructure needs of Internet2

31. Goals and Objectives • Provide high-quality, widely available Interconnect among participating gigaPoPs/universities • Connect to Internet2 members via the vBNS and to other key research/ education sites via Internet2/NGI-class federal and non-US nets

32. Goals and Objectives, continued • Support QoS architecture as it evolves • Support other advanced functionality as it evolves • Maximize Robustness • Minimize Latency • Provide Capacity to Avoid Congestion

33. Evolution of Abilene with Time • Phase 1: use of operational Qwest Sonet • Phase 2: use of separate wavelengths • Phase 3: use of separate fibers • Allows capacity to grow with Internet2 needs

34. Key Attributes • IP over Sonet • Benefit from Qwest OC-48 Sonet capacity and collocation sites • Benefit from Nortel OC-192 Sonet kit and Lucent fiber • Benefit from Cisco GSR 12000 routers

35. Architecture: Core • About 11 (up to 30) core nodes • Each located at a Qwest PoP • Each with a Cisco 12008 router • Rack also contains measurements/ management computers • Interior lines connect core nodes • OC-12 and (eventually) OC-48 Sonet • IP-over-Sonet interfaces

37. Subset of Route Map of Interest to Abilene sttl syrc bstn milw chcg dtrt alby clev eugn mpls nycm pitb ipls phil scrm slkc dnvr tpka kscy lsvl wash rcmd nsvl albq rlgh atln anhm phnx elpa hstn nwor tlhs

38. Attitude toward interior lines Robustness: mesh plus Sonet Latency: direct physical paths Capacity: avoid congestion

39. Architecture: Access Access node at many Qwest PoPs Qwest Sonet switches needed equipment Access lines connect from core node to gigaPoP Local part: gigaPoP to access node Long distance part: access node to core node IP-over-Sonet or IP-over-ATM possible OC-3 and OC-12 typical

40. One cost-sharing implication Long-distance part of access line is considered part of the ‘backbone’ Thus, number/location of core nodes does not affect costs borne by gigaPoP

41. One robustness implication Each access line is Sonet Long-distance part (at least) will be configured from protected Sonet ring Thus, single access line can tolerate a break in the long-distance part of the access line

42. OK, so where’s the map? Self-selection is key Each gigaPoP will determine where, when, at what speed it connects Detailed topology will be based on engineering considerations