Internet2 QBone: Building a Testbed for IP Differentiated Services

Internet2 QBone:Building a Testbed for IP Differentiated Services INET99 June 23rd, 1999 San Jose, California Ben Teitelbaum<ben@internet2.edu>

Internet2 Dogma:There is a circularity between advanced networks and advanced apps Enables NetworkedApplications NetworkEngineering Motivate

QBone Dogma Article1:Inverse Apps  Networking circularity has applied to QoS Inhibited QoS-needyApplications NetworkQoS Prevented

QBone Dogma Article2:Work with the neediest apps, build a testbed, and turn the arrows around! Enables QoS-needyApplications NetworkQoS Motivates

Internet2 QBone Initiative • Build interdomain testbed infrastructure • Balance networking research with providing a service • Experiment and improve understanding of DiffServ • Iterate and improve testbed design • Support intradomain & interdomain deployment • Lead and follow IETF standards work • Some parts of DiffServ architecture cooked; others far from it • Our experience will inform standards process • Openness of R&E community gives us an edge • We can live with somewhat flaky infrastructure • We are open to sharing implementation experiences and measurement data

Osama Aboul-Magd (Nortel) Andy Adamson (Michigan) Grenville Armitage (Lucent) Steve Blake (Torrent) Scott Bradner (Harvard) Scott Brim (Newbridge) Larry Conrad (Florida State) John Coulter (CA*net2) Chuck Song (MCI/vBNS) Fred Baker / Larry Dunn (Cisco) Rüdiger Geib (Deutsche Telekom) Terry Gray (U Washington) Jim Grisham (NYSERNet) Roch Guerin (Penn) Susan Hares (Merit) Joseph Lappa (CMU) Jay Kistler (FORE) Klara Nahrstedt (UIC) Kathleen Nichols (IETF coordination) Ken Pierce (3com) John Sikora (ATT Labs) Ben Teitelbaum (chair) John Wroclawski (MIT) a liaison from each MOU partner Internet2 QoS Working Group

Internet2 Applications • Qualitative and quantitative improvements in how we conduct research, teaching, and learning • Require advanced networks • Examples: • Interactive research collaboration and instruction • Real-time access to remote scientific instruments • Large-scale, multi-site computation and database processing • Shared virtual reality

Big Problem #1: Understanding Application Requirements • What services do tomorrow’s applications need? • Range of poorly-understood needs • Both intolerant apps (e.g. tele-immersion) and tolerant apps (e.g. large FTPs, desktop video conferencing) important • Many apps need absolute, per-flow QoS assurances • Adaptive apps may require a minimum level of QoS, but can exploit additional network resources if available • Some institutions/users want multiple classes of best-efforts service (CoS) with relative precedence levels Better Good Bad Adaptive Intolerant Tolerant Different App Needs • Need better understanding through experience

Big Problem #2: Scalability • Convergence of flows on the core means: • Large numbers of flows through each router • High forwarding rate requirements • Need to support QoS end-to-end, but keep per-flow state & packet forwarding overhead out of the core Lots of flows here!

CampusNetworks CampusNetworks GigaPoPs GigaPoPs Big Problem #3: Interoperability ... between separately administered and designed clouds ... Backbone Networks(vBNS, Abilene, …) … and between multiple implementations of network elements ... … is crucial if we are to provide end-to-end QoS.

DiffServ for Internet2 • July 1997 - February 1998 • WG struggled to understand needs of advanced applications / realities of QoS engineering • Frustrations with RSVP give birth to IETF DiffServ • May 1998 • WG recommends EF/Premium DiffServ focus for I2 QoS • First Internet2 Joint Applications/Engineering Workshop, Santa Clara, CA (report on web site) • October 1998 • QBone initiative launched

DiffServ Overview • Exploits edge/core distinction for scalability • Applications contract for specific QoS profiles • Policing at network periphery • A few simple, differentiated per-hop forwarding behaviors (PHBs) • Indicated in packet header • Applied to PHB traffic aggregates • PHBs + policing rules = range of services • Clouds contract for aggregate QoS traffic profiles • Policing at cloud-cloud boundary • Supports simple, bilateral business agreements

DiffServ Architecture Bandwidth Brokers (perform admissions control, manage network resources, configure leaf and edge devices) Destination Source BB BB Core routers Core routers Ingress Edge Router (classify, police, mark aggregates) Egress Edge Router(shape aggregates) Leaf Router (police, mark flows)

Example Service #1: Premium • Contract: leased line emulation at aspecified peak rate • PHB = “forward me first” (EF) • Policing rule = drop out-of-profile packets • On egress, clouds must shape EF aggregates to mask induced burstiness

Assured Premium Olympic (CoS) Why Premium First? • Simplest absolute service to understand • Strongest flavor of DiffServ • Could support our most demanding applications • Less demanding applications should work fine on emerging high-performance BE infrastructure • Explore other PHBs (AF) later • To understand this, consider “typical” Internet2 performance: • I2 networks are largely uncongested • Jitter and loss still occur • Route flaps to the commodity Internet still occur

Typical 1999 Internet2 Performance East Coast University to West Coast DOE Lab • Minimum Delay • 50th Percentile Delay • 90th Percentile Delay

MREN /STAR TAP MAGPI Texas GP NYSERNet SingAREN NCNI Merit PSC APAN NREN ... ... Initial QIG** 11 February 1999 (actual connectivity and participating networks may vary as deployment progresses) CRC UNB UBC NWU iCAIR EVL CTIT CA*Net2 UMN ARDNOC IU RISQ SURFNet NTU NUS ANL Ames KDD Labs ESNet Other NASA Labs OtherNGIXs Korea UMass Other DOE Labs LBNL Abilene vBNS UNC CMU UMich TAMU Duke NCSU UPenn

QBone BB Group • Open group chartered to recommend BB solutions for the QBone • Lead by Sue Hares - Merit Networks • Six R&E proto-BBs: • Merit • BCIT • UCLA • Extensive participation from corporate partners • QBone BB requirements draft on web site • Prototype inter-BB signaling protocol due soon • Telia / Luleå University of Technology • ANL/Globus • LBNL Clipper

QBone Milestones 1998 - 1999 • Sep 25th - Call for participation • Oct 30th - WG recommends initial QIG participants • Dec 1st - 1st QIG / QBone BB meeting (Evanston) • Jan 1st - WG makes major push on architecture draft • Jan 26th - 2nd QIG / QBone BB Meeting (RTP) • Mar 7th - Measurement sub-group drafts QMA • Mar 9th - 3rd QIG / QBone BB Meeting (Las Cruces) • May 21st - WG opens QIG • June 8th - Open QBone interop BOF (Pittsburgh)

QBone Architecture (10km view) • IETF “Diff” (EF PHB) + QBone “Serv” (QPS) • QBone Premium Service • Idea: converge on Jacobson’s VLL “Premium” service • Well-defined SLS: • Peak rate R & “Service MTU” M implying a token bucket meter • Near-zero loss • Low jitter • Delay variation due to queuing effects should be no greater than the packet transmission time of a service MTU sized packet • All bets are off if the reserved interdomain route flaps • Plus important value-adds: • Integrated measurement/dissemination infrastructure • Experimentation with pre-standards inter-domain bandwidth brokering and signaling

Collection metrics,EFandBE... Active metrics One-way delay-variation One-way loss Traceroutes e.g. IPPM Surveyors Passive metrics Load Discards (suggested) Link bandwidths (suggested) EF reservation load e.g. OCxMon, RTFM, MIBs ActiveMeasurements MIB-basedstatistics Boundary Router AM node Intra-Domain Premium Path Inter-Domain Premium Path PM node PM node PassiveMeasurements PassiveMeasurements QBone Domain2 QBone Domain1 QBone Domain3 QBone Measurement Architecture1

QBone Measurement Architecture2 • Dissemination • HTTP, even for raw data • real-time + archived measurements • Canonical names for: • Metrics • Domains • Standard metric aggregations: • Mostly 5-minute aggregations • Standard URL name space for: • MRTG-style plots • Raw ASCII data • http://<root_URL>/<source_domain>/<dest_domain>/<first_hop>/<date>/<type>.<aggregation>.{html | gif | txt}

H H H H ... ... GigaPoP GigaPoP Starting Simply • Intradomain • Van’s campus example • At least 10Mbps everywhere • “Count to ten” admissions control with no topological knowledge • Interdomain • Could we do something similar in the early QBone? • Problem: Worst case down-stream provisioning starts to look pretty bad even with small initial participant set.

ESNet, NREN, Int’l, ... C C C C C C C C C C C C C C C C C C L L GigaPoP GigaPoP GigaPoP GigaPoP GigaPoP GigaPoP Abilene vBNS Generic Internet2 Topology NGIXs

ESNet, NREN, STARTAP, ... Int’l C C C C C C C C C C C C C C C C C C L L GigaPoP GigaPoP GigaPoP GigaPoP GigaPoP GigaPoP Abilene vBNS Phase 0 Demand Assessment NGIXs

Phase 0 Deployment Planning • Converge on a consensus reservation matrix • Reservations will be static for period of phase • Reservation = {S, D, R, M, TR} • S = source • D = dest • R = peak rate • S, D are on campus network demarks • All bets are off if routing between S and D changes • All SLSs still bi-lateral, but Internet2 engineering will facilitate convergence • M = service MTU • TR = inter-domain traceroute

Phase 0 Demand Matrix … to here Campus EF Ingress Load Implies Maximum EF load to be offered from here R D Campus Policer Config

Coming Attractions... • Jun 99: QBone Architecture in last call • Jun 99: QBone BB Advisory Council will recommend a prototype inter-BB protocol • Jun/Jul 99: “Phase 0” rollout planning • Aug/Sep 99: Interdisciplinary QBone workshop • Fall 99: QBone Connect-a-thon (“QCon”) event • Fall 99: “Phase 0”

For more information... • QBone home page:http://www.internet2.edu/qbone • Internet2 QoS Working Group home page:http://www.internet2.edu/qos/wg

Internet2 QBone: Building a Testbed for IP Differentiated Services