Jon Turner jst@cs.wustl arl.wustl/arl

1. Extreme NetworkingAchieving Nonstop Network Operation Under Extreme Operating ConditionsDARPA PI Meeting, July 23-26, 2002 Jon Turnerjst@cs.wustl.eduhttp://www.arl.wustl.edu/arl

2. Project Overview • Motivation • data networks have become mission-critical resource • networks often subject to extreme traffic conditions • need to design networks for worst-case conditions • technology advances making extreme defenses practical • Extreme network services • Lightweight Flow Setup (LFS) • Network Access Service (NAS) • Reserved Tree Service (RTS) • Key router technology components • Super-Scalable Packet Scheduling (SPS) • Dynamic Queues with Auto-aggregation (DQA) • Scalable Distributed Queueing (SDQ)

3. Prototype Extreme Router ControlProcessor Field Programmable Port Ext. Smart Port Card SDRAM128 MB SRAM4 MB Switch Fabric 64 MB Sys.FPGA APIC NorthBridge IPP OPP IPP IPP IPP IPP OPP OPP OPP OPP IPP OPP Pentium FPX FPX FPX FPX FPX FPX Cache ATM Switch Core SPC SPC SPC SPC SPC SPC Field Programmable Port Extenders ReprogrammableApplicationDevice NetworkInterfaceDevice TI TI TI TI TI TI Transmisson Interfaces Embedded Processors

4. Resource Reservation in Internet? • Bandwidth reservation can provide dramatically better performance for some applications. • Obstacles to resource reservation in Internet. • distaste for signaling protocols • perceived complexity of IntServ+RSVP • requires end-to-end deployment • little motivation for service providers • How to get resource reservation in Internet? • keep it simple • focus on top priorities - one-way unicast flows • avoid complex signaling - leverage hardware routing mechanisms • make it useful when only partially deployed • provide motivation for ISPs to deploy it

5. 10 Mb/s available 5 Mb/s available 20 Mb/s available 5 Mb/s available 2 Mb/s available Basic LFS Operation Select path and attempt to reserve Reserve bandwidth Reserve 8 Mb/s to B Complete reservation A 20 Mb/s available B Select best next hop Select path and reserve • One way, unicast setup with partial reservation. • complete reservations locally when bandwidth released • Optional ack returned by far-end access router. • Reservation may terminate explicitly or time out. • May alter reserved bandwidth but no re-routing.

6. Soft Reservations • Basic LFS provides firm reservations. • user guaranteed bandwidth until releases • Can extend to provide soft reservations as well. • soft reservation can be adjusted by the network as traffic changes • can be intermixed with firm reservations to provide a firm minimum, plus more bandwidth as available • Uses of soft reservation. • apps. that need guaranteed minimum and can sometimes use more, but can adjust use to what’s available • more rapidly responding congestion control for traditional best-effort traffic

7. IP header(fixed part) code length op. flags Rrate Arate trace IP payload Basic IP Option for LFS • Allocated rate stored at “last hop” router for status generation • F.P. rates with 4 bit mantissa, 4 bit exponent. • specify rates from 64 Kb/s to 4 Gb/s , 6% “granularity” • Code identifies LFS option. • Operations • request firm reservation • request soft reservation • release state • Flags • sender status request • sender network status request • public network status request • intra-domain status request • congested path • Rrate: requested rate. • Arate: allocated rate. • Trace used by each domain to track usage.

8. A domainU domainV domainW X Z Y B X Y Z Use of Trace Field acct. record[A,B,..] thru X acct. record[A,B,..] thru Z acct. record[A,B,..] thru Y • Network providers need to monitor LFS usage for network management and accounting purposes. • trace field used by ingress router of each domain to mark LFS packets with domain-specific identification • egress router of each domain maintains record of each LFS flow, including copy of trace field • end-to-end records created through off-line accounting resolution mechanisms

9. sender status sender net status public net status sender LAN ISPU ISPV rcvr. LAN intra-domain status Status Reporting • Basic LFS option supports sender status and trace field for accounting. • Network providers likely to want more. • sender net status allows LFS service verification • public net status allows “end-to-end” status check • intra-domain status for verifying local status • each “extra” status report requires insertion of requestor’s IP address, increasing LFS option length

10. Partial Deployment • Receivers need not be LFS-aware. • web site may use LFS to reserve bandwidth for streaming media - users benefit, even without LFS-aware hosts • Issues with non-contiguous LFS domains. • route changes may create “orphan reservations” • no simple way to determine status reporter • No support for non-contiguous LFS domains. • LFS router forwarding to a non-LFS router (or host) strips LFS option and implements status reporting • status report includes IP address of reporting router, letting sender know how far the reservation went • Public IP carrier can accept LFS option from client networks (LAN) even if client net is not LFS-aware. • Clients may use tunnel to access LFS service.

11. Regulating LFS Use - Net Access Svc • Permitting unconstrained access to LFS creates big security vulnerability. • Limit use to authorized users. • Limit number of reservations and amount of reserved bandwidth by authorized users. • access router keeps record and enforces limits • complication - user may use LFS from multiple locations • maintain records in distributed set of servers - each server keeps records for some fraction of the users - use hashing to select • Access router needs means to identify user. • host IP address insufficient (DHCP, NAT) • encryption-based authentication (IPSEC) • Combine access control with usage accounting. • What special issues arise with multiple domains?

12. LFS Video Demo Configuration cross traffic sinks video source 100 Mb/s links video sink cross traffic sources • Wavelet-coded video with and without LFS. • competing datagram traffic • with no reservation, lost packets cause poor video quality • with reservation, high quality preserved

13. Video Demo - No Reservation video source cross traffic sources all sinks video flow - no reservation datagram cross traffic flow 1 datagram cross traffic flow 2

14. Video Demo - With Reservation video sink cross traffic sinks video flow - with reservation datagram cross traffic flow 1 datagram cross traffic flow 2

15. Competing LFS Flows sources sink 2 sinks sink 1 flow 1 - no reservation flow 2 - reservation added flow 3 - no reservation no reservations reservation for flow 2

16. Partial Reservation source 1 flow 2 sink 1 sink 3 flow 1 - partial reservation made

17. Completing Partial Reservation sink 3 sink 1 flow 1 - completes partial reservation flow 2 - drops reservation

18. Addition of Flow 3 Reservation sink 2 sink 3 flow 3 - adds reservation

19. Performance of LFS at Single Link Pareto distributed session times make little difference OC-48 link can carry 200 flows of 12 Mb/s • m = number of flows link can carry • exponential session times for flows, infinite queue very few flows experience any delay

20. Sensitivity to Load and Hop Count delay probability scales linearly with number of hops at 90% load, less than 1 flow in 100 delayed more than 12% of session time

21. Overload Performance with no buffer most sessions still succeed with infinite buffer, no sessions get small delays (10%) buffer reduces rejection fraction at low loads

22. Summary • LFS provides simple reservations for QoS. • no complex signaling, wire speed setup • limited deployment can be broadly beneficial • support for usage monitoring & accounting gives network providers a motivation to deploy service • Network access service for regulating usage. • preliminary specification has been developed • uses IPSEC for host/user authentication • Performance analysis, simulation study underway. • Routing issues. • evaluate QoS routing with multiple-choice forwarding • link state distribution for inter-domain routing • inter-domain routing policies