Traffic Dynamics at a Commercial Backbone POP

1. Traffic Dynamics at a Commercial Backbone POP Nina Taft Sprint ATL Co-authors: Supratik Bhattacharyya, Jorjeta Jetcheva, Christophe Diot

2. Outline • Part 1: what are the traffic demands between pairs of POPs? • How stable is this demand? • Part 2: what are the paths taken by those demands? • Are link utilizations levels similar throughout the backbone? • Part 3: is there a better way to spread the traffic across paths? • Can we divert some traffic to lightly loaded paths?

3. The Sprint IPMon Project • Passive monitoring • Capture header (44 bytes) from every packet • full TCP/IP headers, no http information • Use GPS time stamping - allows accurate correlating of packets on different links • Day long traces • Simultaneously monitor multiple links and sites. • Collect routing information along with packet traces. • Traces archived for future use

4. IP Backbone : POP-to-POP view OC-192 OC-48 POP fanout: one row of POP-to-POP traffic matrix OC-12

5. City A City B City C City A City B City C POP-to-POP Traffic Matrix For every ingress POP : Identify total traffic to each egress POP Further analyze this traffic Measure traffic over different timescales Divide traffic per destination prefix, protocol, etc.

6. The Mapping Problem What is the egress POP for a packet entering a given ingress POP?

7. Access Access Access Access Monitored links at a single POP Publicpeer 2 Public peer 1 Core Core Core ISP web host Date : Aug 9, 2000

8. Traffic Fanout: POP level granularity

9. Fanout: web host links

10. Time-of-Day for POP level granularity

11. Day-Night Variation : Webhost #1 % reduction at night between 20-50% depending upon access link

12. Summary so far ... • Wide disparity in “traffic demands” among egress POPs • POPs can be roughly categorized as : small, medium, large; and they maintain their rank during the day. • Traffic is heterogeneous in space yet stable in time. • 20-50% reduction at night

13. Outline • Part 1: what are the traffic demands between pairs of POPs? • How stable is this demand? • Part 2: what are the paths taken by those demands? • Are link utilizations levels similar throughout the backbone? • Part 3: is there a better way to spread the traffic across paths? • Can we divert some traffic to lightly loaded paths?

14. Paths used by traffic demands • Our Observations (summary) • routing policies concentrate traffic on a few paths: between two POPs, all the traffic uses either the same route, or 1 or 2 routes • the ISIS weights are changed very infrequently (once a month), so routing is fairly static • there are many underutilized routes

15. Is backbone traffic balanced?

16. Part 3 : Can we divert some traffic to lightly loaded paths? • Approach: to improve load balancing by rerouting only a few flows • scalable • Which flows? Heavy hitters. • How identify heavy hitters: Consider: destination prefix-based flows • at fixed prefix lengths: 8 and 16 • BGP table entries (variable prefix length)

17. Streams based on destination prefix Traffic grouped by egress POPs Stream : all packets in a group with same /8 destination address prefix Similar results for /16 and bgp table prefixes Ingress : Webhost Link

18. Stability of prefix-based streams Stability of prefix rank • Ri(n) = the rank of flow i at time slot n • Di,n,k= | Ri(n) - Ri(n+k) | each time slot

19. Conclusions • We have used our data to build components of traffic matrices for traffic engineering • Heterogeneous traffic fanout from POP • Current routing practices lead to many underutilized links and paths • thus, there is a lot of room for improved load balancing techniques. • Load-balancing using flows selected via destination-prefixes is a simple and promising criterion

20. Ongoing Work • Intra-domain Routing : • Choosing ISIS link weights • Multi-path routing • Flow Characterization at the network prefix level • Inference techniques for building POP-to-POP traffic matices