IP Network Traffic Engineering

1. IP Network Traffic Engineering Albert Greenberg Internet and Networking Systems Research Lab AT&T Labs - Research; Florham Park, NJ See http://www.research.att.com/~jrex/papers/ieeenet00.ps (to appear in IEEE Network Magazine, special issue on Internet Traffic Engineering, March 2000). Joint work with Anja Feldmann, Carsten Lund, Nick Reingold and Jennifer Rexford.

2. IP Network Traffic Engineering • Goal? In operational IP networks, improve performance and make more efficient use of network resources, by better matching the resources with traffic demands • How? By integrating • traffic measurement • network modeling • selection and configuration of network management and control mechanisms. • Time Scale? Tens of minutes, hours, days, … • Applications? • Troubleshooting performance problems. • Why is this link congested? • Incremental load balancing • How to tune intradomain (OSPF, IS-IS) routing weights, or interdomain (BGP) import policies? • Capacity planning and optimization • How to estimate facilities cost from forecasted demands and optimal design? • Focus of this talk: ISP backbone networks • (See framework draft of new IETF, Traffic Engineering Working Group)

3. Traffic Engineering in IP Networks • Topology • Connectivity and capacity of routers and links • Demands • Expected load between points in the network • Routing • Tunable rules for selecting a path for each traffic flow • Performance objective • Balanced load, low latency, service level agreements, … • Question: Given the topology and traffic demands in an IP network, how do you decide which routes to use?

4. Short Answer? • The desired detailed, up to date, network-wide views of the topology are unavailable • The prevailing traffic demands are unknown • The network doesn’t adapt path selection to the load • The static routes aren’t necessarily optimized to the traffic These challenges arise because IP networks are • Decentralized • Self-configuring • Connectionless • Operating in loose confederation with peers Attributes that contributed to success and dominance of IP

5. Example: Congested Link • Detecting that a link is congested • Utilization statistics every five minutes from SNMP • Active probes suffer degraded performance • Customers complain • Reasons why the link might be congested • Increase in demand between some set of source-destination pairs • Failed router/link in our network causes change in our routes • Failure or policy change in another ISP changes traffic flow • How to determine why the link is congested? • How to relieve the congestion on the link?

6. Configuration Debugging Measurements Reporting Configuration Performance Debugging Network Evolution Information Model Provisioning Capacity Planning Long Answer! • Derive topology from network configuration information • Compute traffic demands from edge measurements • Model path selection achieved by IP routing protocols • Build a query and visualization environment for “what-if” analysis

7. Toolkit Architecture Analysis/Visualization Routing Model Important to separate models from methods and data used to populate models Info Model Configuration Measurements

8. Configuration • Information • Backbone topology, link capacities, and router locations • Layer 2 and layer 3 links (e.g., ATM PVCs) • Intra-domain and inter-domain routing (e.g., OSPF weights) • Customer location and IP addresses; external IP addresses • Administrative policies, conventions • Construct • Unified views of the network topology, and of customer and peer reachability • Main sources: router configuration files, forwarding tables

9. Measurements • Performance statistics • Impact of traffic demands on the network • delay, loss, throughput from active probes between edge systems • Utilization, loss statistics from passive monitoring of links, nodes • Mapping of statistics onto the network topology • Traffic Demands • An accurate view of the demands themselves is extremely useful for effective traffic engineering • A large fraction of the traffic is interdomain, and a large number of customers are multihomed • Model traffic demands as loads from an edge interface to a set of candidate edge interfaces

10. Information Model • Abstraction of IP networks • Different views • router complexes, router, physical (layer 2), abstract (for routing) • Objects representing • routers, links, and traffic demands • Methods for manipulating objects • finding and selection of objects • linkage of objects, e.g., router complexes to routers • statistics: histogram, tables, etc. • Salient features • Captures important global network properties • Supports routing simulation (e.g., change of OSPF weights) • Trade off between accuracy and simplicity of model

11. Visualization of Link Utilization and Delay in Backbone Utilization (from passive measurement): link color (high to low) Delay (from active probes): link width (high to low)

12. Y1 X1 Backbone Y2 X2 Y3 access links X3 Y4 peering links X4 Y5 Routing Model • Capture: selection of shortest paths to/from (multihomed) customers and peers; splitting of traffic across multiple shortest paths; multiplexing of layer 3 links over layer 2 trunks

13. Routing Model (continued) • Intradomain (OSPF) routing emulator • Extract backbone topology and link weights • Compute all shortest paths (Dijkstra’s algorithm) • Split load evenly along all shortest paths • Emulates Cisco-style use of multiple routes 0.25 0.25 0.5 1.0 1.0 0.25 0.25 0.5 0.5 0.5

14. Visualization of Traffic Flow in Backbone Color/size of node: proportional to traffic to this router (highto low) Color/size of link: proportional to traffic carried (high to low)

15. Systems • Configuration • construction of network topology: layer 2, 3 connectivity, capacity, OSPF weights, customer and peer IP addresses, router locations • Measurements • Performance (active – delay, loss, throughput; passive – link and node utilization) • Traffic demands • Information model • physical level, IP level, router-complex level, abstract level • router attributes, link attributes • Routing model • shortest-path routing, OSPF tie-break, multi-homing, interdomain routing • bookkeeping to accumulate traffic load on each link • Visualization/analysis environment • querying to subselect links and nodes; histograms; what-if capabilities • coloring and sizing to illustrate link and node statistics

16. Key Ideas • data (configuration, routing, measurement) | models (topology, demands, routing) | analysis • Generate accurate global views of the network, and provide mechanisms to infer network-wide implications of changes in traffic, configuration and control • Architecture—separate systems for measurement, models, methods to populate models, analysis • Can and must evolve with change to underlying infrastructure and network architecture • Interfaces for modules above • E.g., design and optimization (e.g., Bernard Fortz and Mikkel Thorup, "Internet Traffic Engineering by Optimizing OSPF Weights," Proc. IEEE INFOCOM, March 2000. http://www.ieee-infocom.org/2000/papers/165.ps) • … | (informed) provisioning and reconfiguration • Closing the loop… • Improving performance and making more efficient use of network resources