Intra-domain Traffic Engineering

1. Intra-domain Traffic Engineering

2. Outline • Introduction to Internet Traffic Engineering • IP-based Traffic Engineering • MPLS-based Traffic Engineering

3. Do IP networks manage themselves? • In some sense, yes: • TCP senders send less traffic during congestion • Routing protocols adapt to topology changes • But, does the network run efficiently? • Congested link when idle paths exist? • High-delay path when a low-delay path exists? • How should routing adapt to the traffic? • Avoiding congested links in the network • Satisfying application requirements (e.g., delay) • … essential questions of Traffic Engineering

4. Traffic Engineering • What is Traffic Engineering? • Control and optimization of routing, to steer traffic through the network in the most effective way “Internet traffic engineering is defined as that aspect of Internet network engineering dealing with the issue of performance evaluation and performance optimization of operational IP networks. Traffic Engineering encompasses the application of technology and scientific principles to the measurement, characterization, modeling, and control of Internet traffic” RFC 3272 – Overview and Principles of Internet TE

5. Traffic Engineering • Two fundamental approaches: • IP-based Traffic Engineering • MPLS-based Traffic Engineering

6. 2 1 3 1 3 2 1 5 4 3 IP-based Traffic Engineering • Using traditional routing protocols: • Routers flood information to learn topology • Routers determine “next hop” to reach other routers • Path selection based on link weights (shortest path) • Link weights configured by network operator

7. Approaches for setting the link weights • Conventional static heuristic • Proportional to physical distance • Cross-country links have higher weights • Minimizes end-to-end propagation delay • Inversely proportional to link capacity • Smaller weights for higher-bandwidth links • Attracts more traffic to links with more capacity • Can we do better?

8. Approaches for setting the link weights • A traffic engineered approach • Collect measurements of traffic and topology • Tune the weights based on the offered traffic • Network management system sets the link weights • Acting on network-wide view of traffic and topology • Directly minimizes metrics like max link utilization

9. Network-wide “what if” model Offered traffic Changes to the network Topology/ Configuration TE approach: Measure, Model and Control measure control Operational network

10. IP-based Traffic Engineering • Topology • Connectivity and capacity of routers and links • Traffic matrix • Offered load between points in the network • Link weights • Configurable parameters for routing protocol • Performance objective • Balanced load, low latency, … • Question: Given the topology and the traffic matrix, which link weights should be used?

11. IP-based Traffic Engineering • Instrumentation • Topology: monitoring of the routing protocols • Traffic matrix: fine-grained traffic measurement • Network-wide models • Representations of topology and traffic • “What-if” models of shortest-path routing • Network optimization • Efficient algorithms to find good configurations • Operational experience to identify key constraints

12. Topology/Routing • Router configuration files • Daily snapshot of network assets & configuration • Software to parse the router config commands • Network-wide view of topology & routing policies • Also useful for detecting configuration mistakes • Routing monitors • Online monitoring of routing protocol messages • Real-time view of routes via neighboring ASes • Real-time view of paths within the AS • Software for aggregating and querying the data • Also useful for detecting and diagnosing anomalies

13. Formalizing the Optimization Problem • Input: graph G(R,L) • R is the set of routers • L is the set of unidirectional links • cl is the capacity of link l • Input: traffic matrix • Mi,j is traffic load from router i to j • Output: setting of the link weights • wl is weight on unidirectional link l • Pi,j,l is fraction of traffic from i to j traversing link l

14. Formalizing the Optimization Problem • A solution to the optimization problem cannot freely allocate the traffic load between two routers • The traffic between two routers can only flow along a shortest weighted path (routing protocols work this way) • This constraint implies that we achieve a sub-optimal solution to the general routing problem • In case multiple shortest weighted paths exist, traffic can be split among them • In practice, this is allowed by OSPF by means of the ECMP (Equal Cost Multi-Path) function

15. Formalizing the Optimization Problem 0.25 0.25 0.5 1.0 1.0 0.25 0.25 0.5 0.5 0.5 • To enforce the optimal solution, it is necessary to split traffic along multiple shortest paths Values of Pi,j,l

16. Complexity of the Optimization Problem • NP-complete optimization problem • No efficient algorithm to find the link weights • Even for simple objective functions • What are the implications? • Have to resort to searching through weight settings

17. Optimization based on local search • Start with an initial setting of the link weights • E.g., same integer weight on every link • E.g., weights inversely proportional to capacity • E.g., existing weights in the operational network • Compute the objective function • Compute the all-pairs shortest paths to get Pi,j,l • Apply the traffic matrix Mi,j to get link loads ul • Evaluate the objective function from the ul/cl • Generate a new setting of the link weights repeat

18. Making the search efficient • Avoid repeating the same weight setting • Keep track of past values of the weight setting • … or keep a small signature of past values • Do not evaluate setting if signatures match • Avoid computing shortest paths from scratch • Explore settings that changes just one weight • Apply fast incremental shortest-path algorithms • Limit number of unique values of link weights • Do not explore 216 possible values for each weight • Stop early, before exploring all settings

19. Application to AT&T's backbone network • Performance of the optimized weights • Search finds a good solution within a few minutes • Much better than link capacity or physical distance • Competitive with multi-commodity flow solution • Optimal routing possible with more flexible routing protocols (e.g. MPLS) • How AT&T changes the link weights • Maintenance every night from midnight to 6am • Predict effects of removing link(s) from network • Reoptimize the link weights to avoid congestion • Configure new weights before disabling equipment

20. To learn more... • Overview papers • B. Fortz, J. Rexford, M. Thorup, “Traffic engineering with traditional IP routing protocols”, IEEE Communication Magazine, October 2002 • Traffic measurement • R. Caceres, N. Duffield, A. Feldmann, et al., “Measurement and analysis of IP network usage and behavior”, IEEE Communications Magazine, May 2000 • A. Feldmann, A. Greenberg, C. Lund, N. Reingold, J. Rexford, F. True, “Deriving traffic demands for operational IP networks: Methodology and experience”, IEEE/ACM Transactions on Networking, June 2001 • Topology and configuration • A. Feldmann, J. Rexford, “IP network configuration for intradomain traffic engineering”, IEEE Network Magazine, September/October 2001 • Intradomain route optimization • B. Fortz, M. Thorup, “Internet traffic engineering by optimizing OSPF weights”, Proc. Of IEEE INFOCOM 2000

21. Traffic Engineering • Two fundamental approaches: • IP-based Traffic Engineering • MPLS-based Traffic Engineering

22. Multi-Protocol Label Switching • Multi-Protocol • Encapsulate a data packet • Could be IP, or some other protocol (e.g., IPX) • Put an MPLS header in front of the packet • Actually, can even build a stack of labels… • Label Switching • MPLS header includes a label • Label switching between MPLS-capable routers IP packet MPLS header

23. Forwarding Equivalence Class (FEC) • Rule for grouping packets • Packets that should be treated the same way • Identified just once, at the edge of the network • Example FECs • Destination prefix • Longest-prefix match in forwarding table at entry point • Useful for conventional destination-based forwarding • Src/dest address, src/dest port, and protocol • Five-tuple match at entry point • Useful for fine-grain control over the traffic • Sent by a particular customer site • Incoming interface at entry point • Useful for virtual private networks

24. LSP (Label Switched Path) Setup • Protocols used to automatically establish LSPs: • LDP (Label Distribution Protocol – RFC 3036) • RSVP (with the objects needed to request/map labels) • Exploit IP routing tables information to establish paths and bind FECs to LSPs • CR-LDP (Constraint-based Routing LDP – RFC 3212) • RSVP-TE (TE extension to RSVP – RFC 3209) • Support establishment of explicit paths

25. Explicit routing • LSPs can be established along paths other than the shortest path • More flexibility than destination-based IP routing • In an MPLS network, it is (in theory) possible to implement the optimal solution to the general routing problem (multi-commodity flow problem)

26. TE with constraint-based routing • Explicit routing + fine-grained FEC definition • Allows to select a suitable path for each flow • We can select a path that meets the QoS requirements of the flow • Knowledge of network resource usage is required • Extend OSPF to disseminate the extra information (OSPF-TE – RFC 3630) • Maximum bandwidth, maximum reservable bandwidth, unreserved bandwidth, TE metric, etc. • The MPLS ingress LSR (Label Switching Router) computes the constraint-based path and establishes the corresponding explicit LSP

27. TE with constraint-based routing • Given the network topology and the flow requirements, how to select a path? • We assume that flow requirements are expressed in terms of requested bandwidth B between a source s and a destination d • The residual bandwidth on each link is R(l) • A feasible path is such that R(l)>B on all the links • If we prune all the links such that R(l)<B, all the paths between s and d in the pruned topology are feasible • What feasible path can we choose? • The goal is to optimize resource usage

28. Widest Shortest Path (WSP) • WSP selects the path with the minimum hop count among all feasible paths • In case several such path exist, the one the maximal residual bandwidth is selected • The residual bandwidth of a path is the minimum among the residual bandwidths of its links • R. Guerin, D. Williams, and A. Orda, “QoS routing mechanisms and OSPF extensions,” in Proc. IEEE Globecom, 1997

29. Other TE algorithms • Most of TE algorithms • prune links with insufficient bandwidth • assign a cost to each link in the pruned topology • select the least cost path • Different techniques to assign costs • The cost may be an increasing function of link load • Slightly loaded links are preferred • Heavily loaded links are discouraged

30. Minimum Interference Routing (MIRA) • Given the knowledge of ingress-egress pairs, MIRA selects the path which minimizes the interference • Interference on an ingress-egress pair (s,d) due to routing a flow between some other ingress-egress pair is defined as the decrease in the maximum network flow between s and d • The minimum interference path between a particular ingress-egress pair is the path which maximizes the minimum maxflow between all other ingress-egress pairs • This problem is NP-hard • Set link weights proportional to maxflow reduction • K. Kar, M. Kodialam, and T. Lakshman, “Minimum Interference Routing of Bandwidth Guaranteed Tunnels with MPLS Traffic Engineering Applications,” IEEE Journal on Selected Areas in Communications, vol. 18, no. 12, pp. 2566-2579, December 2000.

31. Simple Minimum Interference Routing (SMIRA) • The interference on an ingress-egress pair is evaluated by means of a k-shortest-path-like computation instead of a maxflow computation • The set of k paths is determined by first computing the widest-shortest path between s and d • Then, all the links along this path with a residual bandwidth equal to the bottleneck bandwidth of the path are pruned • The second path is the widest-shortest path in the pruned topology • This procedure is repeated until either k paths are found or no more paths are available • The cost of links belonging to the set of k paths is increased proportionally to the weight of the path and the ratio of bottleneck bandwidth to residual bandwidth. • I. Iliadis and D. Bauer, “A New Class of Online Minimum-Interference Routing Algorithms,” Networking 2002, LNCS 2345, pp. 959-971, 2002.