Large Scale IP Networks

1. Large Scale IP Networks GMPLS and MPLS Examined Vijay Gill <vgill@mfnx.net>

2. Agenda • Background • What is the problem • Solutions - (G)MPLS • Issues with the solutions above • An alternative proposal • Questions

3. Acronyms • PPS: Packets Per Second • ER/TE: Explicit Routing/Traffic Engineering • FEC: Forwarding Equivalence Class • CSPF: Constrained Shortest Path First • GMPLS: Generalized Multi-Protocol Label Switching • IGP: Interior Gateway Protocol (OSPF/IS-IS/RIP) • LDP: Label Distribution Protocol • SPF: Shortest Path First

4. Guide For Talk • Optimize On • Getting 95% of the problem with 15% effort • Flexibility • Operations And Engineering Guy • Expertise in building systems, networks, and organizations that run IP networks • Seen the results of the meeting between “Networks Powered by PowerPoint ™” and the Real World ™ • Hint: The Real World ™ wins every time

5. “I dislike rigidity. Rigidity means a dead hand and flexibility means a living hand. One must understand this fully.” - Miyamoto Musashi

6. Ordinal Vs. Cardinal Optimization* • More important to quickly narrow the search for an optimal solution to a “good enough” subset than to calculate the “perfect solution” • Ordinal (which is better) before Cardinal (value of optimum) • Ballpark estimate • Historical Internet Vs the Telco approach we don't need to boil the ocean - all we want is a poached fish *Based on work done by Yu-Chi Ho

7. Soften Requirements • Softening strict requirement of optimality can make problems tractable Getting the best decision for certain Cost = $1m Getting a decision within the top 5% With probability = 0.99 Cost = $1m/x In real life, we often settle for such a tradeoff with x=100 to 10,000

8. MPLS • M is for Multiprotocol (inside and out) • But despite being able to carry “anything” inside, IP is the single most common payload • IP routers are the most common “outside” • Nameable aggregates of traffic have value • Explicit Routing • Comes with a price • Hype! QoS! Sings, dances, julienne fries! • One potato, to go.

9. What Is The Problem • Dense Network • Protect Paths • Routers out of PPS • Solved by • Constrained Meshed Routing • Mindset Changed

13. Problems Solved • MPLS solved ER/TE problems • RSVP-TE is extensible to ask for particular qualities of service etc. rather than just raw BW • Got perverted by the vendor marketing folks • Try to do everything under the sun • QoS!

14. The Myth of QoS • FECs can be described which ask for particular queuing disciplines inside switches and routers (via the RSVP-TE mechanism) is very popular with some people • Fancy queuing and careful resource management can in theory approach the lack of jitter that TDM provides • Never overbook the jitter-free traffic • Jitter-free traffic squeezes out the more elastic traffic • Belief is that the combination of the control plane and the label-packet format can fully replace traditional TDM • At the cost of some complexity and deploying "new stuff"

15. QoS • Tough thing to define • Tougher to sell • Better make sure Best Effort Internet services work • All Gold, All the Time. • Differentiation must be palpable to the end user • Cost must not be prohibitive • Should not be hard to manage • Integrated with the best effort network • Also keep up with best effort deployment • QoS == Quantity of Service • What Are We Optimizing For?

16. These exhibits were originally published in Peter Ewens, Simon Landless, and Stagg Newman, "Showing some backbone," The McKinsey Quarterly, 2001 Number 1, and can be found on the publication's Web site, www.mckinseyquarterly.com. Used by permission.

17. Best Effort Is Good Enough • Statistical multiplexing saves money • Mixing various queuing disciplines into a statistically multiplexed network is • Complicated • Costly • Full of side effects • Overprovision for now • Less "full" at peak traffic point: less efficient • But, no queue means no need for queuing disciplines • Small risk of jitter/delay for the sake of less complexity vs. much more complexity

18. Cheaper Faster Better • Internet enabled applications will squeeze out (eventually) applications that aren’t. • The number of mobile phone subscribers worldwide is expected to reach 560 million by year-end and to exceed the number of households with televisions by 2003. -Will Daugherty (McKinsey & Company)

19. GMPLS • The RSVP-TE label mechanism is generalized in GMPLS to request resources of any nature, notably lambdas, SDH MUXes and "patch-panel" mappings • GMPLS is a CONTROL PLANE not a packet system: there is no requirement that MPLS "frames" be used in an GMPLS network

20. GMPLS • No centralized provisioning database • Available resources are consumed where the CSPF reservation is allowed • IGP does topology discovery (OSPF) detects faults and allows restart of reservations • OSPF LSP database is also consulted to find the the CSPF, which will be requested (by RSVP or LDP to all the elements along the path) first.

21. Unified Control • The GMPLS argument is that one control and packet system can be used to knit together tremendously different network components • IP Routers • Switching gear • Including ATM, SDH and WDM "switches"

22. Router Router ATM switch ATM switch SONET ADM XC XC SONET ADM ~ ~ ~ ~ ~ ~ GMPLS Flexibility Points • UNI • RSVP-TE or LDP based • Routers request concatenation of resources through the network Control Control Control Control DWDM DWDM DWDM Signalling MPlS Control Plane

23. Benefits Of GMPLS • Meshy Restoral • Clients of all kinds (routers, TDM boxes) • Saves on router ports • Routers make expensive OEO • Mitigation: cost is amortized over lifetime of box • Flattened topology

24. Benefits of GMPLS • Signaling between routers and optical switches • Self provisioning • Faster Provisioning

25. Issues • Best Abstraction Of A Topology Is The Topology • Spend money on packet-handling rather than managing lots of meshed mid-sized boxes • “We have too many boxes now. We’re not going to have a million more boxes in the network. That scenario is utterly unthinkable” -Mike O’Dell

26. Reexamining Optical Network Assumptions • Replacing patch cords with OXCs doesn’t affect the network much • OXCs et al. allow you to redeploy the topology • Real world topology doesn’t change very fast • Extend planning horizon • City-pair macroflows are long lived and tractable • Cost and complexity of running an IGP over the optical boxes to gain speed of restoral over a centralized system needs to be examined carefully

27. Thoughts • Our Control Theory-Fu is weak • Get provisioning from 18 months to a day or two • We don't know anything we could do with 50ms provisioning without making a disaster • Centralize view of topology and lay out paths using expert systems vs. SPF in the network

28. Self Provisioning Issues • Internet is an intentionally overdamped system • The consequences of being underdamped are catastrophic • Got the T-shirt • Frame Relay wars • Improving the frequency response of the implementation implies lots more T-shirts

29. Optimize For The Biggest Consumer • Design Goals Are To Replace • Back-to-back OEO in middle of nowhere • Unnecessary OEO for passthrough • Slow Humans

30. Packet Photons Packet SONET SONET Cross Connect SONET Muxes SONET Muxes Cross Connect DWDM DWDM Routers Routers Multiple levels in Layer 1 Typically

31. Typical Hut ODF ADM Flexibility Points: Add or drop traffic to the network

32. How To • Use strong enough lasers • Avoid turning “pass-through” frequencies into electrons • Attenuation hit (that’s what OEO is for) • Divert frequency bands onto dark or transponders which do frequency conversion

33. How To • Integrate the MUX within the control plane of a large router • Tell router not to use a certain frequency band for p2p traffic with its neighbor any more because it has to be dropped out an optical port. • That port is dark fiber terminating • A small WDM MUX (8 colors) • End piece of equipment @ 2.5GHz, 10GHz, etc.

34. Proposal • Optical ADM emits light as necessary by intercepting one frequency & converting it electrically • The ADM becomes the source of the bits OEO+OADM ADM

35. How To • The router doesn't look at the signal • Doesn’t do • Regeneration • Look for SONET/SDH signaling • Passes through the frequency • Unfortunate attenuation hit, but that's what OEO deals with).

36. How To • Any space not "reserved" is used in whatever way seems optimal for big-router-to-big-router connectivity, for moving packets. • Use some of the spectrum to build a “sub-ring” or smaller p2p circuits for talking to smaller routers in flexibility points along the way, if any • Or use separate fiber, if fiber-rich or for retaining a historical system in parallel • Building a “virtual dark fiber” across this is possible, but you need to do your own regen (OEO), cross-connection, etc.

37. This Solves For • Optimizing the transmission resources for the largest consumer of optical bitstream – IP • Saves money on 1310/1550 lasers • Power • SG&A

38. Save The Hype “You cannot combat glossy magazines with logic” -Jeff Aitken “Somehow “best effort” has become a pejorative.” -Mike O’Dell

39. Conclusions • Even the very wise cannot see all ends • Lets not paint ourselves into corners • Stupid is flexible • Modularity • Theory of Real Options • End2end arguments in system design • Trade upfront CAPEX for long term OPEX • Rise of the Stupid Network • Assumptions still undergoing work

40. References • GMPLS: http://search.ietf.org/internet-drafts/draft-ietf-ccamp-gmpls-architecture-00.txt • MPLS: http://www.rfc-editor.org/rfc/rfc3031.txt

41. Questions Thanks to Mike O’Dell, Sean Doran, and Bill Barns