SAIN Networking

1. SAIN Networking Ray W Sanders Chairman, SAIN Networks, Inc. overcoming unintended consequences in voice and data networks SAIN = Synchronized Adaptive INfrastructure

2. What this talk is about • A simple paradigm that can overcome the unintended consequences of today’s stochastic data network. • The paradigm results in a simpleunderlayer that can ensure a deterministic data network.

3. Sanders Prediction • Packets will be forever, but the global Internet will morph into something that looks a little like a late 1970’s telephone network but with far more capability and without the • fatal flaws of carrying only • connections that must last • for at least a few seconds • and support only voice • conversations

4. Comparing SAIN with existing networks Existing Networks SAIN Networks Route one-connection-at-a time Hop-by-hop routing for each connection Network uses multiple control planes Networks are largely stochastic Wire speed latency can be 100 ns Internet uses many overlay protocols Data sent in bursts Complicated Quality of Service required Head-of-line blocking complications Tough privacy and security problems Bursty data can require overprovisioning Many networking protocols must exist Route aggregations of connections Route aggregations for a one-hop channel Network uses a single control plane Networks are deterministic Latency inversely proportional to data rate NICs make use of single purpose utility Bursts forwarded or smoothed out Guaranteed delivery—one metric: delay Small packet wins by increasing data rate Disjoint objects conceal cellet relevance Aggregated streams require less BW 2 simple algorithms manage BW and routes

5. A thought experiment • Assumptions: • 1,000 people want to see 1,000 two-hour movies starting at 8 p.m. • Each movie contains 9 gigabytes of data • The network can use up to 10 Gbps to deliver a collection of movies • Suppose we use 10 Gbps for each movie • It takes 7.2 seconds to send one movie • How long does it take to send all 1,000 movies one after another? • 1,000 × 7.2 seconds= 2 hours • How long would the average customer need to wait to start seeing his movie? • 1 hour • Now, suppose that we send each movie at 10 Mbps (1/1000th of 10 Gbps) • How long does each of 1,000 customers wait to start watching his movie? • 0 hours • There is a compelling requirement to control bandwidth, (and hence delivery time) to meet each customer’s need • This result can obtain if a network is deterministic

6. Goals for a SAIN network • Define and build elemental pieces of a network architecture that: • can support all existing voice and data network traffic • can support unknown future traffic types • can grow from data centers, to metropolitan networks, to a global interconnected network • is robust, efficient, andsimple • is a circuit-based architecture that can endure and scale for decades

7. Constraining networks to really improve their efficiency • A core principle of the SAIN architecture • Partition a network into small disjointpairs of active objects such as pairs of NICs and pairs of switches • What does this do? • Enhances a network’s privacy and security • Prevents one object in a network from changing the state of another without using a Control Vector to send messages from a source object to a destination object • Prevents any entity outside a network from changing the state of an object inside the network • Simplifies object addressing Basic Aggregation / Disaggregation Switch Pairs

8. Constraining networks to really improve their efficiency • Another core principle of a SAIN architecture • Nodes in a SAIN network are synchronized to a common clock • What does this do? • Enables very cheap high-performance switches that can scale well beyond current limits • Removes the need for complex Quality of Service facilities inside a network

9. Constraining networks to really improve their efficiency • A third core principle of the SAIN architecture • All user data protocolsare separated fromdata transportand its control • What does this do? • Defines an underlay network whose only job is to transfer bits from a data source to a data sink • Enables Network Interface Controllers (NICs) to support devices with any protocol. • Demands that an Egress NIC’s protocols must match its paired Ingress NIC’s protocols • Lets a NIC match from only one other NIC to a large number of NICs in a network

10. Constraining networks to really improve their efficiency • A fourth core principle of the SAIN architecture • Build a lot of a network’s physical and logical connectivity a priorito its use • What does this do? • Enables each port of a network to have a physical connection to every other port of the network with a matching NIC; the connections are set up when the network is built or modified • Enables every possible route to be computed when the network is built and need not be recomputed until new nodes are added to the network

11. Constraining networks to really improve their efficiency • A fifth core principle of the SAIN architecture • All connections are ‘virtual’ that consume network bandwidth only when there are data bits to transport • What does this do? • Enables each connection to be set up prior to use • Assures that no bandwidth is used until data is to be sent • Assures that an amount of bandwidth allocated to a connection is just enough to meet a customer’s needs

12. Constraining networks to really improve their efficiency • A sixth core principle of the SAIN architecture • All connections from a source node to a destination node are aggregated into a single logical data flow • What does this do? • Significantly reduces the number of objects to be routed through a network • Packets do not get routed independently ; they are combined into aggregations sent from a source to a destination node through preset routes • No computing needed at each tandem node • A route is a virtual connection between two nodes; if it approaches congestion, another route can be quickly added

13. Constraining networks to really improve their efficiency • A seventh core principle of the SAIN architecture • The amount of available bandwidth and delay must be known for each possible route through a networkbefore a connection is made • What does this do? • Prevents discarding packets because of network congestion • Dynamically provides the most cost-effective route with bandwidth to meet each a connection’s need • How can this be accomplished? • Delay over a route is known when nodes are installed • Each node connected to a transport connection (trunk) sends the trunk’s bandwidth availability to each source node in the network periodically (e.g. 1,000 times per second)

14. What are data networking’s unintended consequences? • Some examples of unintended consequences in today’s networks • Traffic congestion and discarded packets • Jitter (= delay variation); traffic shaping and policing • Overprovisioning and Quality of Service • Flow-based traffic and circuit emulation • Lack of privacy, security and survivability

15. Network Behavior Constraint 1 Eliminate Traffic Congestion • Packets and packet buffers are not going away in a SAIN network • For each end-to-end connection there is a packet buffer at its ingress node and one at its egress node • Each connection that occurs at a source-node/destination-node pair within a given period (an ‘epoch’ for a group of connections) originates within a pair of switches Basic Source Aggregation / Destination Disaggregation Switch Pairs Interconnecting Elements include source/destination node switches in three aggregation tiers above the lowest tier The lowest tier aggregates customer data; the higher tiers forward aggregations Each higher tier aggregates the next lower tier’s data

16. Network Behavior Constraint 2 • Eliminate jitter, traffic shaping and policing Jitter (also known as delay variation) is the aperiodic arrival of each packet. Aperiodic arrivals of packets in data flows can cause service disruptions Changing bandwidth of a connection can assure that either the start time of a received packet or the time required to receive an entire packet provides uninterrupted service SAIN network synchronization provides ‘traffic shaping’ and ‘policing’ without additional complexity

17. Network Behavior Constraint 3 Reduce overprovisioining • Aggregating connections into channels can benefit from the Law of Large Numbers • The law can result in the bandwidth of a large aggregation changing slowly compared to faster bandwidth changes of the lowest tier • Node synchronization can result in a network not needing Quality of Service as currently defined • A desirable metric is end-to-end delay of entire packets—not wire speed starting time of sending a single packet

18. Network Behavior Constraint 4 Flow-based traffic without Circuit Emulation • Nodal clocks can provide physical circuits in a simple mannercompared to the current complexity of circuit emulation • The physical circuits operate at all levels of aggregation and can be virtual or real • The necessity of providing circuits for flow-based traffic is a major reason to implement the SAIN architecture • In addition to basic algorithms, a third ‘floating frame’ algorithm exists for plesiochronous operation where span lengths of trunks vary (e.g., for moving nodes and environment variations)

19. Network Behavior Constraint 5 Provide better privacy, security and survivability • Overcome current core network privacy and security weaknesses • A SAIN network can assure that all network objects used to forward packet data through the network are disjoint. • Network data forwarding control can be massively distributed with centralized monitoring and fault management • A network object cannot change the state of another object except by using a certified Control Vector connected from a source node to a destination node • A destination node can authenticate certification of a connected Control Vector. • Certification can use round-trip delay of destination and source nodes • Bandwidth management algorithm results in ever-changing aggregation frames • ‘Floating frames’ enhance security

20. More Network Behavior Architecture scales beyond current limits • Instead of forwarding entire packets a SAIN network forward only one or a few bits of a packet at a time • This results in using very simple switches that forward large aggregations without requiring expensive large routers • Not only are costs reduced;energy needs are reduced as well

21. More Network Behavior A single metric defining application needs • There is no need for traffic shaping or policing;there is no need for circuit emulation;there are no out-of-order packets; and the packet loss rate is zero Synchronized network nodes and implicit addressing achieves this goal • Node synchronization can result in a single metric that defines required delays for application types • The single metric defines end-to-end delay of entire packets—not just the wire speed starting time of sending a single packet

22. More Network Behavior Results from simulations of a model network The green circles are transit nodes (T-Nodes) The red rectangles are entry/exit nodes [E-Nodes] Each [E-Node] (T-Node) contains source switches connecting to paired destination switches in all other [E-Nodes] (T-Nodes) in a network A Metropolitan Area Network Example with 20 T-Nodes & 80 Simplex Trunks 500 E-Nodes each able to support >4,000 ports each with multiple IP addresses

23. Sanders Suggestion • We should not let ourselves make another management mistake that the future of networking will be based entirely on using packet switches for routing • Our focus should morph into efforts that enhance IP* addressing and DNS*in a circuit-based world with advanced NIC applications * Internet Protocol addressing and * Domain Name System

24. Thank you If you like what we are doing…we need pacesetting partners

25. What follows are a few more slides you might find interesting

26. How to support goals 1 & 2 (Support existing and future traffic types) • Transport of bits is independent of data type • Packets appear only at ingress and egress ports with connected NICs • Packet or circuit data appears at an ingress NIC and is transferred to an egress NIC • An ingress/egress pair of NICs can support any matched data type • NIC pairs can support secure topologies and methods • Packets are transferred bit-by-bit at a deterministic data rate • An Egress NIC delivers the protocol entering its paired Ingress NIC

27. How SAIN works #1 • What a packet flow can look like: This method of multiplexing uses ‘explicit addressing’ What a SAIN flow can look like: The size of each cellet is fixed for a given link in which a frame occurs The duration of an Epoch can depend on the desired end-to-end network delay of all embedded packets This method of multiplexing uses ‘implicit addressing’ where the position of each cellet defines its connection or channel identity

28. How SAIN works #2 • A SAIN network containssimple network switches with a very different approach that uses very simple parts • The ‘Interconnecting Elements’ are primarily made up of Aggregation Switch / Disaggregation Switch pairs that exist in three levels of aggregation • Each tier contains Aggregation / Disaggregation Switch Pairs • The three aggregation levels pass data use three network tiers plus an exchange tier to other networks and a virtual distribution sub-tier shown in the next slide Basic Aggregation / Disaggregation Switch Pairs

29. How SAIN works #3 • Connections exist in an Entry/Exit E-Node tier that includes a virtual VE-Node subnetwork uses for traffic distribution • Each E-Node connects large aggregations of connections within large channels to and from a parent Transit T-Node tier Each T-Node routes the aggregations of E-Node traffic for delivery from a SourceT-Node to a Destination T-Node In addition to its T-Node tierrouting functionality, a T-Node can connect to an eXchangeX-Node that can have a channel to other X-Node domains including those that make up a global domain

30. More Network Behavior Routing in the model network • Each E-Node connects to a parent T-Node • Each T-Node has full period connections to every other T-Node • Each Source T-Node can set up a loop-less route through T-Nodes to every other T-Node • Each route can be computed at network instantiation • The computation begins with a table of single hops among the T-Nodes • A second hop for each entry can be added for each second hop that does not include the first hop • Repeat this process recursively for a two-hop table to build a three-hop table and continue for tables with more hops • The process results in finding all routes that do not contain loops • A 10-hop table has over 500,000 entries for all source to destination routes in a 20 T-Node model network • The average number of routes for each of the 380 paired connections is about 1300

31. Today’s traffic is mostly flow-based, not bursty • Today’s networks are based on early 1970’s needs: using minicomputers to • send messages and transfer files • Queuing theory provided solutions for an asynchronous stochastic world • Today’s needs are circuit-based tosatisfy a burgeoning market for flow-based traffic • What is needed now is a network with synchronized nodes that support dynamic data rate connections

32. Another experiment • Assumptions: • A financial trading firm wants to minimize its network delay • The smallest Ethernet frame is 84 bytes including a 46-byte payload • A SAIN network frame can have 5 bytes plus the 46-byte payload • In either case, a 1 Gb/s channel is carrying the data • A SAIN 408-bit (51-byte) packet could be guaranteed delivery in one microsecond or less • This compares to 672-bit (84-byte) Ethernet needing nearly one microsecond if there is no other traffic using the channel. Its delay is not guaranteed. • There is a compelling requirement to control bandwidth, (and hence delivery time) to meet a customer’s need

33. A brief look at a basic principle that really matters Delay vs. Data Rate An 8 x 8 orders of magnitude look at a key fundamental of data networking 33

34. Can we cover the earth with a SAIN network? The earth’s land mass area totals ~148,940,000 sq km. The area of each square within a 2 millisecond radius circle is ~320,000 sq km. The number of supermetro networks needed to cover the land mass: 466 In the real world, sizes will likely be based on number of users and/or number of ports and market to determine a diameter