Introduction to SD-WAN

1. Introductionto SD-WAN

2. Why do we pay for services ? Generally good (and frequently much better than toll quality) voice service is available free of charge (e.g., Skype) So why does anyone pay for voice services ? Similarly, one can get free • (WiFi) Internet access • email boxes • search services • file storage and sharing • web hosting • software services So why pay for any service ?

3. Paying for QoS The simple answer is that one rarely pays for Best Effort service one pays for Quality of Service guarantees For example, for voice communications services Of course, one really would like to pay for Quality of Experience but that may be hard to measure QoS parameters proxy for QoE price toll quality with mobility QoS BE

4. SLAs and QoS parameters SLAs detail measurable network parameters that influence (correlate with) QoE, such as : Customer parameters customer service response time numerous legalities Connectivity parameters availability (e.g., the famous five nines) time to repair (e.g., the famous 50 ms) Noise (error) level parameters SNR, BER, etc. Packet Loss Ratio defect densities Information rate parameters bandwidth, throughput, goodput Information latency parameters 1-way delay round trip delay service parameters fault parameters performance parameters

5. Carrier-grade services It follows that there are two kinds of services: Best Effort services (e.g., Internet) Carrier-grade services (e.g., Carrier Ethernet or MPLS VPN) Best Effort services may be very inexpensive or free but come with very loose consumer-grade SLAs, e.g., • repair within 5 business days • no latency expectations • no PLR guarantees Carrier-grade services may be very expensive and SLA-dependent SLAs are often complex and may take weeks to negotiate This leads end-users to wonder whether they really need them

6. For example ... High availability services are usually 5-nines (99.999%) while BE services are often 3-nine (99.9%) or less High availability services have sub-50 millisecond repair times while BE services do not specify this time at all CE services provide dual token bucket bandwidth profiles Internet services deal with physical layer bit-rate Low delay services may guarantee delay 15% higher than physically possible while BE services may deliver latency double the physically possible Low PLR services may guarantee PLR under 0.01% while BE services may have PLR of 1% Tight SLAs can cost the end-user 10 times more than loose ones

7. The conventional approach It is clear that banks, brokers, financial services critical infrastructure operators web retail presence require high-availability It is equally clear that interactive applications require low delay enterprises require good security/traffic isolation Hence, enterprises tended to order SLA-based services for all of their networking needs This led to a situation wherein Internet browsing, email, other non-critical communications were priced outrageously high and performed outrageously poorly

8. The solution - SD-WAN SD-WAN is mostly a mechanism to optimize expenditures Traffic flows are separated according to application Flows are run over paths with the minimum required QoS thus minimizing expenditure Carrier Grade network 1 high priority SD-WAN controller or may be two different TE paths in a single network HQ SD-WAN end-point low priority branch SD-WAN end-point Carrier Grade network 2

9. Hybrid SD-WAN Application flows not requiring any special QoS treatment may be run over low-cost Internet connections This is often called hybrid SD-WAN Of course the Internet flows must be monitored and encrypted Carrier Grade network highest priority lower priority lowest priority public Internet

10. Pure Internet SD-WAN It may even be possible to provide SD-WAN services entirely over Internet connections by employing two disjoint ISPs and duplicating traffic for high availability ISP 1 Carrier Grade network SD-WAN controller branch SD-WAN end-point HQ SD-WAN end-point ISP 2

11. Some SD-WAN use cases The first defined use case is application-based QoS this is similar to standard CoS-based services but enables CoS mapping based on Deep Packet Inspection and explicitly takes expense into account Another common use case is bandwidth augmentation that is, adding bandwidth to an assured service without major increase in expense by separating/offloading low priority traffic Another use case is rapid site commissioning where a new site can be rapidly added to an existing VPN by initially connecting over Internet until the SLA-guaranteed connectivity is available

12. Rapid site commissioning HQ on-net branch sites Carrier Grade network on-net traffic off-net traffic off-net branch public Internet

13. SD(-WA)N What is the connection between SD-WAN and SDN ? SDN applies IT technologies to routing inside a DC SD-WAN advocates extending SDN technologies to the enterprise WAN SD-WAN can be considered SDN at the application layer Even SDN-based enterprise datacenters are interconnected today via static MPLS VPNs Why ? because that is what service providers are offering (until now) because enterprises are used to assured services, rather than build-to-fail because wide area interconnections are not dynamic (except during failures) SD-WAN can be considered as extending internal SDN to the outside world

14. What is needed for SD-WAN? What is required to implement SD-WAN? A branch SD-WAN end-point, capable of identifying applications and mapping them to flows (DPI) tunneling (at least for Internet paths) monitoring QoS level (at least of non-SLA-guaranteed paths) encrypting/decrypting packets (at least for Internet paths) For branch-to-HQ cases: a central (HQ) SD-WAN end-point, for reconstructing flows (including tunneling) monitoring QoS level (at least of non-SLA-guaranteed paths) encrypting/decrypting packets (at least for Internet paths) gateway functionality to external networks (e.g., Internet) A central SD-WAN controller, responsible for finding paths or networks with desired QoS levels instructing SD-WAN end-points of the required mappings fault event handling

15. Some carrier SD-WAN vendors Cisco Viptela – Verizon, NTTPC, Singtel, PCTL, ... Cisco IWAN – Telstra, Vodafone, BT Nuage – BT, China Mobile, Telia-Sonera, Telfonica, Telus, Charter, ... Silver Peak – Masergy, China Telecom, NTT Riverbed – Orange, Verizon TELoIP – Access One, X10, California Telecom, Freewire VMware VeloCloud – AT&T, Sprint, Telstra, Vonage, ... Versa – CenturyLInk, Tata, Comcast, Colt, China Telecom Global, ... • Some more SD-WAN HW/SW vendors: • NTT Virtela • Barracuda Networks • CloudGenix • FatPipe • Mushroom • Sonus Networks • Oracle Talari Networks • Some more SD-WAN SPs: • Aryaka Global SD-WAN network • BigLeaf • Cato Networks • Citrix NetScaler and there are many more DIY enterprise deployments!

16. SD-WANaccording to ONUGandMEF

17. Two types of SD-WAN Who builds the SD-WAN? There are two cases Enterprise SD-WAN (called DIY SD-WAN by SPs) • end-user deploys/manages equipment and controls • end-user contracts/pays for standard (SD-WAN-agnostic) point-to-point, VPN, or Internet service for underlay connectivity Service Provider SD-WAN • service provider markets new SD-WAN service offering • offering cannibalizes traditional MPLS/CE VPN service but avoids losing customer altogether • SP deploys/manages SD-WAN end-points and controller

18. Enterprise SD-WAN The original impetus for SD-WAN came from IT professionals They observed that corporate WANs, consisting of branch-to-branch branch to HQ branch-to-DC connectivity, suffered from high pricing and complexity slow provisioning cycles (not dynamic, ) poor IT visibility (no dashboard) limited security features (FIPS certification, event logging, etc.) inefficient bandwidth management (including asymmetry) impossibility of bandwidth pooling from different SPs In 2014 ONUG proposed building over the Internet an alternative to traditional SP VPN services

19. How can enterprise IT do it? Instead, or in addition to, today’s static and expensive L3VPN L2VPN IPsec-VPN based services An enterprise rents connectivity between its sites based on inexpensive large bandwidth non-assured links and build an assured service by combining these links Since the enterprise only sees end-to-end link attributes it must continuously monitor the QoS status of these links Since these links are not under its own control the enterprise must provide its own end-to-end encryption

20. ONUG The Open Network User Group (ONUG) was created in 2012 by Nick Lippis (IT analyst and advisor) and IT professionals, mostly from the financial sector in order to organize conferences to share IT best practices The first ONUG event was held in February 2013 conferences are semiannual, and have expanded to Europe ONUG’s board consists of IT executives, mostly from banks ONUG forms working groups to define use case requirements OSE Open SD-WAN Exchange M&A Monitoring and Analytics S-DSS Software Defined Security Services HMC Hybrid Multi/Cloud CN Container Networking

21. ONUG’s Open Networking ONUG defines Open networking as interoperable software and/or hardware that provides choice and design options to enterprise IT This is achieved by decoupling of specialized network hardware and software open industry standards (consortium, community or SDO) The goals are mitigating vendor lock-in significant reduction of the total cost of ownership (especially OPEX)

22. ONUG SD-WAN In 2014 ONUG published its SD-WAN Use Casewhitepaper This whitepaper provides requirements to guide enterprises in implementing SD-WAN by in-house teams although it acknowledges SD-WAN services by managed SPs The whitepaper surveys WAN architecture models and details requirements for scalability efficiency reliability seamless integration security manageability

23. Service Provider SD-WAN Service providers recognized Enterprise (DIY) SD-WAN as an existential threat to be countered even at the risk of cannibalizing their most profitable business SPs realized that DIY SD-WAN suffers from disadvantages end-users do not have network-wide visibility and thus can not efficiently choose paths enterprise IT are not knowledgeable or capable of handling complex networking enterprise management desires certain key services to be guaranteed by a recognized SP connectivity to public DCs can’t be handled in-house and decided to offer managed SD-WAN services

24. MEF SD-WAN The MEF forum has detailed its view of SD-WAN Understanding SD-WAN Managed Services • introduces terminology for SD-WAN managed service components • illustrates how they fit into the LSO Reference Architecture (RA) • specifies the required LSO reference architecture interfaces • explains how SD-WAN facilitates multi-vendor interoperability • provides example SD-WAN managed service use cases SD-WAN Service Attributes and Definition (2019Q1 ?) • industry-neutral SD-WAN service definition • accelerates adoption/certification of MEF 3.0 compliant SD-WAN • orchestration across global ecosystem of automated networks and has carried out 3 PoC demonstrations Zero Touch Services with Secure SD-WAN Multi-Vendor LSO Orchestrated SD-WAN with container-based uCPEs SD-WAN over virtualized/orchestrated wholesale CE Access Service

25. Relationship to MEF-3.0 and LSO In MEF’s view, SD-WAN is a good match for LSO

26. MEF LSO architecture

27. Required SD-WAN characteristics MEF views the following characteristics as mandatory for SD-WAN: Secure, IP-based Virtual Overlay Network typically IPsec-based, usually NAT and firewall Transport-independence of Underlay Network including MPLS (or CE) over fiber, broadband Internet, TDM, LTE Service Assurance of each SD-WAN Tunnel QoS KPIs are measured in real-time for each tunnel Application-Driven Packet Forwarding typically based on DPI High Availability through Multiple WANs but not necessarily hybrid (multiple WAN technologies) WAN Policy-based Packet Forwarding packets are placed in tunnels based on QoS policy and security policy Service Automation via Centralized Management, Control and Orchestration ZTP of CPE, visibility, user access via web portal and/or APIs (optionally) WAN Optimization data dedup/compression/caching, 1+1 to eliminate packet loss

28. SD-WANviamultiple InternetconnectionsTechnical Appendix

29. Reminder: carrier grade services MEF defines carrier-grade based on several QoS KPIs availability specified in “nines” notation (e.g., five nines) bandwidth profile (committed data-rate) specified by CIR, CBS, EIR, EBS, … packet loss fraction of frames to be delivered that are actually delivered specified by T (time interval) and L (loss objective) packet delay measured UNI-N to UNI-N on delivered frames specified by T, P (percentage) and D (delay objective) packet delay variation specified by T, P, D, V (PDV objective)

30. Multiple Internet connections Some QoS KPIs can be statistically guaranteed using multiple Internet connections in 1+1 fashion Using different ISPs should provide disjoint resources and thus improve many MEF-defined QoS KPIs ISP 1 Carrier Grade network ISP 2

31. Availability Availability is the easiest advantage to understand – combining truly disjoint Internet connections leads to increased availability What is the quantitative relationship ? Theorem:Nines add When we say that an end-to-end connection is N nines we mean that it is available 99.99... % of the time i.e., probability of being unavailable at any given time is 10-N

32. Availability (cont.) For disjoint networks, these probabilities are independent and thus the probabilities multiply Since nines are logarithmic in probabilities, nines add For two disjoint Internet connections with the same nines rating We can readily extend the rule nines add for : 2 Internet connections with different availability ratings >2 parallel connections

33. Bandwidth profile MEF has an intricate definition for bandwidth profile which is mainly interesting when data-rate is limited Internet connections will provide much higher rates for the same cost Theorem:data-rate minimizes For 1+1 over 2 Internet connections with identical data-rate R the data-rate equals R For 1+1 over 2 Internet connections with data-rates R1 and R2 the data-rate equals min(R1,R2)

34. Bandwidth profile (cont.) If higher data-rates are required we can combined 1+1 with inverse mux, such as ECMP For example, using 4 Internet connections in 2+2 configuration Alternatively, we can use N Internet connections with Shamir Secret Sharing to obtain k<N times the data-rate of each connection Carr ISP 1 ISP 2 ISP 3 ISP 4 Carrier Grade network

35. Packet loss For independent connections, each with given Packet Loss Ratio Theorem:PLRs multiply Given 2 IID Internet connections with PLR = P1 and P2 If we send a packet over both connections due to the independence assumption the probability of both losing a particular packet is P1 *P2 Thus, if we have 2 Internet connections with PLR = 1% (high!) by using 1+1 the resulting PLR will be 0.0001 This will easily provide the required super-low PLRs

36. Packet delay Real-time interactive MEF services (for example, voice/video conferencing) require bounded delay When combining Internet connections with delays D1and D2 one might conclude that we get min(D1,D2) since we use the packet that arrives first Unfortunately, this is not completely true! When a packet is lost on the faster connection, we use the packet which arrives later So unless PLR=0 for the fast connection the worst-case delay is max(R1,R2) MEF criteria penalize for tardy packets so the precise delay is in between the min and max and depends on the PLRs

37. Packet delay (cont.) The precise formula is not as important as the following argument MEF services with tight delay requirements have objectives close to the physical time-of-flight (which is 2/3 the speed of light times the fiber length) Expedited packets are prioritized at switches and thus experience minimum switching latency Time of flight in a city is sub-millisecond US coast-coast is < 40 ms half-way around the world at the equator is 100 ms

38. Packet delay (cont.) Expedited packets are prioritized at switches and thus experience minimum switching latency What do we mean by minimal ? The worst-case transit delay for an expedited packet equals the time for a maximal-size packet (1500B) already being output to exit Even with dozens of switches, these added latencies are negligible! Internet packets need to wait in queues for processing and this time can be very significant and is unpredictable For Internet connections latencies of twice time of flight are considered good!

39. Packet delay variation Packet delay variation is interesting only for a very limited number of applications such as timing transfer and teleprotection signaling Using 1+1 and accepting the first packet to arrive reduces PDV as compared to either connection separately although the precise formula depends on the distribution Theorem:V < min(V1,V2) This should usually yield acceptable PDV

40. Summary By combining 2 or more Internet connections one can statistically provide a MEF-like service from the points of view of availability bandwidth profile packet loss packet delay variation (typically) But, there are no guarantees regarding delay objectives Since bounded-delay services are important and prevalent SD-WAN over pure Internet connections can not fully replace MEF services