1 / 25

The Netflix Open Connect Network

The Netflix Open Connect Network. DKNOG – 21 March 2013. Today we are in 22 locations across the world... (16 Primary, 6 “network extensions”). How do we decide what locations to go to?. Where is Netflix presently providing service?

lucus
Download Presentation

The Netflix Open Connect Network

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. The Netflix Open Connect Network DKNOG – 21 March 2013

  2. Today we are in 22 locations across the world...(16 Primary, 6 “network extensions”)

  3. How do we decide what locations to go to? • Where is Netflix presently providing service? • Our service is limited by the content licenses that we can buy • For example, we are in the Nordic countries, the UK, and Ireland but not France/Germany • Can we interconnect with networks that are relevant? • Going to a peering point that’s all content providers does no good • Can we reduce the cost and improve traffic management for networks who need to deliver Netflix traffic to their end users?

  4. Netflix in the Nordics • We provide our streaming service to • Denmark • Finland • Norway • Sweden • Iceland and Greenland • Currently co-located at Telecity Stockholm 1 & 2 • Stockholm 1 is a network extension from Stockholm 2 due to space/power constraints • Likely to expand to a second market in the Nordics this year

  5. IXes • Netflix delivers their Nordic traffic from four different IXes • NetNod • NIX • AMS-IX • LINX • The latter two may not seem relevant here, but many networks have significant capacity built there

  6. Standard build - European • A European Netflix Open Connect IX build has: • 20 Netflix Open Connect Appliances • Two Juniper MX480 routers • OpenGear console server and Juniper EX2200’s for OOB access • Prewired MTP to LC cabling from Elpeus • A “network extension” Open Connect build has: • Two Juniper EX4550 switches • Two MRV 40 channel passive muxes • OpenGear console server for OOB access

  7. Stockholm

  8. Network Extension POP

  9. Standard Build - US • A US Netflix Open Connect IX build has: • 40 Netflix Open Connect Appliances • 60 Netflix Open Connect Flash Appliances • Two Juniper MX960 routers • OpenGear console server and Juniper EX2200’s for OOB access • Prewired MTP to LC cabling from Elpeus • A “network extension” Open Connect build has: • Two Juniper EX4550 switches • Two MRV 40 channel passive muxes • OpenGear console server for OOB access

  10. Converting some POPs to Cisco ASR • Due to concerns with Juniper code delivery and the need for higher density platforms, we decided in early 2013 to integrate some Cisco ASR 9K into our network to replace Juniper MX • In order to deliver our traffic goals (~1T per router), we need denser cards than the currently shipping Juniper 16 port MPC3’s

  11. High Density… • Juniper MX960 w/ MPC3’s can deliver 176 ports per chassis • A single fabric failure brings the chassis to 3:2 oversubscription • Juniper MX960 w/ MPC4’s can deliver 336 ports per chassis (oversubscribed) • A single fabric failure (out of 3) brings the chassis to 2:1 oversubscription – unacceptable • Cisco ASR9010 w/ Typhoon-based 36 port cards can deliver 288 ports per chassis • A single fabric failure brings the chassis to 3:2 oversubscription

  12. High Density, continued… • We conservatively need 130 host ports per router • Those 130 ports can deliver 1.13T of traffic • Assuming we deliver 1 bit outbound for every bit inbound from a host, that puts us at 1.1:1 oversubscribed at peak • 3:2 and 2:1 just do not work for us • Juniper MPC4’s also require brand new 12.3 code • What version of code are you running?

  13. Cabling a new ASR…

  14. The Netflix Open Connect Appliance (OCA) • Developed in response to ISP requests to help scale Netflix traffic efficiently • Reduces ISP cost by serving Netflix traffic from the local ISP datacenter, CO or headend, rather than upstream network interconnects • Speeds up internet access for consumers to all third-party internet sites, because Netflix traffic is no longer a source of middle-mile or backbone congestion • Netflix bears the capital and maintenance costs, not ISP • ISP provides space, power and a network port • An OCA is a component of the Netflix CDN (vs a cache)

  15. OCA Hardware • Space optimized: 4U high-density storage • Power optimized for low power/cooling requirements (≅500W) • Redundant 10GE optical network interfaces • Redundant power supplies (AC or DC)

  16. Why Deploy a Netflix OCA? • Netflix data is a significant percentage of ISP traffic in the markets we serve • Serve >80% of Netflix traffic from the local ISP datacenter • Remaining traffic can be served by upstream OCA’s, peering • Saves on transit, transport and other upstream scaling costs • Provided free of charge to participating ISPs • ISPs with > 3Gbps of Netflix traffic • ISP provides rack space, power, 10 Gbps optical port(s)

  17. OCA Operation • Used exclusively for Netflix content • Completely integrated with the Netflix content delivery system • Greater effectiveness than transparent or proxy caches (>80% with Netflix OCA, versus 20-50% with other caches) • Efficient content fill mechanisms • Outside peak times (ISP selected time and BW per OCA) • OCA offline during fill, staggered fill recommended. • Fill can source from neighboring, peer or transit OCA’s • ~5TB fill per day (i.e ~4Gb/s for ~2.5 hrs) • OCA must be reachable by end users (port 80)

  18. Directing Clients to OCAs Broadband ISP Netflix Control Servers • User routing is done by Netflix control servers, not dependent on client DNS configuration • Request is routed to the nearest available OCA • Working sets of popular content deliver up to >80% network offload 1. Client contacts Netflix control servers 2. Netflix control servers respond with address of local OCA Netflix OCA 3. Client connects to local OCA 4. Local OCA delivers video stream

  19. Network Positioning • OCA’s are installed close to aggregations of end users • Netflix works with ISPs to determine proper location and user-to-OCA mapping • ISP Controls Traffic Flow • End-user netblocks are associated with OCA via BGP advertisement • Netflix honors AS path-length and MEDs allowing ISP to configure OCA hierarchy and failover • OCA’s only serve content to netblocks provided by ISP • ISPs choose peak throughput each OCA is allowed to generate

  20. The Netflix ISP Speed Index • Published monthly • Helps consumers make an intelligent choice about their broadband provider • We want our users to have the best experience possible • We also want to help networks deliver Netflix traffic as efficiently as possible

  21. A recently enabled network (night of 18/03/13) No marked increase in bandwidth utilization, but a more stable experience for users

  22. Questions? Additional information: http://openconnect.netflix.com

More Related