The Next Wave of Massive Disruptions to the Peering Ecosystem

The Next Wave of Massive Disruptions to the Peering Ecosystem William B. Norton Co-Founder & Chief Technical Liaison Equinix, Inc. Global Peering Forum March 26, 2007 30-minute-disruption

On the Internet Everyone is a Publisher

Internet Operations White Papers • “Interconnection Strategies for ISPs” • “Internet Service Providers and Peering” • “A Business Case for Peering” • “The Art of Peering: The Peering Playbook” • “The Peering Simulation Game” • “Do ATM-based Internet Exchanges Make Sense Anymore?” • “Evolution of the U.S. Peering Ecosystem” • “The Asia Pacific Internet Peering Guidebook” • “The Folly of Peering Traffic Ratios?” Freely available. See Web site or send e-mail to wbn@equinix.com Internet makes anyone a publisher, similar effect now emerging for video

Massive Disruption in U.S. Peering Ecosystem  Short Videos • YouTube – founded 2005 • Short video clips – 50 million view per day! • 20Gbps of peering traffic Feb 2006 • $1M/month in Sept 2006! • Entering Peering Ecosystem • 30 Other competitors600Gbps peerable? • DoveTail • Video may dwarf current peered traffic • 2010 – 80-90% Internet is Video • Inculcate video guys into peering ecosystem On the Internet Everyone is a Broadcaster Short video clips…Full TV shows… Source: http://digg.com/tech_news/YouTube_Gets_Bandwidth_Boost_from_Level_3 Source: http://www.nanog.org/mtg-0606/norton.html

Massive Disruption in U.S. Peering Ecosystem  Full Episodes • “Desperate Housewives” – 210MB/hour • For 320x240 H.264 Video iTunes image • 10,000,000 households • 2,100,000,000 MB = 2.1 peta-Bytes • How long will that take to download? 3 days @ 64Gbps non-stop ! Just one show Try 250M*180 Channels*HDTV Historical Perspective…review 5yr disruptions… Source: http://www.pbs.org/cringely/pulpit/pulpit20060302.html

Research Topic • Massive Wave of Internet Traffic • 90% of all Internet bits by 2010 • How will Video Service Providers distribute this massive amount of Video Traffic over the Internet? How will Video Service Provider distribute?

Modeling the Video Service Provider Distribution Networks Four Models • Commodity Transit • CDN • Transit/Peering/DIY CDN • Peer2Peer Four Load Models A: Small Load B: Medium Load C: Large Load Goal : estimate cost : $/video downloaded

Model 1: Commodity Transit Business Premise: • VSP focuses on core competence • Transit Providers handle traffic better and cheaper • Economies of scale, Aggregation, Expertise, Billing, Peering, etc.

Model 1A: Transit Light Load

Model 1B: Transit Medium Load

Upstream ISPs Model 1C Router4 Router2 10G Router2 8 * 10GE to upstreams each Server1 GigE Switch1 Router1 : Server24 : 10G : GigE Switch14 : : Server262 Server263 Server264 : Distribution GigE Switch 48 port GigE for servers 2 10GE for upstream $10,000 Add another every 24 servers Routers Cisco 6509Sup720-3bxl w/4*4-port 10GE, $150,000 80Gbps from switches, 80Gbps to upstreams

Model 2: Content Delivery Networks (CDNs) for the Distribution of Video Content • Business Premise: Single-site transit traffic traverses potentially many network devices, increasing latency and the potential of packet loss: • By spreading web objects closer to the eyeball networks latency is reduced • Fewer network elements are traversed so reliability is improved • Congestion points in the core of the Internet are avoided • CDNs have the expertise, deployed infrastructure, economies of scale from aggregation efficiencies.

Assumption CDN Price Points

Model 2A: CDN Light Load

Model 2B: CDN Medium Load

Model 2C: CDN Large Load

Model 3: Transit/Peering/DIY CDN • Business Model Premise: Operation of the Internet distribution is seen as strategic to the VSP: • End-user experience is mission-critical so outsourcing the end user experience to a transit provider or CDN puts the VSP at risk. • The VSP has visibility into what video are being released, which ones are likely to be hot and which ones don’t require special infrastructure adjustments. • Internet Video distribution is so new that the VSP prefers control. This is a strategic focus of the VSP: ensuring reliability, scalability, through the constant monitoring and evolution of the infrastructure to ensure the end user experiences during these early phases of Internet Video Distribution. • The traditional CDN may be ill-suited to distribute very large video object, therefore we have to do it yourself.

Model 3: Transit/Peering Light Load

Model 3A: Peering/Transit Light Load

Model 3B: Transit/Peering Medium Load

Model 3C: Transit/Peering Heavy Load

Model 4: Peer2Peer • Business Model Premise: The current Internet Service Providers and CDNs at the core can not handle the load across single or even multiple locations: • Backbone, peering interconnects, and the hundreds of thousands of routers deployed can not handle the load of today and tomorrows video. • the leaf nodes (i.e. Grandma’s PCs left on) in aggregate have the cycles and network capacity, if shared, to distribute popular content today. • Popular content can be chopped up into small chunks such that many downloaders become sources, and topologically close downloaders will prefer the topologically close sources. This ‘swarmcasting’ requires only a source ‘seed’, and a lookup mechanism for the first downloaders to find the seed, and then to direct future downloaders to topologically closer sources.

Model 4A: P2P Light Load

Model 4B: P2P Medium Load

Model 4C: P2P Large Load

Summary Per Video Cost Of delivery

Observation/Implications • Internet Transit Supply ▼ • Internet Transit Price ▲ • Internet Transit Model  src/dst specific • Bottlenecks • IX Power, Router Capacity, Peer’s Capacity, • Last Mile bottleneck • Do I need to upgrade $$$$ gear to support my competitor (peer)? • Geoff Huston: “P2P has won. Telco/Cable co trying to keep its 1998 biz plan relevant.” I look forward to discussing this over the next few days.

The Next Wave of Massive Disruptions to the Peering Ecosystem