The Next Wave of Massive Disruptions to the Peering Ecosystem

1. The Next Wave of Massive Disruptions to the Peering Ecosystem William B. Norton Co-Founder & Chief Technical Liaison Equinix, Inc. Asia Pacific Peering Forum Sydney, Nov. 8, 2006 Slide set v1.1

2. Definitions of Peering and Transit Def:Peering is the business relationship whereby ISPs reciprocally announce reachability to each others’ transit customers. Peering Peering WestNet USNet EastNet Routing Table • (Note: Peering is a Non-Transitive Relationship) Def:Transit is the business relationship whereby one ISP sells access to all destinations in its routing table. Game of relationships….

3. Peering is a game of relationships White Papers….My job these days…

4. On the Internet Everyone is a Publisher

5. Internet Operations White Papers • “Interconnection Strategies for ISPs” • “Internet Service Providers and Peering” • “A Business Case for Peering” • “The Art of Peering: The Peering Playbook” • “The Peering Simulation Game” • “Do ATM-based Internet Exchanges Make Sense Anymore?” • “Evolution of the U.S. Peering Ecosystem” • “The Asia Pacific Internet Peering Guidebook” • “The Folly of Peering Traffic Ratios?” Freely available. See Web site or send e-mail to wbn@equinix.com Internet makes anyone a publisher, similar effect now emerging for video

6. Massive Disruption in U.S. Peering Ecosystem  Short Videos • YouTube – founded 2005 • Short video clips – 50 million view per day! • 20Gbps of peering traffic Feb 2006 • $1M/month in Sept 2006! • Entering Peering Ecosystem • 30 Other competitors600Gbps peerable? • DoveTail • Video may dwarf current peered traffic • 2010 – 80-90% Internet is Video • Inculcate video guys into peering ecosystem On the Internet Everyone is a Broadcaster Short video clips…Full TV shows… Source: http://digg.com/tech_news/YouTube_Gets_Bandwidth_Boost_from_Level_3 Source: http://www.nanog.org/mtg-0606/norton.html

8. Massive Disruption in U.S. Peering Ecosystem  Full Episodes • “Desperate Housewives” – 210MB/hour • For 320x240 H.264 Video iTunes image • 10,000,000 households • 2,100,000,000 MB = 2.1 peta-Bytes • How long will that take to download? 3 days @ 64Gbps non-stop ! Just one show Try 250M*180 Channels*HDTV Historical Perspective…review 5yr disruptions… Source: http://www.pbs.org/cringely/pulpit/pulpit20060302.html

9. Research Topic • Massive Wave of Internet Traffic • 90% of all Internet bits by 2010 • How will Video Service Providers distribute this massive amount of Video Traffic over the Internet? How will Video Service Provider distribute?

10. Modeling the Video Service Provider Distribution Networks Four Models • Commodity Transit • CDN • Transit/Peering/DIY CDN • Peer2Peer Four Load Models A: Small Load B: Medium Load C: Large Load Goal : estimate cost : $/video downloaded

11. Model 1: Commodity Transit Business Premise: • VSP focuses on core competence • Transit Providers handle traffic better and cheaper • Economies of scale, Aggregation, Expertise, Billing, Peering, etc.

12. Model 1A: Transit Light Load

13. Model 1B: Transit Medium Load

14. Upstream ISPs Model 1C Router4 Router2 10G Router2 8 * 10GE to upstreams each Server1 GigE Switch1 Router1 : Server24 : 10G : GigE Switch14 : : Server262 Server263 Server264 : Distribution GigE Switch 48 port GigE for servers 2 10GE for upstream $10,000 Add another every 24 servers Routers Cisco 6509Sup720-3bxl w/4*4-port 10GE, $150,000 80Gbps from switches, 80Gbps to upstreams

15. Model 2: Content Delivery Networks (CDNs) for the Distribution of Video Content • Business Premise: Single-site transit traffic traverses potentially many network devices, increasing latency and the potential of packet loss: • By spreading web objects closer to the eyeball networks latency is reduced • Fewer network elements are traversed so reliability is improved • Congestion points in the core of the Internet are avoided • CDNs have the expertise, deployed infrastructure, economies of scale from aggregation efficiencies.

16. Assumption CDN Price Points

17. Model 2A: CDN Light Load

18. Model 2A: CDN Light Load

19. Model 2B: CDN Medium Load

20. Model 2C: CDN Large Load

21. Model 3: Transit/Peering/DIY CDN • Business Model Premise: Operation of the Internet distribution is seen as strategic to the VSP: • End-user experience is mission-critical so outsourcing the end user experience to a transit provider or CDN puts the VSP at risk. • The VSP has visibility into what video are being released, which ones are likely to be hot and which ones don’t require special infrastructure adjustments. • Internet Video distribution is so new that the VSP prefers control. This is a strategic focus of the VSP: ensuring reliability, scalability, through the constant monitoring and evolution of the infrastructure to ensure the end user experiences during these early phases of Internet Video Distribution. • The traditional CDN may be ill-suited to distribute very large video object, therefore we have to do it yourself.

22. Model 3: Transit/Peering Light Load

23. Model 3A: Peering/Transit Light Load

24. Model 3B: Transit/Peering Medium Load

25. Model 3C: Transit/Peering Heavy Load

26. Model 4: Peer2Peer • Business Model Premise: The current Internet Service Providers and CDNs at the core can not handle the load across single or even multiple locations: • Backbone, peering interconnects, and the hundreds of thousands of routers deployed can not handle the load of today and tomorrows video. • the leaf nodes (i.e. Grandma’s PCs left on) in aggregate have the cycles and network capacity, if shared, to distribute popular content today. • Popular content can be chopped up into small chunks such that many downloaders become sources, and topologically close downloaders will prefer the topologically close sources. This ‘swarmcasting’ requires only a source ‘seed’, and a lookup mechanism for the first downloaders to find the seed, and then to direct future downloaders to topologically closer sources.

27. Model 4A: P2P Light Load

28. Model 4A: P2P Light Load

29. Model 4B: P2P Medium Load

30. Model 4B: P2P Medium Load

31. Model 4C: P2P Large Load

32. Model 4C: P2P Large Load

33. Summary Per Video Cost Of delivery

34. Observation/Implications • Internet Transit Supply ▼ • Internet Transit Price ▲ • Internet Transit Model  src/dst specific • Bottlenecks • IX Power, Router Capacity, Peer’s Capacity, • Last Mile bottleneck • Do I need to upgrade $$$$ gear to support my competitor (peer)? • Geoff Huston: “P2P has won. Telco/Cable co trying to keep its 1998 biz plan relevant.” I look forward to discussing this over the next few days.