1 / 50

Experiences with CoralCDN A Five-Year Operational View

Experiences with CoralCDN A Five-Year Operational View. Michael J. Freedman Princeton University www.coralcdn.org. A Cooperative, Self-Organizing CDN. Client. Resolver. 2. 5. 1. CoralCDN HTTP Proxy. CoralCDN DNS Server. CoralCDN HTTP Proxy. CoralCDN DNS Server. Coral index node.

aerona
Download Presentation

Experiences with CoralCDN A Five-Year Operational View

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Experiences with CoralCDNA Five-Year Operational View Michael J. Freedman Princeton University www.coralcdn.org

  2. A Cooperative, Self-Organizing CDN Client Resolver 2 5 1 CoralCDN HTTP Proxy CoralCDN DNS Server CoralCDN HTTP Proxy CoralCDN DNS Server Coral index node Coral index node 3 6 CoralCDN 4 CoralCDN HTTP Proxy CoralCDN DNS Server CoralCDN HTTP Proxy Coral index node Coral index node Goal: To make desired content widely available regardless of publisher’s own resources, by organizing and utilizing any cooperative resources

  3. http://example.com/path http://example.com.nyud.net/path

  4. Adopted by: Clients Servers Third-parties

  5. Many of you have used CoralCDN

  6. Many of you have used CoralCDN

  7. Many of you have used CoralCDN

  8. Many of you have used CoralCDN

  9. Many of you have used CoralCDN

  10. Many of you have used CoralCDN

  11. Daily Request Volume 2M clients – 2 TB content – 20K origin domains From 300-400 PlanetLab servers

  12. Based on peer-to-peer DHT CoralCDN HTTP Proxy CoralCDN DNS Server CoralCDN HTTP Proxy CoralCDN DNS Server Coral index node Coral index node CoralCDN CoralCDN HTTP Proxy CoralCDN DNS Server CoralCDN HTTP Proxy Coral index node Coral index node Weakened consistency + algorithms that prevent tree saturation during lookup Decentralized clustering for locality and hierarchical lookup Cooperative HTTP / DNS that leverages locality

  13. Based on peer-to-peer DHT CoralCDN HTTP Proxy CoralCDN DNS Server CoralCDN HTTP Proxy CoralCDN DNS Server Coral index node Coral index node CoralCDN CoralCDN HTTP Proxy CoralCDN DNS Server CoralCDN HTTP Proxy Coral index node Coral index node Weakened consistency + algorithms that prevent tree saturation during lookup Decentralized clustering for locality and hierarchical lookup Cooperative HTTP / DNS that leverages locality

  14. CoralCDN HTTP Proxy CoralCDN DNS Server CoralCDN HTTP Proxy CoralCDN DNS Server Coral index node Coral index node CoralCDN CoralCDN HTTP Proxy CoralCDN DNS Server CoralCDN HTTP Proxy Coral index node Coral index node Interactions with the External Environment Virtualization Layer Clients Origin Domains

  15. Experiences • Naming • Fault Tolerance • Resource management • Revisit CoralCDN’s design

  16. Naming  Flexible, open API x Mismatch with domain-based access control policies

  17. CoralCDN’s Platform-as-a-Service API Rewrite rules in origin webservers RewriteEngine on RewriteCond %{HTTP_USER_AGENT} !^CoralWebPrx RewriteCond %{QUERY_STRING} !(^|&)coral-no-serve$ RewriteRule ^(.*)$ http://%{HTTP_HOST}.nyud.net% {REQUEST_URI} [R,L]

  18. Elastic Provisioning CoralCDN’s Platform-as-a-Service API Rewrite rules in origin webservers RewriteEngine on RewriteCond %{HTTP_USER_AGENT} !^CoralWebPrx RewriteCond %{QUERY_STRING} !(^|&)coral-no-serve$ RewriteCond %{HTTP_REFERER} slashdot\.org [NC] RewriteCond %{HTTP_REFERER} digg\.com [NC,OR] RewriteCond %{HTTP_REFERER} blogspot\.com [NC,OR] RewriteRule ^(.*)$ http://%{HTTP_HOST}.nyud.net% {REQUEST_URI} [R,L] • Sites integrate with load/bandwidth monitoring

  19. Naming Conflation http://domain/path .service1 .service2  x x Location to retrieve content Human-readable name for administrative entity Security policies to govern objects’ interactions

  20. Domain-based Security Policies Web Page evil.com target.com Cookies Document Object Model

  21. Domain-based Security Policies Web Page evil.com .nyud.net target.com .nyud.net Cookies Document Object Model Defaults violate least privilege

  22. Fault Tolerance: Failure Decoupling  • Internal failures: • DHT nodes • DNS servers, HTTP proxies • Management service x • External failures: • Decouple IPs from hosts • Interactions with origin sites • - Node failures • - DHT / DHS / HTTP level • - Management layer • Couldn't decouple IPs from host

  23. happens! Origin Status CoralCDN Reaction Cache negative results Serve stale content Use whole-file overwrites • Unresponsive • Returns error code • Reply truncated

  24. happens! Origin Status CoralCDN Reaction Cache negative results Serve stale content Use whole-file overwrites • Unresponsive • Returns error code • Reply truncated Maintain status quo unless improvements are possible

  25. What is “failure”? Return values should have fail-safe defaults

  26. Resource Management  Control over bandwidth consumption x Control and visibility into environment’s resources • - Node failures • - DHT / DHS / HTTP level • - Management layer • Couldn't decouple IPs from host

  27. Some timeline… Mar 2004 CoralCDN released on PlanetLab

  28. Some timeline… Mar 2004 CoralCDN released on PlanetLab Aug 2004 Slashdotted

  29. Some timeline… Mar 2004 CoralCDN released on PlanetLab Aug 2004 Slashdotted Dec 2004 Asian Tsunami PlanetLab traffic jumps Site threatens to yank PL PL admin kills slice Slice restored next day Initiates discussion of resource limits for slices

  30. Demand >> Supply:Enter Fair-Sharing Algorithms ? Σi di ≤ S Avg MB per hour (di) Domains with heaviest consumption

  31. Demand >> Supply:Enter Fair-Sharing Algorithms find max λ, s.t. Σimin (λ, di) ≤ S λ Avg MB per hour (di) Domains with heaviest consumption

  32. Demand >> Supply:Enter Fair-Sharing Algorithms find max λ, s.t. Σimin (λ, di) ≤ S λ Domains with heaviest consumption

  33. Admission Control under Fair-Sharing ~10 kB imgs 3.3% rejected ~5 MB videos 89% rejected Demand > 10 TB Supply ≤ 2 TB

  34. Some timeline… Mar 2004 CoralCDN released on PlanetLab Aug 2004 Slashdotted Dec 2004 Asian Tsunami Mar 2006 PL deploys bandwidth throttling PlanetLab traffic jumps Site threatens to yank PL PL admin kills slice Slice restored next day Initiates discussion of resource limits for slices

  35. Resource Management: Us vs. Them Application Hammer Platform Hammer Track all network traffic If total > 80% daily rate, BW shaping in kernel • Track HTTP traffic • If site > fair share rate, reject via HTTP 403 • If total > peak rate, close server socket

  36. Resource Management: Us vs. Them Application Hammer Track all network traffic If total > 80% daily rate, BW shaping in kernel Platform Hammer • Track HTTP traffic • If site > fair share rate, reject via HTTP 403 • If total > peak rate, close server socket Result: HTTP traffic is 1/2 - 2/3 of all traffic Lower layers should expose greater visibility and control over resources

  37. Experiences • Naming • Fault Tolerance • Resource management • Revisit CoralCDN’s design

  38. Usage Scenarios Resurrecting old content Accessing unpopular content

  39. Usage Scenarios Resurrecting old content Accessing unpopular content Serving long-term popular content

  40. Usage Scenarios Resurrecting old content Accessing unpopular content Serving long-term popular content

  41. Usage Scenarios Resurrecting old content Accessing unpopular content Serving long-term popular content

  42. Usage Scenarios Resurrecting old content Accessing unpopular content Serving long-term popular content Surviving flash crowds to content

  43. Usage Scenarios Resurrecting old content Accessing unpopular content Serving long-term popular content Surviving flash crowds to content

  44. Usage Scenarios 5 second epochs 10 minute epochs Resurrecting old content Accessing unpopular content Serving long-term popular content Surviving flash crowds to content

  45. Conclusions? Most requested content is long-term popular and already cached locally “Flash” crowds occur, but on order of minutes

  46. Conclusions? • Focus on long-term popular • Little / no HTTP cooperation • Global discovery (e.g., DNS) • Focus on flash crowds • Regional coop. as default • Global coop. as failover Most requested content is long-term popular and already cached locally “Flash” crowds occur, but on order of minutes

  47. Reconfiguring CoralCDN’s design Latency 90% Origin Load 5% Failover to global 0.5% Leverage Coral hierarchy for lookup

  48. Reconfiguring CoralCDN’s design Σimin (λ, di) < S λ heavily weight history in ewma Leverage Coral hierarchy for lookup During admission control, bias against long-term use

  49. Conclusions • Experiences • Naming • Fault Tolerance • Resource management • Revisit CoralCDN’s design • Current design unnecessary for deployment / most use • Easy changes to promote flash-crowd mitigation

  50. Can we reach Internet scale? www.firecoral.net Initial beta-release of browser-based P2P web cache

More Related