Distributing Caching on Large IP Networks

1. Distributing Caching on Large IP Networks Adrian Chadd & Andrew Khoo Versatel Telecom – Amsterdam NL NANOG17 – Distributed Caching in Large IP Environments Adrian Chadd & Andrew Khoo - 10th October 1999

2. Why Cache? • Average client bandwidth growth is exceeding backbone capacity growth • The internet is the ‘unknown’ factor – congestion problems outside your control NANOG17 – Distributed Caching in Large IP Environments Adrian Chadd & Andrew Khoo - 10th October 1999

3. Caching Problems Today • Bad server implementations of HTTP/1.1 • Bad implementation of dynamic content timeouts • Some websites use client IP as part of authentication NANOG17 – Distributed Caching in Large IP Environments Adrian Chadd & Andrew Khoo - 10th October 1999

4. Inter-cache communication NANOG17 – Distributed Caching in Large IP Environments Adrian Chadd & Andrew Khoo - 10th October 1999

5. ICP • Very basic method of inter-cache communication • Only HIT/MISS information is propagated between caches • Connectionless (UDP) based NANOG17 – Distributed Caching in Large IP Environments Adrian Chadd & Andrew Khoo - 10th October 1999

6. Cache Digests • Utilizes a ‘bloom filter’ to determine caches with higher probabilities of having given objects • Periodic exchanges of digests between caches • Not Exact! Miss = 0% chance of having object, Hit = >95% chance of having object • Work ‘in progress’ NANOG17 – Distributed Caching in Large IP Environments Adrian Chadd & Andrew Khoo - 10th October 1999

7. Caching a large network- a case study NANOG17 – Distributed Caching in Large IP Environments Adrian Chadd & Andrew Khoo - 10th October 1999

8. Cache Objectives • Primary goal: Improve client response times • Secondary goal: Reduce bandwidth usage NANOG17 – Distributed Caching in Large IP Environments Adrian Chadd & Andrew Khoo - 10th October 1999

9. 622 LHR L3 CDG BRU FRA M2 VT AMSIX PARIX SFINX B-NIX DCIX LINX Backbone Europe 100 155Mbps level3 155+155 100 622 Amsterdam 155 T3-ATM 2x100 100 155 622 100 NANOG17 – Distributed Caching in Large IP Environments Adrian Chadd & Andrew Khoo - 10th October 1999

10. PAIX Backup transit to “Internet” ORD NYC DC AADS PAIX CIX MAE West MAE East Backbone US 155Mbps 155Mbps AMS 45Mbps 100Mbps 155Mbps 155Mbps 100T 100Mbps 100Mbps FDDI FDDI 100T NANOG17 – Distributed Caching in Large IP Environments Adrian Chadd & Andrew Khoo - 10th October 1999

11. Cache Backbone United States Europe London 2 x OC3/STM-1 OC3/STM-1 New York Amsterdam OC3/ STM-1 OC3/STM-1 Paris Frankfurt OC12/ STM-4 Each POP has a single cache cluster Brussels NANOG17 – Distributed Caching in Large IP Environments Adrian Chadd & Andrew Khoo - 10th October 1999

12. Backbone POP Design Cisco GSR12008 Core Router Juniper M40 Core Router 155Mbps POS 155Mbps POS 155Mbps POS Cisco 7206vxr Aggregator #1 Cisco 7206vxr Aggregator #2 100baseT 100baseT Cisco 2948 100T cust attach Cisco Cat5505 100T cust attach 100baseT Routing Path #1 Routing Path #2 NANOG17 – Distributed Caching in Large IP Environments Adrian Chadd & Andrew Khoo - 10th October 1999

13. NL Core Network Topology AMS-IX #2 100bT to Carrier1 155Mbps to NYC AMS-IX #1 VERSATOWER Gigabit Ethernet 100bT 7204 NIKHEF SARA Business Internet (via ATM) 100bT 45Mbps 7206vxr 100bT 100bT M40 155Mbps 7206vxr 7202 100bT 4-way 155Mbps router mesh 155Mbps 155Mbps Dial Access Platform GSR 100bT GSR 7206vxr 7202 155Mbps to Svianed #1 155Mbps to Svianed #2 Data Operations MATRIX-2 Free Internet/ISP Platform NANOG17 – Distributed Caching in Large IP Environments Adrian Chadd & Andrew Khoo - 10th October 1999

14. Single Cache Cluster Clients Frontend 1 Frontend 2 Frontend 3 Backend 1 Backend 2 Backend 3 Internet NANOG17 – Distributed Caching in Large IP Environments Adrian Chadd & Andrew Khoo - 10th October 1999

15. Single Cache Cluster (2) • Frontends handle client requests • Backends handle server requests • Frontends ICP each other • Each backend exports a cache digest to each frontend • Backends do NOT communicate NANOG17 – Distributed Caching in Large IP Environments Adrian Chadd & Andrew Khoo - 10th October 1999

16. Hierarchy construction NANOG17 – Distributed Caching in Large IP Environments Adrian Chadd & Andrew Khoo - 10th October 1999

17. Problems with conventional hierarchies • Conventional hierarchies use static or domain based parent/peer selection • Network/server failures can affect client response times • Limited network topology intelligence through a ‘whois’ interface NANOG17 – Distributed Caching in Large IP Environments Adrian Chadd & Andrew Khoo - 10th October 1999

18. How to ‘route’ cache peer selection • Using transparent redirection at each border, let your network route requests for you • Network failures are handled correctly • Method is performance-friendly, but not optimally bandwidth-friendly NANOG17 – Distributed Caching in Large IP Environments Adrian Chadd & Andrew Khoo - 10th October 1999

19. Cache pre-population • Cache prepopulation WORKS ! • Parsing ‘friendly’ provider logs from timezones before your own • Walk ‘popular’ websites before sunlight • Parsing your own logs and issuing IMS requests NANOG17 – Distributed Caching in Large IP Environments Adrian Chadd & Andrew Khoo - 10th October 1999

20. Melting squid caches NANOG17 – Distributed Caching in Large IP Environments Adrian Chadd & Andrew Khoo - 10th October 1999

21. Melting squid caches – filesystem performance • Directory seek overhead • No concept of object relationships / locality, defeating ‘read-ahead’ • Current implementation uses threads to fake async file access • Disks are still linear :-) NANOG17 – Distributed Caching in Large IP Environments Adrian Chadd & Andrew Khoo - 10th October 1999

22. Melting squid caches - IFS • IFS – “inode filesystem” • Exporting flat inode namespace to squid • Not optimal by far, but relaxes disk seek and memory usage • Integration into squid is under way NANOG17 – Distributed Caching in Large IP Environments Adrian Chadd & Andrew Khoo - 10th October 1999

23. Melting squid caches – poor network performance • Using larger buffer sizes on outgoing requests • Reduce the TIME_WAIT period to cycle through sockets quicker • QoS on your internal network NANOG17 – Distributed Caching in Large IP Environments Adrian Chadd & Andrew Khoo - 10th October 1999

24. Melting squid caches – ICP • Dropped ICP packets result in noticeable client delays • ICP not suitable for busy WAN links • Cache digests are a ‘solution in progress’ • Cache the ICP replies NANOG17 – Distributed Caching in Large IP Environments Adrian Chadd & Andrew Khoo - 10th October 1999

25. Melting squid caches – other problems • Proprietary protocols (eg WEBDAV) • Some browsers (MSIE) have a habit of issuing non-cache friendly requests • Issues with transparent redirection – eg MTU Path Discovery, IP-based authentication NANOG17 – Distributed Caching in Large IP Environments Adrian Chadd & Andrew Khoo - 10th October 1999

26. Caching – the next level • True object-based caching, instead of HTTP/FTP only • True clustering support, providing a single logical cache to the network • Smart ‘predictive’ caches adapting to client usage patterns • Video/Audio stream caching NANOG17 – Distributed Caching in Large IP Environments Adrian Chadd & Andrew Khoo - 10th October 1999

27. Questions and Answers NANOG17 – Distributed Caching in Large IP Environments Adrian Chadd & Andrew Khoo - 10th October 1999

28. The End adrian.chadd@ip.versatel.net andrew.khoo@ip.versatel.net NANOG17 – Distributed Caching in Large IP Environments Adrian Chadd & Andrew Khoo - 10th October 1999