1 / 53

A Flexible and Efficient API for a Customizable Proxy Cache

A Flexible and Efficient API for a Customizable Proxy Cache. Vivek S. Pai, Alan L. Cox, Vijay S. Pai, and Willy Zwaenepoel. iMimic Networking, Inc. http://www.imimic.com. Motivation. More features moving into proxy caches The ubiquitous layer 7 device

jonah
Download Presentation

A Flexible and Efficient API for a Customizable Proxy Cache

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. A Flexible and Efficient API for a Customizable Proxy Cache Vivek S. Pai, Alan L. Cox, Vijay S. Pai, and Willy Zwaenepoel iMimic Networking, Inc. http://www.imimic.com

  2. Motivation • More features moving into proxy caches • The ubiquitous layer 7 device • Filtering, reporting, CDN support, transformation • Lots of this being done one-off, ad hoc • Can’t know everything at deployment • Some approaches for generalization • ICAP/OPES, proprietary mechanisms • But design considerations shifting • Goal: new approach for modern environments

  3. Contributions • Designed event-friendly proxy API • Implemented on iMimic DataReactor cache • Imposes negligible performance overhead • Demo modules • High performance • Low interference

  4. Outline • Background • API Design • API Functions • Implementation and Performance • Conclusions

  5. Proxy Cache Concepts clients WAN proxy cache LAN origin servers

  6. Why Program a Proxy? • It’s at the right point in network • Sees all client-side and server-side HTTP traffic • Can react to both LAN and WAN conditions • Already examines layer 7 • Groundwork in place for value-adds • Content filtering, access control, etc.

  7. Enabling Technologies • Moore’s Law • CPU speeds outstripping all other components • Lots of cycles to burn… • Proxy software • Increasing efficiency in managing connections, disk storage, etc. • Commodity OS/hardware improvements • No longer need specialized systems to run efficient proxy caches

  8. Commodity System Improvements • 1997: Appliances 4x faster than software running on a 2-processor UltraSparc • [Source: Danzig, “NetCache Architecture and Deployment”]

  9. Commodity System Improvements • 1997: Appliances 4x faster than software running on a 2-processor UltraSparc • [Source: Danzig, “NetCache Architecture and Deployment”] • 1st NLANR cacheoff (April ’99): gap only 2.5 x • 600 req/sec (Peregrine) vs. 1500 (InfoLibria)

  10. Commodity System Improvements • 1997: Appliances 4x faster than software running on a 2-processor UltraSparc • [Source: Danzig, “NetCache Architecture and Deployment”] • 1st NLANR cacheoff (April ’99): gap only 2.5 x • 2nd cacheoff (Jan ’00): gap only 1.7x • 1450 req/sec (iMimic) vs. 2400 (Compaq)

  11. Commodity System Improvements • 1997: Appliances 4x faster than software running on a 2-processor UltraSparc • [Source: Danzig, “NetCache Architecture and Deployment”] • 1st NLANR cacheoff (April ’99): gap only 2.5 x • 2nd cacheoff (Jan ’00): gap only 1.7x • 3rd cacheoff (Oct ’00): gap only 15% • 2083 req/sec (Microsoft) vs. 2400 (Compaq)

  12. Commodity System Improvements • 1997: Appliances 4x faster than software running on a 2-processor UltraSparc • [Source: Danzig, “NetCache Architecture and Deployment”] • 1st NLANR cacheoff (April ’99): gap only 2.5 x • 2nd cacheoff (Jan ’00): gap only 1.7x • 3rd cacheoff (Oct ’00): gap only 10% • 4th cacheoff (Dec ’01): commodity system best • Performance record: 2700 req/sec (Cintel/iMimic)

  13. How free is the CPU? • Stratacache Dart-10, with Nokia phone • 120 req/sec (7 Mbps) with 300 MHz CPU • CPU mostly idle; performance disk-limited

  14. Outline • Background • API Design • API Functions • Implementation and Performance • Conclusions

  15. Previous Customization Approaches • Write your own proxy or modify Squid • Huge code, changes likely to conflict with updates • ICAP: TCP-based offload • Proxy redirects requests/responses to a separate server for modification • Filter-style processes • Plugins where proxy designers anticipated a need (e.g., content filtering) • Kernel modules • Difficult programming model, but needed for kernel-integrated proxies

  16. Reasons for a New Approach • Scalability needed to > 10,000 flows • Filter processes may not scale • Limitations of ICAP-style offloading • Offloading small requests adds latency • Need for separate ICAP server with own CPU • Programmers want flexibility • Program in C using standard OS and libraries • Avoid problems from later code conflicts

  17. Design of the Proxy API • Event-aware • Modules notified as requests/responses arrive • Maps well to implementation of modern proxies • HTTP-Complete • Capture all key interactions in HTTP request-response protocol for full flexibility • Support various programming models • Events, threads, processes • Communication via function call or socket

  18. HTTP Data Flows Cache Misses Requests Server Client Proxy Cache New Content Responses Cache Hits Cached Content Storage System

  19. HTTP Data Flows and the API Server modify modify Client Proxy Cache modify modify modify Storage System

  20. Response Status Code Response header line 1 Response header line 2 ... Response header line N <blank terminating line> Actual response “body," containing HTML file, image binary data, etc. HTTP Request-Response Structure Requested URL Request header line 1 Request header line 2 ... Request header line N <blank terminating line> Header block – special first line followed by more detail about request/response Optional request “body" used in POST requests for forms, etc. Body data

  21. Design of API Notifications • typedef struct DR_FuncPtrs { • DR_InitFunc *dfp_init; // on module load • DR_ReconfigureFunc *dfp_reconfig; // on config change • DR_FiniFunc *dfp_fini; // on module unload • DR_ReqHeaderFunc *dfp_reqHeader; // when req hdr done • DR_ReqBodyFunc *dfp_reqBody; // on each piece of req body • DR_ReqOutFunc *dfp_reqOut; // before req to remote srv • DR_DNSResolvFunc *dfp_dnsResolv; // when DNS resolution needed • DR_RespHeaderFunc *dfp_respHeader; // when resp hdr done • DR_RespBodyFunc *dfp_respBody; // on each piece of resp body • DR_RespReturnFunc *dfp_respReturn; // when resp returned to clt • DR_TransferLogFunc *dfp_logging; // log entry after req done • DR_OpaqueFreeFunc *dfp_opaqueFree; // when each resp completes • DR_TimerFunc *dfp_timer; // periodic maintenance • int dfp_timerFreq; // timer period (sec) • } DR_FuncPtrs;

  22. Outline • Background • API Design • API Functions • Implementation and Performance • Conclusions

  23. API Functions • Content Adaptation • Content Management • Customized Administration • Utility Functions

  24. Content Adaptation • Functions to allow modules to inspect and modify requests and replies through cache Server modify modify Client Proxy Cache modify modify modify Storage System

  25. Content Adaptation (cont’d) • Example uses • Integration into a CDN based on URL rewriting • Transcoding for mobile devices • Special features of cache integration • Store modified content • Return multiple versions using HTTP Vary header

  26. Content Management • Fine-grained control over cacheability • Content-freshness modification/eviction • Content preloading • Content querying • Example uses • News CDN needs new home page on major event • Premium services

  27. Customized Administration • Notifications on logging • Example uses • Aggregation at network operation centers • Detection of high error rates indicates bad links

  28. Utility Functions • Interfaces to underlying OS event-notification • Module may register or clear interest on FD events • API will automatically call back module • Independent of underlying OS mechanisms (e.g., poll, select, /dev/poll, kevent) • Configuration options processing

  29. Outline • Background • API Design • API Functions • Implementation and Performance • Conclusions

  30. Implementation in DataReactor • Commercial proxy server • Portable (x86, Alpha, Sparc), and (FreeBSD, Linux, Solaris) • Fast (exposes overheads) • Independently measured at Proxy Cache-Offs (alone or via OEMs) • Support requires < 1000 lines of code • Implementation < 6 person-months

  31. Sample Modules • Ad Remover • Matches ad patterns in Hostname, URI • Dynamic Compressor • Uses zlib to compress, store, & serve object • Image Transcoder • Color stripping via NetPBM & ijpeg helpers • Text Injector • Finds <head> tag, asks helper what to insert • Content Manager • Local telnet, then query, fetch, inject, evict objects • ICAP client • Implements ICAP 1.0 draft to use external server

  32. Web Surfing Now

  33. Web Surfing Without Ads

  34. Sample Module Implementation

  35. Measurement • Polygraph and PolyMix-3, Measurement Factory • De facto standard for proxy testing • Scales with load • Number of clients • Number of servers • Data set size • Working set size • Very long test time • Fill phase (~14 hours) • Test phase (~10 hours)

  36. PolyGraph Test Phases Fill Phase 1st Load Phase 2nd Load Phase 0 5 10 15 20 25 30 Time (hours)

  37. PolyGraph Hit Rates Cacheable Offered Actual

  38. Our Test Environment • Proxy - 1.4GHz Athlon, 2GB memory • 5 SCSI disks, GigE, FreeBSD • Harness • 10 Polygraph client/server machines • Target load: 1450 reqs/sec • 16000 simultaneous connections • Pmix-3: Modified Polymix-3 • Single fill phase for all tests • Load phase time cut in half • Slight increase in hit rate

  39. API Performance

  40. Module Performance

  41. Outline • Background • API Design • API Functions • Implementation and Performance • Conclusions

  42. Summary • CPUs getting more idle • Commodity OS suitable choices • High-concurrency servers needed • Customizable, efficient event-friendly API • Implemented with low overhead • Sample results, deployments promising

  43. Ongoing Work • CoDeeN – a CDN system on PlanetLab • Uses a customized version of DataReactor • Being built at Princeton • Prototype: 1 week reading + 1 week reading • Currently: ~42 nodes (one per site) • Lessons • API easy enough for busy grad students • Logging infrastructure would be nice • Want to mask non-HTTP failures

  44. Questions? vivek@imimic.com iMimic Networking, Inc. http://www.imimic.com/

  45. Cacheoff-3 Hit Times

  46. Cacheoff-3 Miss Times

  47. Cacheoff-3 Improvements

  48. Cacheoff-3 Price/Performance

  49. CacheOff-3 Results

  50. CacheOff-3 Results

More Related