A Scalable Content Distribution Service for Dynamic Web Content

1. A Scalable Content Distribution Service for Dynamic Web Content Seejo Sebastine Department of Computer Science University of Virginia

2. Outline • Why cache dynamic web content? • What has been done before? • Our solution • The “Active Web Caching Architecture” • Implementation and Protocol Details • Sample Consistency Management Model • Evaluation • Conclusions and Future Work

3. Why cache dynamic content? • Increasing number of applications generating dynamic content • Client-perceived latency improvements • Problems in caching dynamic content

4. What has been done before? • Caching static content • Co-operative caching of dynamic content on distributed web servers • Cao’s “Active Cache” • Consistency management problem

5. Our solution • Cache dynamic/active content by migrating the content generating scripts from servers to proxies (called “active caches”) close to the clients • Also migrate the data used by these scripts • Execute the scripts on the proxy cache and cache the results as static content

6. Our solution • Maintain data consistency by periodically checking the status of the data on the server • Period of checking is determined by server specified consistency bounds on the data • E.g: Stock quotes – 2 to 5 minutes, traffic information – 2 to 10 minutes etc. • Aim to provide as good consistency as can be provided in spite of the bottlenecks • Different consistency management algorithms can be plugged in

7. Active Web Caching Architecture • Replicates content generating scripts and their data on-demand onto a set of “active caches” • Executes the scripts on the active cache and caches the results • Maintains consistency of the data with the server within specified bounds • Entire architecture uses only components that are currently in use on the web with some modifications

8. Active Web Caching Architecture

9. Active Web Caching Architecture • Each active cache has a static proxy cache and one or more active servers • Each active server handles one or more types of related dynamic content • We use: proxy cache – Squid 2.3 active server – Apache 1.3.9 • Proxy redirector, URL rewriter, Consistency Manager, Script Execution Module

10. Implementation & Protocol Details • Client Request flow: • Client Proxy Cache • Proxy cache handles static content, forwards dynamic content to active server • Active server fetches the script & data (if reqd.), executes it, caches the script and forwards the results to the Squid proxy cache

11. Protocol Details (Squid Proxy) • Squid forwards request to Redirector (JESred) • Redirector separates out static & dynamic content: • abort.html, .jpg, .shtml, .java, .jar, .htm – do not redirect static content • regex|^http://[^\]*/cgi-bin/(.*$)| http://tarek8.cs.virginia.edu:8080/cgi-bin/\1|/cgi-bin/ • What this rule does is: • http://www.xyz.com/cgi-bin/hello.cgi becomes • http://tarek8.cs.virginia.edu:8080/cgi-bin/hello.cgi where tarek8.cs.virginia.edu is the active server • By default Squid rewrites redirected host headers – disable that, because we need to remember the original host for whom the request was meant. • Default:GET http://www.xyz.com/cgi-bin/hello.cgi HTTP1.1 • Host: www.xyz .com • Becomes:GET http://www.tarek8.cs.virginia.edu:8080/cgi-bin/hello.cgi HTTP1.1 • Host: tarek8.cs.virginia.edu:8080 • We need: GET http://www.tarek8.cs.virginia.edu:8080/cgi-bin/hello.cgi HTTP1.1 • Host: www.xyz.com

12. Protocol Details (Active Server) • Squid does not cache dynamic content by default, so change the default behavior by disabling the configuration option: • aclQUERYurlpath_regex cgi-bin \? • no_cachedenyQUERY • Active server receives requests for only dynamic content from Squid. • First time: Forwards request to content server. Changes the HTTP Accept header to include: x-script/x-activeScripts, so that content server can differentiate between active cache and normal cache. • Forwarding is done by the URL rewriter module (Apache’s mod_rewrite) by using the Hostname from the HTTP “Host:” header – sample rewrite rule: • RewriteRule^/cgi-bin/(.*$)http://%{HTTP_HOST}/cgi-bin/$1[P] • Forwards the request as a proxy request (i.e. the mod_proxy module is used) so that the reply can be cached • http://www.xyz.com/cgi-bin/hello.cgi http://tarek8.cs.virginia.edu:8080/cgi-bin/hello.cgi http://www.xyz.com/cgi-bin/hello.cgi

13. Protocol Details (Content Server) • Content server receives request with modified Accept Header (with x-script/x-activeScripts) and returns the script without executing it locally. (Requests without this special header are treated as normal CGI requests and executed locally.) • Active server receives the “script” as the reply from the content server and caches it in its script cache (which is Apache’s mod_proxy module’s proxy cache). • Each script may use multiple data files – they need to be fetched from the content server too using HTTP requests. To do this we provide our Script API which handles fetching the files automatically. • Active server then executes the script locally, and sends the results to the Squid proxy.

14. Script API • ASContentServer (Content server name, port) • FD = ASopen (<filename>, attributes, Consistency model, minimum Consistency, maximum Consistency) • Consistency Model: “EXPIRES” or “TEMPFILE” • Minimum Consistency – best case – say, 2 minutes for stock quotes • Maximum Consistency – worst case – say, 5 minutes for stock quotes • Status = ASread (FD, buffer, count) • Status = ASwrite (FD, buffer, count) • Status = ASclose (FD)

15. Script Library Implementation

16. Script Library Implementation • Each script is linked with the Active Script Library which implements the Script API. • ASContentServer: To know if the script is executing on the active server or the content server. Must be used before any of the others API calls are. • ASopen: “Filename” specified is w.r.t local path on content server. But on active server, the path is not known. To find out the local path, the Script Library sends a request to the active server, with a special prefix. • E.g.:ASopen (“input.txt”, O_RDONLY, EXPIRES, 60, 120) Translates to:GET http://tarek8.cs.virginia.edu:8080/cgi-data/input.txt • Special prefix: “cgi-data” – indicates to the active server that it must fetch the file, cache it and in the reply include the local path to that file. • Reply contains a Cache-Control header extension which indicates the local path: Cache-Control: file-name=<local path> • Active cache rewrites the “/cgi-data/input.txt” path to “/cgi-bin/input.txt” by default, i.e. all data files are assumed to be in a path w.r.t to the /cgi-bin/ directory on the content server.

17. Script Library Implementation • The fetched data files are also cached on the active server. • Each data file is registered as (ScriptURL, DataURL) with the Consistency Manager when it is fetched (ScriptURL = URL of script using the data file) • The Consistency Manager then handles the job of maintaining the consistency of that data file by periodically fetching it from the content server. • ASread: Read the local file. • ASwrite: Write a temporary file. • ASclose: Close the file. Delete it also if it’s a temporary file • Example script: ASContentServer (“tarek7.cs.virginia.edu”, 8080); FD = ASopen (“1.txt”, O_RDONLY, EXPIRES, 60, 420); Count = ASread (FD, buffer, 10); ASclose (FD);

18. Active Server and its Components

19. Active Server and its Components • URL rewriter, Script Library • Script Execution Module: When a request for a script is received, check if it is in the script cache • Yes: Check if valid (i.e. not expired – default expiry 72 hours) • No: Fetch it from the content server and cache it. • Yes: Check if the script needs to be re-executed by querying the consistency manager (script will need to be re-executed if any of the data it uses has changed) • Yes: Re-execute script and send new results • No: Reply with a 304 (HTTP NOT_MODIFIED) code. • No: Fetch it from the content server and cache it.

20. Consistency Manager • Accepts two kinds of messages: • Notification messages – new (ScriptURL, DataURL) • Query messages – should ScriptURL be re-executed? • Accepts multiple requests in parallel (multi-threaded) • Has a single timer thread that times out at periodic intervals and fetches data files at every timeout • A data file is added to this timer queue based on the consistency it requires. • Spawns a separate-thread to handle each data file fetch. • Each data file fetch is an HTTP IF_MODIFIED_SINCE request – the content server replies with the new data and its last modified time or a 304 NOT_MODIFIED

21. Consistency Manager • It is difficult to provide strong consistency since the data is not cached on the server where it is generated and it would mean forwarding every request over the Internet to the server • So we provide weak consistency, but within server specified bounds, while trying to improve the client-perceived response time (i.e. timeliness) • Server specifies minConsistency and maxConsistency required • E.g: minC = 2 minutes, maxC = 5 minutes for stock data • The policy is used to decide the consistency provided to a particular data file (and therefore its fetch period) • Can use various consistency policies. The policy can be easily replaced – all that needs to be changed is one function.

22. Consistency (QoS) Model • A sample consistency Model we have implemented is presented here (based on K.D.Kang’s PhD proposal) • We frame the consistency policy as a timeliness vs. consistency trade-off problem where consistency is sacrificed (in a bounded manner) to improve timeliness of response. We present it as a QoS optimization problem: • We define the rewards as:

23. Factors affecting QoS • Internet backbone delay – measured as: • Active cache utilization • Network Bandwidth

24. QoS Management Scheme • Consistency is managed by periodically polling the content server for the status of the data. • The consistency level is adjusted dynamically as various factors change. • Polling Period Piis: • More specifically if polling is performed by timing out at constant intervals (t), then:

25. QoS Management steps • At every timeout, the steps taken are: • Check for CPU overload • Check for backbone overload • Fetch data files and cache • If CPU overload, current CPU utilization (U) > utilization threshold ( ). So degrade Ciof Nuscripts (out of N scripts) by a constant Ku:

26. QoS Management steps • If no CPU overload, threads spawned to fetch data files from content server if changed. • At each fetch, available network bandwidth on outgoing link (B) is monitored. If B < B (threshold), then degrade Ci for Nb data files (out of nq remaining in current timeout queue) by a constant Kb:

27. QoS Management Steps • If no bandwidth overload too, data file is fetched and its backbone network delay (d) is measured • Let d = dcurr – dprev then: where, Kd is a constant • Everytime Ci changes, Pi is recalculated and the data file is added to the appropriate timer queue

28. Evaluation • Test setup: one content server, one active cache and multiple clients • Content server and active cache both are: 700MHz Pentium IIIs with 256 MB RAM running Linux kernel 2.2.12 • Clients are: 700MHz Pentium IIIs with 64MB RAM running Linux kernel 2.2.12 • Clients and active cache are connected by a 100Mbps Ethernet link • Nominal Internet backbone delays are simulated in between the active cache and server

29. Evaluation • Two sets of Internet backbone delays are simulated: (25-75 ms) and (200-250 ms). • The clients are automatic URL generators (SURGE) which generate about 100,000 requests for 2000 scripts in about 10 minutes. • The data on the content server is classified into two classes: • Strict consistency bounds (60s min, 120s max) • Flexible consistency bounds (60s min, 420s max)

30. Evaluation • System is tested by: • Sending requests directly to the content server • Sending requests through the active cache • Three results: • Average latency reduction • Response time for a single script • Consistency tradeoff

31. Average Latency Reduction

32. Response time for a single script

33. Consistency Tradeoff

34. Conclusions • Novel architecture for active content caching • Can be deployed easily on the current Internet – works with components already in use • Consistency management in a user-specified bounded manner • Improves scalability of content servers

35. Future Work • Current support only for read-only scripts • Security issues • Possible Platform inconsistencies • Tie up with database backend on content server