Cataclysm: Policing Extreme Overloads in Internet Applications

1. Cataclysm: Policing Extreme Overloads in Internet Applications Bhuvan Urgaonkar and Prashant Shenoy University of Massachusetts

2. Motivation • Internet applications used in a variety of domains • Online banking, online brokerage, online music store, e-commerce • Internet usage continues to grow rapidly • Broadband deployment is accelerating • Outages of Internet applications more common “Site not responding” “connection timed out”

3. Internet Application Outages Holiday Shopping Season 2000: Down for 30 minutes Periodic outages over 4 days Average download time ~ 260 sec 9/11: site inaccessible for brief periods Cause: Too many users leading to overload

4. Internet Data Centers • Internet applications run on data centers • Server farms • Provide computational and storage resources • Applications share data center resources • Problem: How can the platform handle extreme overloads seen by applications?

5. Handling Extreme Overloads • Existing work is based on three approaches • Request policing [Kanodia00, Li00, Verma03, Welsh03, …] • Dynamic capacity provisioning [Chase01, Ranjan04] • Degrade performance of admitted requests [Abdelzaher99] • Shortcomings of existing work: • Does not attempt to integrate these three approaches • Does not address scalability of the policer! • The policer itself may become the bottleneck during overloads

6. Our Contribution: Cataclysm • Comprehensive approach • Novel policer that can scale during overloads • Dynamic provisioning for both application and policer • SLA-based performance adaptation • Implementation and evaluation on a Linux cluster • Focus of this talk: design of the policer

7. Talk Outline • Motivation • Internet data center model • Request policing • Cataclysm Server Platform • Experimental results • Summary

8. Data Center Model Retail Web site streaming • Dedicated hosting: each application runs on a subset of servers in the data center • Subsets are mutually exclusive: no server sharing • Data center hosts multiple applications • Free server pool: unused servers

9. Internet Application Model load balancing sentry • Internet applications replicated on multiple servers • E.g., clustered HTTP • Each application employs a sentry • Load balancing and request policing • One or more request classes • Service-level agreement • Specifies certain guaranteed request admission rate per class • Specifies allowed degradation in response time with arrival rate requests dropped requests http

10. Talk Outline • Motivation • Internet data center model • Request policing • Cataclysm Server Platform • Experimental results • Summary

11. Policer: Design Goals • Class-based differentiation • Each class should sustain its guaranteed admission rate • Revenue maximization • Challenging due to online nature of the problem • An admitted request may cause a more important request arriving later to be dropped • Approach: Preferential admission to higher class requests • Scalability • The policer should remain operational even under extremely high arrival rates

12. Overview of Policer Design Admission control dgold • Cataclysm policer has three components • Request classifier and per-class leaky buckets • Class-specific queues • Admission control Class gold admitted dsilver Class silver Classifier dropped dbronze Class bronze Leaky buckets Class-specific queues

13. Class-based Differentiation Admission control dgold Class gold admitted dsilver Class silver Classifier dropped dbronze Class bronze Leaky buckets Class-specific queues • Each incoming request undergoes classification • Per-class leaky buckets used to ensure that rates guaranteed in SLA are admitted

14. Revenue Maximization Admission control dgold Class gold admitted dsilver Class silver Classifier dropped dbronze Class bronze Leaky buckets Class-specific queues • Idea: Add different delays in processing of requests of different classes • More important requests processed more frequently • Methodology to compute delay values in online manner • Bounds probability of a request denying admission to a more important request

15. Admission Control Admission control dgold Class gold admitted dsilver Class silver Classifier dropped dbronze Class bronze Leaky buckets Class-specific queues • Goal: Ensure that an admitted request meets its response time target • Measurement-based admission control algorithm • Use information about current load on servers and estimated size of new request to make decision

16. Scalability of Admission Control • Idea #1: Reduce the per-request admission control cost • Admission control on every request may be expensive • Bursty arrivals during overloads => batches get formed • Delays for class-based differentiation => batches get formed • Admission control test that operates on batches instead of requests • Idea #2: Sacrifice accuracy for computational overhead • When batch-based processing becomes prohibitive • Threshold-based scheme • E.g., Admit all Gold requests, drop all Silver and Bronze requests • Thresholds chosen based on observed arrival rates and service times • Extremely efficient • Wrong threshold => bad response times or fewer requests admitted

17. Scaling Even Further … • Protocol processing overheads will saturate sentry resources at extremely high arrival rates • Indiscriminate dropping of requests will occur • Important requests may be turned away without even undergoing the admission control test • Loss in revenue! • Sentry should still be able to process each arriving request! • Idea: Dynamic capacity provisioning for sentry • Pull in an additional sentry if CPU utilization of existing sentries exceeds a threshold (e.g., 90%) • Round-robin DNS to load balance among sentries

18. Talk Outline • Motivation • Internet data center model • Request policing • Cataclysm Server Platform • Experimental results • Summary

19. Cataclysm Server Platform • Prototype data center • 20 Pentium servers • Gigabit switches • Linux-based platform • Sentry implemented in Layer-7 switch • Linux module ktcpvs • Replicated Web server applications using Apache • Dynamic content using PHP

20. Fraction admitted 1 0.8 Fraction admitted Gold 0.6 Silver 0.4 Bronze 0.2 0 0 100 200 300 400 500 Time (sec) Class-based Differentiation • Three classes of requests: Gold, Silver, Bronze • Policer successful in providing preferential admission to important requests

21. Threshold-based: Higher Scalability • Threshold-based processing allows the policer to handle upto 4 times higher arrival rate • Single sentry can handle about 19000 req/s

22. Threshold-based: Loss of Accuracy • Higher scalability comes at a loss in accuracy of admission control • Occasional violations of response time targets

23. Arrical rate 50000 40000 30000 Total arrival Arrival rate (req/s) Arival at sentry 1 20000 10000 0 0 100 200 300 400 500 600 Time (sec) Sentry Provisioning

24. Summary • Cataclysm: a comprehensive overload management technique consisting of • Request policing • Dynamic capacity provisioning • SLA-based performance adaptation • Cataclysm achieves the following • Class-based differentiation • Revenue maximization • Ability to scale to extreme overloads • More information: http://lass.cs.umass.edu

25. Policing and Provisioning

26. Policing and Provisioning