1 / 33

Design and Evaluation of a Model for Multi-tiered Internet Applications

Design and Evaluation of a Model for Multi-tiered Internet Applications. Bhuvan Urgaonkar Internship project talk – Services Management Middleware Dept, IBM Aug. 20, 2004. Internet Data Centers. Internet applications run on data centers Server farms

chesna
Download Presentation

Design and Evaluation of a Model for Multi-tiered Internet Applications

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Design and Evaluation of a Model for Multi-tiered Internet Applications Bhuvan Urgaonkar Internship project talk – Services Management Middleware Dept, IBM Aug. 20, 2004

  2. Internet Data Centers • Internet applications run on data centers • Server farms • Provide computational and storage resources • Service-level agreements • Response time guarantees • Problem: Need good application models to determine right resource allocations

  3. requests Load balancing gateway database http J2EE Multi-tiered Applications • Internet applications: multiple tiers • Example: 3 tiers: HTTP, J2EE app server, database • Replicable components • Example: clustered HTTP, J2EE server

  4. Existing Application Models • Several models for single-tier apps • Queuing models for web servers: Chase et al (USITS 03), Chandra et al (IWQoS 2003) • PODC 2004: G/G/1 based model • Model only one (bottleneck) tier • Ranjan et al (IWQoS 2002), Villela et al (IWQoS 2004)

  5. Black-box Approach • Black-box approach • Treat application as a black-box • Measure response time from outside • Increase allocation if response time > SLA • Use a model to decide how much to allocate • Strawman #1: black-box for multi-tier apps • Problems: • Unclear which tier needs more capacity • Bottleneck tier may not be replicable

  6. Black-box Approach

  7. Extension of Single-tier Model • Strawman #2: use single-tier provisioning independently at each tier • Example: • Breakdown resp time into per-tier delays • Use G/G/1 model for each tier • Problems: • How to breakdown resp time? • G/G/1 based model found to be very conservative! • Wasted capacity

  8. Talk Outline • Motivation • Multi-tier Application Model • Preliminary Evaluation • Ongoing Work / Discussion • Summary

  9. Key Insights/Observations • Tier i requests service from tier (i+1) • Scheduling: PS closest tractable among policies • Session-based workloads: • A session consists of a succession of requests • Think times are user dependent • Quantity of interest: Per-request resp time requests Load balancing gateway database http J2EE

  10. Queuing-theoretic Model • Example: 3-tier application • Natural model: Network of queues shown below X_2 X_3 X_1 Z . . . • Capturing session-based workload • Infinite server system • Closed-queuing system

  11. Mean-value Analysis • MVA algorithm • Inputs: Avg service times, visit ratios • Computes avg. delays and resp. time E[Z] E[X_3] E[X_2] E[X_1] . . . Visit ratios V_0=1 V_2 V_1 V_3

  12. MVA Algorithm • Key relation: Am(N) = Qm(N-1) • Am : # customers an arriving customer finds in queue m • Qm : # customers in queue m for m = 1 to M do Qm = 0 for n = 1 to N do begin for m = 1 to M do Rm = Dmfor the infinite server Rm = Dm*(1+Qm) for other servers X = n / ΣRm ……… throughput for m = 1 to M do Qm = X*Rm …… Little’s Law end

  13. MVA Algorithm: Discussion • Can handle any service time distrib if scheduling discipline is PS • Extension to multiple classes exists • Need to measure service times and visit ratios on a per-class basis • Gives only averages, not distribs • Each queue is really only modeling a single resource

  14. Finding Model Parameters • Visit ratios • Easy to obtain from various logs • E.g. Apache-tomcat-mysql • V_apache = 2 • V_mysql = avg # queries per servlet • V_tomcat = V_mysql + 1

  15. Finding Model Parameters • Service times • Apache and Tomcat can be made to log time spent at and beyond them • X_apache • (T_apache–T_tomcat)/2 • X_tomat • (T_tomcat–V_mysql*X_mysql)/(V_mysql+1) • X_mysql • avg. query exec. time

  16. What we haven’t captured … • Inter-tier load balancers • Resources held at tier i while awaiting response from tier (i+1) • Increased service times at high loads • E.g. context switches, protocol processing, contention for locks • Tails of response times • Multiple resources • Load imbalances due to session affinity

  17. Test Applications • RUBiS (eBay like auction app) • BrowseItems, PutBid, AuthorizeBid, PutComment, RegisterUser, SellItem, SearchItem, StoreBid, … • RUBBoS (slashdot like b-board app) • AcceptStory, BrowseStories, ModerateComment, PostComment, RegisterUser, RegisterStory, ViewStory, …

  18. Experimental Setup • Rubis (e-auctions), Rubbos (b-board) • Apache, Tomcat, Mysql • Apache mod_jk redirector • Tiers 1 and 2 are replicable • Java client • Average think time = 1 sec • One thread per session Mysql Tomcat Apache+mod_jk Client

  19. RUBiS: Response Times • Model works well in a restricted region • Tomcat had a connection limit of 75

  20. RUBiS: CPU Utilizations • App tier is the bottleneck

  21. RUBiS: Processes and Conns.

  22. RUBBoS: Response Times • Again, works well in a restricted region

  23. RUBiS: DB-intensive workload • Replaced SELECT with SELECT SQL_NOCACHE • Database tier is bottleneck

  24. Query Caching at the Database • Able to capture effect of query caching at DB • Interesting to do: Caching at app tier • Reduced visit ratio at database

  25. Multiple Classes of Sessions • Class 1 : App server intensive • Class 2 : Database intensive

  26. Talk Outline • Motivation • Multi-tier Application Model • Preliminary Evaluation • Ongoing Work / Discussion • Summary

  27. Multiple Servers at a Tier Mysql Apache+mod_jk Client Tomcat Mysql Apache+mod_jk Client Tomcat

  28. rubis - 2 app servers 2500 2000 Obs at apache 1500 Obs at tomcat1 Avg resp time (msec) 1000 Obs at tomcat2 500 0 1 2 5 10 20 50 80 100 Num sessions Load Imbalance • Session affinity • Variable session requirements • Ongoing: Introduce a skew factor for adjusting the visit ratios to the servers in the replicated tier

  29. Applying the Model to more Apps/Implementations • EJB based implementations of RUBiS and RUBBoS • TPC-W • Workloads that stress resources other than CPU • More??

  30. Misc. Issues/Discussion • Investigate utility of the model in • Capacity planning • Dynamic provisioning • Admission control • Measurements • How many observations to gather? • Handling incr. svc times at higher loads • Context switching, locks, protocol processing, … • Response time tails

  31. Summary • Network of queues based model • Experimental evaluation for 2 apps • Model works well in limited operating regions • Simple enhancements to handle multiple classes, multiple servers, load imbalance • More work needed on several aspects • updating service times, how many observations, resp time tails, more apps and workloads

  32. Acknowledgements • Mike Spreitzer • Asser Tantawi • Giovanni Pacifici Thank you!

  33. Extension of Single-tier Model

More Related