An Analytical Model for Multi-tier Internet Services and its Applications

1. An Analytical Model for Multi-tier Internet Services and its Applications Bhuvan Urgaonkar, Giovanni Pacifici, Prashant Shenoy, Mike Spreitzer, Asser Tantawi University of Massachusetts and IBM TJ Watson

2. Internet Applications • Proliferation of Internet applications auction site online game online store • Growing significance in personal, business affairs • Focus: Modeling Internet applications

3. Why Model Internet Applications? • Capacity provisioning • How many servers does the application need? • Performance prediction • E.g., predict response time • Application configuration • Tune various application parameters • Request policing • Turn away excess requests during overloads

4. Internet Application Architecture queries search “moby” response Melville’s ‘Moby Dick’ Music CDs by Moby HTTP J2EE Database request processing in an online bookstore • Multi-tier architecture • Each tier uses services provided by its successor • Session-based workloads • Caching, replication

5. Existing Application Models • Models for Web servers [Chandra03, Doyle03] • Do not model Java server, database etc. • Black-box models [Kamra04, Ranjan02] • Unaware of bottleneck tier • Extensions of single-tier models[Welsh03] • Fail to capture interactions between tiers • Existing models inadequate for multi-tier Internet applications

6. Talk Outline • Motivation • Application Model • Evaluation of the Model • Dynamic Capacity Provisioning • Summary and Future Research

7. Baseline Application Model • Model consists of two components • Sub-system to capture behavior of clients • Sub-system to capture request processing inside the application clients application

8. Modeling Clients • Clients think between successive requests • Infinite server system to capture think time Z • Captures independence of Z from processing in application Z Client 1 Z Client 2 clients application Z Client N Q0

9. Modeling Request Processing p1 p3 pM=1 p2 S1 S2 SM Q1 Q2 QM N tier 1 tier 2 tier M • Transitions defined to capture circulation of requests • Request may move to next queue or previous queue • Multiple requests are processed concurrently at tiers • Processor sharing scheduling discipline • Caching effects get captured implicitly!

10. Putting It All Together p1 p3 pM=1 • A closed-queuing model that captures a given number of simultaneous sessions being served Z p2 S1 S2 SM client Z client Q1 Q2 QM Q0 N tier 1 tier 2 tier M

11. Model Solution and Parameter Estimation • Mean Value Analysis (MVA) Algorithm • Computes mean response time • Visit ratios • Equivalent to trans. probs. for MVA • Vi ≈ λi / λreq ; λreq at policer, λifrom logs • Service times • Use residence time Xi logged at tier i • For last tier, SM ≈ XM • Si = Xi – ( Vi+1 / Vi ) ·Xi+1 • Think time • Measured at the entry point of application

12. Talk Outline • Motivation • Application Model • Evaluation of the Model • Dynamic Capacity Provisioning • Summary and Future Research

13. 30000 Observed 25000 Basic Model 20000 Avg resp time (msec) 15000 10000 5000 0 0 100 200 300 400 500 Num sessions Evaluation of Baseline Model • Auction site RUBiS • One server per tier Apache JBOSS Mysql 75 150 • Concurrency limits not captured

14. Handling Concurrency Limits Z • Requests may be dropped due to concurrency limits • Need to model the finiteness of queues! S1 S2 SM Z Q1 Q2 QM Q0 N dropped requests

15. Handling Concurrency Limits Z • Approach: Subsystems to capture dropped requests • Distinguish the processing of dropped requests S1 S2 SM Z Q1 Q2 QM Q0 N drop Q1 QM drop pM drop p1 drop drop drop S1 SM

16. 30000 Observed 25000 Basic Model Enh Model 20000 Avg resp time (msec) 15000 10000 5000 0 0 100 200 300 400 500 Num sessions Response Time Prediction • Enhanced model can capture concurrency limits

17. Query Caching at the Database • Caching effects • Captured by tuning Vi and/or Si • Bulletin-board site RUBBoS • 50 sessions • SELECT SQL_NO_CACHE causes Mysql to not cache the response to a query • More model enhancements • Replication at tiers • Multiple session classes

18. Prototype Data Center • 40+ Linux servers • Gigabit switches • Multi-tier applications • Auction (RUBiS) • Bulletin-board (RUBBoS) • Apache, JBOSS (replicable) • Mysql database

19. Dynamic Capacity Provisioning • Auction application RUBiS • Factor of 4 increase in 30 min Server allocations Workload Response time • Server allocations increased to match increased workload • Response time kept below 2 seconds

20. Talk Outline • Motivation • Baseline Application Model • Evaluation of the Model • Dynamic Capacity Provisioning • Summary and Future Research

21. Summary and Future Work • Analytical model for Multi-tier Internet Applications • Mean-value analysis • Concurrency limits, replication, caching, multiple classes • Model validation using 3-tier applications • Dynamic provisioning, request policing • Future work • Handling load imbalances at replicated tiers • Handling more diverse workloads • Handling other kinds of scheduling disciplines at servers

22. Thank you! More information at: http://www.cs.umass.edu/~bhuvan