1 / 59

for Web based System

2006.02.01 http://www.javaservice.com. for Web based System. Mathematical Approach on. Performance Analysis. WonYoung Lee. Question: What is the definition of Performance?. A.jsp. B.jsp. http://192.168.0.2/a.jsp. http://192.168.0.2/b.jsp. Unit Response time 0. 5 seconds.

avye-summers
Download Presentation

for Web based System

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. 2006.02.01 http://www.javaservice.com for Web based System Mathematical Approach on Performance Analysis WonYoung Lee

  2. Question: What is the definition of Performance? A.jsp B.jsp http://192.168.0.2/a.jsp http://192.168.0.2/b.jsp Unit Response time 0.5 seconds Unit Response time 4.0 seconds

  3. A.jsp B.jsp http://192.168.0.2/a.jsp http://192.168.0.2/b.jsp Unit Response Time 0.5 seconds Unit Response Time 4.0 seconds Application A: Response Time Grahp 10 Response Time(sec) 5 1 5 10 15 20 25 Concurrent Users

  4. A.jsp B.jsp http://192.168.0.2/a.jsp http://192.168.0.2/b.jsp Unit Response Time 4.0 seconds Unit Response Time 0.5 seconds Application B: Response Time Graph 10 Response Time(sec) 5 5 10 15 20 25 Concurrent Users

  5. Comparison of A and B A 10 Response Time(sec) 5 B 5 10 15 20 25 Concurrent Users

  6. 1. Objective of Performance 1.How many Clients 2.Reasonable response time 3.How to test or measure

  7. 1.1 Visit time & Think time 1. Definition Click ! Click ! Click ! Click ! Click ! Click ! New User Visited Leave Visit Time Click ! Click ! Response Time Think Time Request Interval Request Interval(sec) = Response Time(sec) + Think Time(sec)

  8. 1. Definition 1.2 Think time ThinkTime look like a constant value for the specific business domain Click ! Click ! Response Time Think Time Request Interval

  9. 1.3 Concurrent User 1. Definition Request Interval( ≈Think Time) Visit Time Definition Different IP Addresses Max Thinktime Concurrent User = ActiveUser + Inactive User Concurrent Users : 6 Time Note: HTTP is connection-less Note: How many active users?

  10. 1.4 Active User 1. Definition Concurrent User Response Time Time Think Time Worker-Thread Active Service Request Interval System Click Click Active User

  11. number of request Throughput(tps) = measure time(sec) 1.5 Throughput 1. Definition Unit : tph, tpm, tps, (pps, rps, ops, hit/sec) 3,600 tph = 60 tpm = 1 tps Definition Time Measure Time NOTE: Arrival Rate, Service Rate

  12. 2. Request/Response System Model 1.Mathematical Approach 2.Queuing Theory 3.Quantitative Analysis 4.Measuring

  13. 2. Request/Response System Model 2.1 Request and Response Average Response Time Assumption: No ThinkTime Ave. Resp. Time # of Test Users Throughput Question: What is the ability of her ? # per minutes Right Answer: Maximum throughput # of Test Users

  14. 2. Request/Response System Model 2.2 MeasuringThroughput Increasing Response Time Virtual User (Assumption: ThinkTime=0) Throughput(tps) ResponseTime(sec) Virtual User(ThinkTime=0)

  15. 2.3 Little’s Law 2. Request/Response System Model Response Time(sec) Throughput(tps) T ResponseTime(sec) R ActiveUser(ThinkTime=0) N ActiveUser N = T x R Number of ActiveUser Throughput(tps) = Average Response Time(sec) Number of ActiveUser = Throughput(tps) x Average Response Time(sec)

  16. 2.4 ActiveUser’s Law 2. Request/Response System Model Concurrent User Response Time Think Time Active User Worker-Thread Active Service Request Interval System Click ResponseTime’s Law Ave. ResponseTime(sec) ActiveUser = ConcurrentUser x Ave. ResponseTime(sec) + ThinkTime(sec)

  17. 2. Request/Response System Model 2.4.1 Proff of ActiveUser’s Law ActiveUser Throughput(tps) = Little’s Law Resp.Time(sec) ConcurrentUser Throughput(tps) = Request Interval(=Resp.Time+ThinkTime) ActiveUser ConcurrentUser = Resp.Time(sec) Request Interval(=Resp.Time+ThinkTime) Resp.Time(sec) ActiveUser = ConcurrentUser x Request Interval(=Resp.Time+ThinkTime)

  18. 2. Request/Response System Model 2.4.2 Meaning of Active User -1- Ave. ResponseTime(sec) ActiveUser = ConcurrentUser x Ave. ResponseTime(sec) + ThinkTime(sec) Service Rate Request Rate Active User

  19. 2. Request/Response System Model 2.4.3 Meaning of Active User -2-

  20. 2. Request/Response System Model 2.5 Concurrent User Equation Definition Resp.Time(sec) ActiveUser = ConcurrentUser x Request Interval(=Resp.Time+ThinkTime) ActiveUser Throughput(tps) = Little’s Law Resp.Time(sec) Equations Resp.Time + ThinkTime ConcurrentUser = ActiveUser x ResponseTime ThinkTime ① = ActiveUser x ( 1 + ) ResponseTime ② = ActiveUser + ( Throughput x ThinkTime) ③ = Throughtput x ( ResponseTime + ThinkTime) (Note: Throughput(tps) : ArrivalRate or ServiceRate)

  21. 2. Request/Response System Model 2.6 Example of Concurrent User Monitoring • Permanent Cookie • Thinktime • Arrival Rate • Responsetime • Active User(Service) • - Concurrent User

  22. 2. Request/Response System Model 2.7 SLA & ThinkTime SLA(Service Level Agreement) - Concurrent User : 5,000 - Average Response Time: less than 3 sec ThinkTime Agreement - Tele-market : 10-15sec - MIS Intranet : 15-20 sec - Internet Banking : 25-35 sec - Online Shopping Mall : 30-40 sec - Community : more longer + Real ThinkTime Data SLA(Service Level Agreement) - Concurrent User : 5,000 - Think Time = 30 (Example) - Average Response Time: less than 3 sec

  23. 2. Request/Response System Model 2.8 Performance TestThinkTime = 30 Throughput Graph 235.3 Response Time Throughput(tps) 158.7 AverageResponseTimeGraph 4.0 1.5 5,000 8,000 Virtual User(ThinkTime=30  ConcurrentUser) ConcurrentUser = Throughtput x { ResponseTime + ThinkTime(30) } 5000 = 158.7 x ( 1.5 + 30 ) 8000 = 235.3 x ( 4.0 + 30 )

  24. 2. Request/Response System Model 2.9 Performance TestThinkTime=0 Throughput Graph 235.3 Average Response Time Graph ResponseTime Throughput(tps) 158.7 4.0 1.5 941 238 Virtual User(ThinkTime=0  ActiveUser) ConcurrentUser 5,000 8,000 ConcurrentUser = ActiveUser + Throughput(tps) x ThinkTime(sec) ThinkTime(sec) ConcurrentUser = ActiveUser x { 1 + } Resp.Time(sec) ConcurrentUser = Throughput(tps) x { Resp.Time(sec)+ThinkTime(sec)}

  25. 2. Request/Response System Model 2.10 Queuing Theory - G/G/1 Max Throughput Arrival Rate Throughput (λ<μ) λ μ (λ<μ) ActiveUser Utilization Response Time

  26. 2. Request/Response System Model 2.11 Maximum Concurrent User ActiveUser Response Time Arrival Rate(λ) Arrival rate Arrival rate req/sec Response Time(R) Active User(N) Concurrent User ? ? Concurrent User Concurrent User

  27. 2. Request/Response System Model 2.12 Saturation Point, Buckle Zone

  28. 2. Request/Response System Model 2.13 Understanding of Throughput Graph Tuning ? What’s mean?

  29. 2. Request/Response System Model 2.14 Throughput and Active User Demo

  30. 2. Request/Response System Model 3. Multiple Applications • - Different Hit Ratio • - Different Performance • Homogeneous/heterogeneous • Bottleneck Condition

  31. 3. Multiple Applications 3.1 2-Application model A application B application TPSmax TPS TPS 40 40 40 20 20 20 Saturation Point Saturation Point 40 Active Threads 40 Active Threads A B 1 : 1 3 : 1 1 : 3 TPS TPS TPS 40 40 40 A 30 A B 20 15 B 10 10 B 5 A 30 10 40 Active Threads 10 30 40 Active Threads 20 40 Active Threads TPSmax TPSmax TPSmax 40 40 40 30 20 20 15 10 10 5 A B A B A B Graph 1 Graph 2 Graph 3

  32. 3. Multiple Applications 3.1.1 2-Applicatoin Model A 40 Different Arrival Rate graph 2 30 graph 1 20 graph 3 10 Limited by same resource bottleneck  Homogeneous bottleneck condition O 10 20 B

  33. 3. Multiple Applications 3.1.2 2-Applicatoin Model A: r1 (req/sec) B: r2 (req/sec) y T2 y x = r2 r1 R2 Q (r1,r2) y x r2 P = 1 + T1 T2 x O T1 r1 R1

  34. 3. Multiple Applications 3.2 3-Application Model z (0, 0, T3) T3 y z x = = r2 r3 r1 Q P (r1, r2, r3) r3 (0, T2, 0) (0, 0) O r2 y T2 r1 y x z T1 = 1 + + T2 T1 T3 (T1, 0, 0) x

  35. 3. Multiple Applications 3.3 n-Applicatoin Model ( Critical Inequality Performance Equation ) NOTE: Under Homogeneous Bottleneck Condition

  36. 3. Multiple Applications 3.4 Performance Utilization z T3 y x z = 1 + + T2 T1 T3 Q P 0.12 + 0.13 + 0.45 = 0.70 ∑ { ri/ Ti} x 100 = 70 % y T2 Utilization of Critical Performance T1 ∑ { ri/ Ti} ≤ 1.0 x

  37. 3. Multiple Applications 3.5 Revision of Utilization of Critical Performance Arrival Rate a.jsp: 5.4 req/sec b.jsp: 1.9 req/sec c.jsp: 10.7 req/sec d.jsp: 4.5 req/sec ….. ∑ { ri/ Ti}≤ρ* 1.0(ρ is total hit ratio of ∀ai)  ∑ { ri/ Ti} / ρ=∑ { ri/ Ti} / (22.5/ 25) = 88%

  38. 3. Multiple Applications 3.6 Multiple Bottleneck Theory z Bottleneck surface Arrival rate line y z x = = r2 r3 r1 Q P r3 (r1, r2, r3) bottleneck saturation flat (0, 0) r2 O Critical Inequality Performance Equation (final) ∑ { ri/ Ti} ≤ 1.0 +ε(ε>0) x Heterogeneous bottleneck condition

  39. 3. Multiple Applications 3.7 Performance Matrix NOTE: 8:2 Rule ∑ λi/Τi ∑ λi/Τi ≤ρ(1.0+ε)(ε≥0) (ρ is total hit ratio of ∀ai) Critical Performance Utilization: ≤1.0 ρ(1.0+ε) (When ConcurrentUser = N) (ε≈ 0)

  40. 3. Multiple Applications 3.8 Objective of Performance Estimation How many concurrent users can be accepted? How many times than now? Concurrent Users Arrival Rate kN kλ Future Future N Current Current λ ∑ λi/Τi xk Current Performance Utilization : 1.0 (100%) ρ(1.0+ε)

  41. 3. Multiple Applications 3.9 Performance Test for Multiple Scenario ThinkTime = 28.5 sec(Example) Response Time Graph Acceptable Response Time 942 Virtual User(ThinkTime = 28.5 sec)

  42. 3. Multiple Applications 3.10 Example of Performance Test Result of Test using each application test. Maximum adaptable concurrent user: 219 Expected maximum TPS : 6.515(tps) Result of Test using Multiple Scenario at the same time Maximum adaptable concurrent user: 202 Expected maximumTPS : 6.719 (tps) 1 + 1 = 2.5 ?

  43. 3.11 Termination about Performance Test Load Test Stress Test Availability Test Performance Test

  44. Two Types of Performance Problem • Relative Performance Problem • SQL Query bottleneck(DB index, Full Scan, Heavy Query) • Bottleneck on back-end transactions(CICS,TUXEDO, TCP/IP Socket) • Relative bad performance on specific application(synchronized, CPU time) • Relative bad performance on most of application caused by too small H/W • Other side issue, for example, Network bottleneck. • Conditional Performance Problem • JDBC Connection resource leakage • Memory leakage(need too large memory, Memory Leak, Native Memory Leak) • Unbalanced WAS tuning(Pool Size, Number of Thread, Heap Size) • Caused by a bug on JVM/WAS/JDBC (Sybase JDBC, JVM Bug, WAS Bug) • Thread Lock/Dead Lock (Application issue, Firewall issue)

  45. Relative Performance Problem Request Rate Service Rate Active User

  46. Relative Performance Problem: Ramp up Test

  47. Relative Performance Problem: Ramp up Test

  48. Conditional Performance Problem • JDBC Connection resource leakage • Memory leakage(need too large memory, Memory Leak, Native Memory Leak) • Unbalanced WAS tuning(Pool Size, Thread 개수, Heap Size) • Caused by a bug on JVM/WAS/JDBC (Sybase JDBC, JVM Bug, WAS Bug) • Thread Lock/Dead Lock (Application/Framework issue, Firewall issue) • Database lock caused by uncommited nor unrollbacked • database issue(buffer full, unexpected Batch Job, ..) • upload or download a large file • unexpected infinite loop on an application: CPU 100% • Disk/memory Full • bad performance on specific application or specific usres

  49. 3.1 Issue on Capacity Planning 3.1.1 K-university KMS System S80(12-way,8GB, 78,126 tpmC) S80(6-way,6GB, 41,140 tpmC) Visitors a day: 4,506 users PeakConcurrent users: 275 users Peak Arrival Rate : 18.3 tps Request Interval : 18 sec Vistors vs Concurrent users: 6.1% CPU : ?(unkown, no issue) 119,266tpmC/18.3tps = 6,517 tpmC/tps

  50. 3.1.2 B company M80 (4-way, 3GB, 34,588 tpmC) x 2 S80 (12-way, 8GB, 78,126 tpmC) x 2 6F1 (4-way, 4GB, 44,500 tpmC) x 1 H80 (2-way, 1GB) x 8 [Seoul:M80(4-way)x2: 69,176 tpmC ] Visitors a day: 1,621(total 3,435) PeakConcurrent Users: 600 users Hits a day : 466,639 hits Peak Arrival Rate : 20.0 tps Request Interval : 30 sec Average Visit time: 1:25:43 Visitors vs Concurrent Users: 31% CPU Utilization: 70-100% 69,176tpmC/20tps= 3,459 tpmC/tps 6F1(4-way,4GB, 44,500 tpmC )Added. CPUUtilization: 60% 113,676tpmC/20tps= 5,684 tpmC/tps

More Related