1 / 44

Internet Applications: Performance Metrics and performance-related concepts

Internet Applications: Performance Metrics and performance-related concepts. E0397 – Lecture 2 10/8/2010. Generic Internet Service. Client. User (Client). Service Provider (Server systems) Assume no “cloud”. Server. Performance Concerns. User/Client: Response time Blocking

josephjwilliams
Download Presentation

Internet Applications: Performance Metrics and performance-related concepts

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Internet Applications:Performance Metricsand performance-related concepts E0397 – Lecture 2 10/8/2010

  2. Generic Internet Service Client User (Client) Service Provider (Server systems) Assume no “cloud” Server

  3. Performance Concerns • User/Client: • Response time • Blocking • Service Provider/Server • Assuring good performance to client • High volume of usage (many clients) • Keeping costs low

  4. Service Provider’s concerns Determining server, host and network architecture How will the network affect the performance? (LAN vs WAN) How many machines? Machine configuration? (how many CPUs, what speed, how many disks?) On which machines should IMAP server be deployed? The Web Server? What should be the configuration of the Web server? (Number of threads, buffer size,…) Resource sizing to meet user performance expectations

  5. Where’s the conflict? (in other words, where is the engineering problem?) • Clients’ performance needs and Service provider’s needs work against each other • Universal Law of Resource Usage and: • Heavy Usage by multiple entities Degraded performance • Queuing Theory allows looking at systems which have resources under contention by multiple entities, in a formal manner • Allows prediction of performance under various system parameters, by using mathematical models

  6. What/Why is a Queue? • The systems whose performance we study are those that have some contention for resources • If there is no contention, performance is in most cases not an issue • When multiple “users/jobs/customers/ tasks” require the same resource, use of the resource has to be regulated by some discipline

  7. …What/Why is a Queue? • When a customer finds a resource busy, the customer may • Wait in a “queue” (if there is a waiting room) • Or go away (if there is no waiting room, or if the waiting room is full) • Hence the word “queue” or “queuing system” • Can represent any resource in front of which, a queue can form • In some cases an actual queue may not form, but it is called a “queue” anyway.

  8. Examples of Queuing Systems • CPU • Customers: processes/threads • Disk • Customers: processes/threads • Network Link • Customers: packets • IP Router • Customers: packets • ATM switch: • Customers: ATM cells • Web server threads • Customers: HTTP requests • Telephone lines: • Customers: Telephone Calls

  9. Elements of a Queue Queueing Discipline Service time Customer Inter-arrival time Waiting Room/ Buffer/ Queue Server

  10. Elements of a Queue • Number of Servers • Size of waiting room/buffer • Service time distribution • Nature of arrival “process” • Inter-arrival time distribution • Correlated arrivals, etc. • Number of “users” issuing jobs (population) • Queuing discipline: FCFS, priority, LCFS, processor sharing (round-robin)

  11. Elements of a Queue • Number of Servers: 1,2,3…. • Size of buffer: 0,1,2,3,… • Service time distribution & Inter-arrival time distribution • Deterministic (constant) • Exponential • General (any) • Population: 1,2,3,…

  12. Queue Performance Measures • Queue Length: Number of jobs in the system (or in the queue) • Waiting time (average, distribution): Time spent in queue before service • Response time: Waiting time+service time • Utilization: Fraction of time server is busy or probability that server is busy • Throughput: Job completion rate

  13. Queue Performance Measures • Let observation time be T • A = number of arrivals during time T • C = number of completions during time T • B = Total time system was busy during time T • Then: • Arrival Rate =  = A/T • Throughput = C/T • Utilization = ρ = B/T • Average service time = = B/C • Service rate = 1/ = m

  14. Basic Relationships • In “steady-state” for a stable system without loss (i.e. infinite buffer system) • Completionrate = Arrival Rate, since “in-flow = out-flow”) • If arrival rate > service rate, then  = 1/ • Utilization =  • B/T = (B/C) x (C/T) = Average Service Time x Completion Rate =  = for a loss-less system. • For loss-full systems, if p = fraction of requests lost, • (1 – p) • Throughput of a system,  = Utilization x Service Rate =  m = • (C/T) = (B/T) x (C/B)

  15. Little’s Law: N =  R • Average number of customers in a queuing system = Throughput x Average Response Time • Applicable to any “closed boundary” that contains queuing systems • Some other assumptions • Also, if L is the number in queue (not in service), and W is waiting time: • L = LW

  16. Simple Example (Server) • Assume just one server (single thread) • Requests come in @ 3 requests/second • Request processing time = 250 ms. • Utilization of server? • 75% • Throughput of the server? • 3 reqs/second • What if requests come in 5 reqs/second? • Utilization = 100%, Throughput = 4 reqs/second • Waiting time (for 3 reqs/second?) • L/3, where L is queue length. But what is L? Need Queuing Model

  17. Classic Single Server Queuing model : “M/M/1” • Exponentially distributed service time • Exponentially distributed inter-arrival time • Single Server • Infinite buffer (waiting room) • FCFS discipline Can be solved very easily , using theory of Markov chains

  18. Exponential Distribution • Memory-less distribution • Distribution of remaining time does not depend on elapsed time • Mathematically convenient • Realistic in many situations (e.g. inter-arrival times of calls) • X is EXP()  P[X < t] = 1 – e-t • Average value of X = 1/

  19. Important Result! M/M/1 queue results • Let  be arrival rate and  be service time, and  = 1/ be service rate • Utilization • Mean number of jobs in the system • /(1-) • Throughput •  • Average response time (Little’s law): • R = N/

  20. Response Time Graph This region, after which there is a sharp growth is often termed “knee of the curve”. Note that it is not a very well defined point • Graph illustrates typical behaviour of response time curve

  21. Important quantity: termed traffic intensity or offered load Multiple server queue: M/M/c • One queue, c servers • Utilization,  =c • a= Average number of busy servers. • Queue length equation exists (not shown here) • For c = 2, queue length is: 2 -  • Average Response Time? • Little’s Law! For c = 2, R = N/ = t - 

  22. Finite Buffer Models: M/M/c/K • c servers, buffer size K (total in the system can be c+K) • If a request arrives when system is full, request is dropped • For this system, there will be a notion of loss, or blocking, in addition to response time etc. • Blocking probability (p) is probability that arriving request finds the system full • Response time is relevant only for requests that are “accepted”

  23. ...Finite Buffer Queues • Arrival rate: l • Service rate:m • Throughput? • L = l (1 – p) • Utilization? • L t/c • Blocking probability? Queue length (N)? • Formula exists (from queuing model) • Waiting time? (Little's law = N/L)

  24. Finite Buffer Queue:Asymptotic Behavior • As offered load increases (l  infinity) • Utilization (r) • r  1 • Throughput • L  cm • Blocking probability • p  1 • Queue length • N  c+ K • Waiting time • K/(cm) + 1/m

  25. Examples

  26. Example-1 • You are developing an application server where the performance requirements are: • Average response time < 3 seconds • Forecasted arrival rate = 0.5 requests/second • What should be the budget for service time of requests in the server? • Answer:  <1.2 seconds.

  27. Example-4: Multi-threaded Server • Assume multi-threaded server. Arriving requests are put into a buffer. When a thread is free, it picks up the next request from the buffer. • Execution time: mean = 200 ms • Traffic = 30 requests/sec • How many threads should we configure? (assume enough hardware). • Traffic Intensity = 30 x 0.2 = 6 = Average number of busy servers •  At least 6 • Response time = (Which formula?)

  28. ...Example-4 • Related question: estimate average memory requirement of the server. • Likely to have: constant component + dynamic component • Dynamic component is related to number of active threads • Suppose memory requirement of one active thread = M • Avg. memory requirement= constant + M* 6

  29. Example-5: Hardware Sizing • Consider the following scenario: • An application server runs on a 24-CPU machine • Server seems to peak at 320 transactions per second • We need to scale to 400. • Hardware vendor recommends going to 32 CPU machine. • Should you?

  30. Example-5: Hardware Sizing • First do bottleneck analysis! Suppose logs show that at 320 transactions per second, CPU utilization is 67% - What is the problem? What is solution?

  31. Example-5: Hardware Sizing • Most likely Explanation: Number of threads in server is < number of CPUs • Possible diagnosis: • Server has 16 threads configured • Each thread has capacity of 20 transactions per second • Total capacity: 320 reqs/second. • At this load, 16 threads will be 100% busy  average CPU utilization will be 16/24=67% • Solution: Increase number of threads – no need for CPUs.

  32. Closed Queuing Systems

  33. Server Clients Typical Scenario • M clients issue requests, wait for response, then “think”, then issue request again • Let 1/l be mean think time, 1/m be mean service time. • Scenario is actually of “closed” queuing network

  34. Server Clients Observations • Throughput? Arrival rate? • At steady-state, “flow” through both queues is equal • Server throughput = Server utilization X Service rate • U X m • Request is generated by each user once every [think time + response time] = 1/ (1/l + R) • Overall request arrival rate = M / (1/l + R) = U *m

  35. Server Clients Observations • M / (1/l + R) = U *m = Throughput • Response time = number of clients/Throughput – think time • As number of clients increase, can be shown to tend to: M/m – 1/l. • Linear function of M • Increase M by 1  Response time increases by 1/m

  36. Metrics from model R M • Saturation number (number of clients after which system is “saturated”) • M* = 1 + m/l

  37. Queuing networks

  38. Queuing Networks • Jobs arrive at certain queues (open queuing networks) • After receiving service at one queue (i) , they proceed to another server (j), with some probability pij, or exit

  39. Open Queuing Network - measures Throughput • Maximum Throughput • Bottleneck Server

  40. Open Queuing Network - measures start finish • Total time spent in the system before completion (overall response time, from the point of view of the user)

  41. Open Queuing Network: Example 1 pA1w pwA1 D1 A1 pA1D1 l W A2 D2 pw0 pA2D2 pwA2 • W: Web Server • A1, A2: Application Server 1, 2 • D1, D2: Database Server 1,2 pA2w

  42. ...Open Queuing Network- Example 1 pA1w pwA1 pA1D1 l pw0 pA2D2 pwA2 pA2w • Each server has different service time • But what is the request rate arriving to each server? • Need to calculate this using flow equations

  43. ...Open Queuing Network- Example 1 pA1w pwA1 pA1D1 l pw0 pA2D2 pwA2 pA2w • Equations for Average number of visits before leaving the server network • vA1 = pwA1. vw+ vD1 , vD1 = pA1D1 . vA1 • vA2 =pwA2. vw + vD2 , vD2 = pA2D2 . vA2 • Vw = 1 + pA1w. vA1 + pA2w. vA2

  44. ...Open Queuing Network- Example 1 pA1w pwA1 pA1D1 l pw0 pA2D2 pwA2 pA2w • Spreadsheet calculation shows • How bottleneck server changes • How throughput changes • How response time changes • Results are accurate for only some types of networks, for others, they are approximate

More Related