CSC407: Software Architecture Winter 2007 Performance Analysis I

CSC407: Software ArchitectureWinter 2007Performance Analysis I Greg Wilson gvwilson@cs.utoronto.ca

Introduction • Getting the right answer is important • Getting the right answer quickly is also important • If we didn’t care about speed, we’d do things by hand • Choosing the right algorithm is part of the battle • To search a number in a sorted array ? • Choosing a good architecture is the other part • Only way to tell good from bad is to analyze and measure actual performance

Performance Concerns • Key concern is quality of service (QoS) • Throughput • Transactions/sec • Pages/sec • Response time • And variation in response time • People would rather wait 10 minutes every day than 1 minute 9 days, and 20 minutes the tenth

…Performance Concerns • Availability • 99.99% available = 4.5 minutes lost every 30 days • That’s not good enough for 911

Queueing Theory • Analytic model of a computer system as: • Resources (like CPU, disk, etc.) • Jobs which require these resources • Derive and calculate performance measures like: • Throughput, response time, resource utilization, etc.

A Single Server Network Arriving Jobs • Circles show resources (CPU, disk, etc.) • Boxes show queues (jobs are waiting for resources) • Throughput and response times depend on: • Service demand: how much time do requests need from resources? • System load: how many requests are arriving per second? Queue Server

Model Parameters λ = 0.25 • Service Order • How the jobs will be served (e.g. First Come First Served) • Average Arrival Rate • The rate at which jobs arrive at the server (λ=1 job every 4 seconds) • Mean Interarrival Time = 1/ λ • Service Requirement (S): • Time for the job to finish on the server in isolation • Job dependent: job P may take 5 sec to finish whereas job R may take 8 sec. FCFS S

…Model Parameters μ = 0.2 λ = 0.25 • Mean Service Time: • Expected value of S (E[S]) is the average time required to service a job • 5 jobs require 4 sec each, 5 jobs need 6 sec => E[S] = 5 sec • Average Service Rate μ = 1/E[S] • Average rate at which the jobs are served. • For example given above: μ = 1 job/5 sec • For single server model, require λ ≥μ • Otherwise, queue grows infinitely long FCFS S

Performance Metrics Ns Nq μ = 0.2 λ = 0.25 • Response time, time in system (Ts) • The time a job spends in the system (tdept - tarrival) • Waiting time (Tw) • The time spent in the queue, waiting to get served • Ts = Tw + S, for a single job => E[Ts]= E[Tw] + E[S] • Number of jobs in System (Ns) • Total number of jobs in the system • Number of jobs in queue (Nq) • Total number of jobs in the queue Tw Ts

…Performance Metrics Busy Period: B Jobs Completed: C Total Time: T Server Busy • Utilization (Ui): • The fraction of time resource i is busy • Ui = Bi/Ti • Throughput (Xi): • The rate of job completions at resource i • Xi = Ci/Ti = (Ci/Bi)(Bi/Ti) = Ui/ E[S] (from previous definitions) Xi = Uiμi

Utilization Law • Utilization Ui = Bi/Ti • We have shown that: • Xi= ( 1/ E[S] ) Ui • Ui = Xi. E[S] • I.e., utilization is the throughput times the service time, which makes sense • Example: • Throughput = 3 jobs per minute • Service time required per job = 15 sec • Utilization ?

Example 1/ μ 1/ μ 1/ λ 1/ λ • What is the throughput of a single server network, if the arrival rate is λ, average service rate is μ? • Using Utilization law: X = μ U • What is U for this case? • U = total busy period/ total time = (average time required by a job)/(average interarrival time) = (1/ μ )/(1/ λ) = λ/ μ • X = μ (λ/ μ) = λ • It just depends on the arrival rate

Service Demand Law • Service demand Di is the total average time required per request from ith resource • Let C0be the total number of jobs completed in time T • Di = UiT/C0 • I.e., fraction of time busy, times total time, over number of requests • But UiT/C0 = Ui/(C0/T) = Ui/ X • I.e., service demand is utilization of resource i over throughput • Ui/X = (Bi/T)/(C0/T) = Bi/C0 = (Bi/Ci)(Ci/C0) =ViSi • I.e., service demand is average number of visits times mean service time

Little’s Law • N = Average number of jobs in the system • T = Average time for which a job stays in the system • λ = Average arrival rate • Little’s Law: N = λT • Average number of jobs in the system = arrival rate times the average time for which each job stays in the system

…Little’s Law • Example 1 • Arrival Rate = 5 requests per min • Time spent in the system = 2 sec • Number of requests in the system at any time = 52 = 10 • Example 2 • Time spent in the system = 3 sec • Number of requests at any time = 18 • Arrival rate = N / T = 18 / 3 = 6 requests per second

…Little’s Law • N(t) = Number of jobs in the system at time t • E[N] = Average number of jobs in the system • i.e., Average value of N(t): Can be calculated from the figure. • T = Average time for which a job stays in the system • Number of jobs in time t = λ t • Area of the shaded region = E[N]t = (λ t) T => E[N] = λ t

Little’s Law (Another Form) • Ave. number of requests being processed at any time = throughput × average time each request is in the system • So: • 0.5 requests per second (= throughput) • 10 second response time (= time each request stays in system) • There must be 5 servers • Amazon handles 100 million requests per day • Or 11,574 requests per second • With a 5 second response time • Must have at least 55,000 servers

Amdahl's Law • Let: • t1 be a program’s runtime on one CPU • tp be its runtime on p CPUs • ß be the algorithm’s serial fraction

…Amdahl's Law • Example: • Want 32X speedup on 64-processor machine • So ß must be 0.984 • I.e., no more than 2% of the code can be purely sequential • Ouch • What if only half the code can run in parallel? • s32 is 1.97 • Ouch again

CSC407: Software Architecture Winter 2007 Performance Analysis I