570 likes | 1.03k Views
A CLASS OF POLICIES THAT PRIORITIZE SMALL JOBS Adam Wierman Carnegie Mellon University Pittsburgh, PA USA Web Server Use Shortest Remaining Processing Time First (SRPT) A MOTIVATIONAL EXAMPLE: Web servers Bandwidth is the bottleneck Processor Sharing (PS) Goal Minimize user
E N D
A CLASS OF POLICIES THAT PRIORITIZE SMALL JOBSAdam Wierman Carnegie Mellon University Pittsburgh, PA USA
Web Server Use Shortest Remaining Processing Time First (SRPT) A MOTIVATIONAL EXAMPLE: Web servers Bandwidth is the bottleneck Processor Sharing (PS) Goal Minimize user response times requests for files [Harchol-Balter, Schroeder, Bansal, Agrawal, TOCS 2003]
PS mean response time ? Why SRPT? SRPT load 0 1 A MOTIVATIONAL EXAMPLE: Web servers [Harchol-Balter, Schroeder, Bansal, Agrawal, TOCS 2003]
users CPUs Internet Databases Routers Disks Prioritize small jobs Locks SCHEDULING SUCCESS STORIES ACROSS COMPUTER APPLICATIONS Web Servers …also p2p, wireless, operating systems… Rai, Urvoy-Kelley, Biersack, Harchol-Balter, Schroeder, McWherter, Rawat, Dinda, & many others
SRPT RS PSJF PS LCFS SJF DPS PLJF FCFS LJF FSP PLCFS FB LRPT SCHEDULING SUCCESS STORIES ACROSS COMPUTER APPLICATIONS RENEWED INTEREST IN THEORETICAL WORK Harchol-Balter, Biersack, Ayesta, Nunez-Queija, Nuyens, Boxma, Zwart, Borst, & many others
There is a gap between the theoretical results and the practical implemplentations
Slowdown & Fairness are also important Hybrid policies outperform SRPT on these Theory a small set of individual, idealized policies are analyzed multiple metrics are important Practice real policies are hybrids policies studied in theory are “tweaked” for use in practice
Prioritize small jobs BRIDGING THE GAP Analyze classes of policiesinstead of individual policies SRPT RS PSJF PS LCFS SJF DPS PLJF • Classes include practical hybrids • Classes allow optimization of secondary metrics FCFS LJF PLCFS FSP FB LRPT
OUR GOAL TODAYDefine a class of policies that “prioritizes small jobs” and is bothBROADandTIGHT behaves like SRPT includes practical hybrid policies
THE SMART CLASS SMART PSJF RS SRPT SMAll Response Times • Bias Property • Consistency Property • Transitivity Property PS LJF PLCFS ROS FCFS LRPT
TWO NOTIONS OF “SMALL” JOBS small original size small remaining size
BIAS PROPERTY If OriginalSize(a) < RemainingSize(b) then a has priority over b a b [Wierman, Harchol-Balter, Osogami 2005]
remaining size 0 0 original size BIAS PROPERTY If OriginalSize(a) < RemainingSize(b) then a has priority over b [Wierman, Harchol-Balter, Osogami 2005]
BIAS PROPERTY If OriginalSize(a) < RemainingSize(b) then a has priority over b lower priority remaining size ? higher priority 0 0 original size [Wierman, Harchol-Balter, Osogami 2005]
SRPT PSJF ! RS remaining size Partial ordering allows time varying policies ? original size EXAMPLES If OrigSize(a) < RemSize(b) then a has priority over b Many others! e.g. iRj+S
! Partial ordering allows time varying policies B B A C A D remaining size *PSJF D C time
! Partial ordering allows time varying policies B A ? B A D remaining size *PSJF *PSJF D C *SRPT time
! Partial ordering allows time varying policies • Use a parameterized policy setthat is (nearly) dense in SMART,e.g. iRj + S • Search (i,j) space for policy thatoptimizes secondary objectives,e.g. fairness and predictability ONLINE MULTI-OBJECTIVE SCHEDULING USING SMART
THE BIAS PROPERTYISN’T ENOUGH CONSISTENCY TRANSITIVITY remaining size ? orig. size [Wierman, Harchol-Balter, Osogami 2005]
THE BIAS PROPERTYISN’T ENOUGH CONSISTENCY If a is served ahead of b then a will always have priority over b + TRANSITIVITY If an arriving job b preempts c, then until b leaves, every arriving job a with original size smaller than b has priority over c. remaining size ? at most 1 has higher priority orig. size [Wierman, Harchol-Balter, Osogami 2005]
DOING THE “SMART” THING lower priority SMAll Response Times remaining size • Bias Property • Consistency Property • Transitivity Property at most 1 job with higher priority higher priority 0 0 original size
? Which policies are notSMART? SMART PSJF RS SRPT FB SJF PS LJF PLCFS ROS FCFS LRPT Blind policies Non-preemptive policies
OUR GOAL TODAYDefine a class of policies that “prioritizes small jobs” and is bothBROADandTIGHT behaves like SRPT includes practical hybrid policies
M/GI/1 preempt-resume queue ANALYSIS SETTING: E[T]SMART APPROACH: Bound T(x)SMART Pr(TSMART>y)
Residence time Residence time Waiting time Waiting time T(x) RESULT, plot for E[T(x)] CONDITIONAL RESPONSE TIME UNDER SMART POLICIES Theorem: Under the M/GI/1, for all SMART policies P, Response time for a job of size x [Wierman, Harchol-Balter, Osogami 2005]
SRPT SMART PSJF original size T(x) RESULT, plot for E[T(x)] CONDITIONAL RESPONSE TIME UNDER SMART POLICIES Theorem: Under the M/GI/1, for all SMART policies P, Picture “proof”: Waiting time remaining size ?
SRPT PSJF T(x) RESULT, plot for E[T(x)] CONDITIONAL RESPONSE TIME UNDER SMART POLICIES Theorem: Under the M/GI/1, for all SMART policies P, Picture “proof”: Residence time SMART remaining size No higher priority jobs ? ? original size
M/GI/1 preempt-resume queue ANALYSIS SETTING: E[T]SMART APPROACH: Bound T(x)SMART Pr(TSMART>y)
Theorem: In the M/GI/1, mean response time PS E[T] ! These bounds are tight SMART load, ρ 0 1 SMART POLICIES ARE 2-COMPETITIVE [Wierman, Harchol-Balter, Osogami 2005]
Theorem: In the M/GI/1, ! These bounds are tight SMART POLICIES ARE 2-COMPETITIVE Consider the M/D/1 SRPT does FCFS (only in M/D/1). So as ρ1 PLCFS is in SMART (only in M/D/1) As ρ1, E[T]PLCFS 2 E[T]SRPT
Theorem: In the M/GI/1, ? Tight under Deterministic Tight under Pareto (as ρ1) What is the heavy traffic behavior? SMART POLICIES ARE 2-COMPETITIVE With work, the bounds can be rewritten as
M/GI/1 preempt-resume queue ANALYSIS SETTING: E[T]SMART APPROACH: Bound T(x)SMART Pr(TSMART>y)
λ1 Nμ λ2 λN NB Many sources Large buffer ! SMART policies are asymptotically equivalent in both TAIL BEHAVIOR OF SMART Pr(T>y)is difficult to study directly so it is typically studied asymptotically [Nuyens, Wierman, Zwart 2007] [Yang, Wierman, Shakkattai, Harchol-Balter 2006]
OUR GOAL TODAYDefine a class of policies that “prioritizes small jobs” and is bothBROADandTIGHT • behaves like SRPT: • E[T] • Pr(T>y) includes practical hybrid policies But…
Theory a small set of individual, idealized policies are analyzed multiple metrics are important implementation restrictions Practice real policies are hybrids policies studied in theory are “tweaked” for use in practice
Theory a small set of individual, idealized policies are analyzed implementationsonly have job size estimates implementation restrictions Practice implementations only use 7-10 priority levels policies studied in theory are “tweaked” for use in practice
? What is the impact of inexact job size information? no k such that use a SMART policy know exact job sizes 1 ? have an estimate of job sizes prioritize jobs with small estimated sizes 2 know only the distribution of job sizes if sizes are variable use FB otherwise use FCFS 3
OUR NEW GOALBROADENtheSMART class while keeping itTIGHT 2 k E[T]P ≤ E[T]SRPT include policies that use job size estimates
rem. size ? orig. size rem. size ? Weaken the Bias Property orig. size
remaining size ε(x) ? ? original size original size SMART SMARTε If OrigSize(a) = x and ε(x) < RemSize(b) then a has priority over b If OrigSize(a) < RemSize(b) then a has priority over b
ε(x) ? original size SMARTε EXAMPLES If OrigSize(a) = x and ε(x) < RemSize(b) then a has priority over b How do you get back SMART? ε(x) = x How can you characterize job size estimates? ε(x) = (1+σ) x You can also capture policies with a finite number of priority levels...
OUR NEW GOALBROADENtheSMART class while keeping itTIGHT k E[T]P ≤ E[T]SRPT include policies that use inexact job sizes
M/GI/1 preempt-resume queue ANALYSIS SETTING: E[T]SMART APPROACH: Bound T(x)SMART Pr(TSMART>y)
ε(x) rem. size ? ! Under SMART σ=δ=0, so E[T] ≤ 2E[T]SRPT orig. size ! x (1+σ) = (1-δ)1-α under Pareto(α) distributions SMARTε POLICIES ARE NEAR OPTIMAL Theorem: In an M/GI/1 under SMARTεpolicy P σbounds the SIZE of larger jobs that get higher priority δbounds the LOAD of larger jobs that get higher priority
? What is the effect of inexact job size information? Theorem: In an M/GI/1 with Pareto(α) job sizes and SMARTεpolicy P where for all x, 20 E[T] / E[T]SRPT 10 2 0 200% 100% estimate accuracy (σ) INTERPRETING THE THEOREM Take α=1.1
? What is the effect of inexact job size information? Theorem: In an M/GI/1 with Pareto(α) job sizes and SMARTεpolicy P where for all x, PS INTERPRETING THE THEOREM mean response time SMARTε SRPT load 0 1
M/GI/1 preempt-resume queue ANALYSIS SETTING: E[T]SMART APPROACH: Bound T(x)SMART Pr(TSMART>y)
λ1 Nμ λ2 λN NB Many sources Large buffer ! SMARTε policies are asymptotically equivalent for unbounded service distributions TAIL BEHAVIOR OF SMARTε POLICIES Pr(T>y)is difficult to study directly so it is typically it is studied asymptotically
! Other forms of ε(x) are also useful SMARTε when PSJF RS E[T]≤ k E[T]SRPT SRPT Pr(T>x)~ Pr(TSRPT>x) PS LJF FCFS Show that SMARTε policies behave like SRPT Formalize the heuristic of “prioritizing small jobs” • Bias Property • Consistency Property • Transitivity Property