Download
1 / 30
300 likes | 379 Views
Always race to sleep?. i.e. how we managed to confuse ourselves by talking about two kinds of race to sleep. Always race to sleep?. i.e. how we managed to confuse ourselves by talking about two kinds of race to sleep.
E N D
Always race to sleep? i.e. how we managed to confuse ourselves by talking about two kinds of race to sleep
Always race to sleep? i.e. how we managed to confuse ourselves by talking about two kinds of race to sleep Single node – amortize static power costMany nodes – minimize parallelization overhead
1. Background: Common system power and work rate behaviors2. Analysis: Required conditions for race to sleep3. Empirical data: When are those conditions met in Hadoop?
Two common behaviors for system power Resource proportional Not resource proportional
Three common behaviors for system work rate Linear speed up Parallelization overhead Bottleneck elsewhere
1. Background: Common system power and work rate behaviors2. Analysis: Required conditions for race to sleep3. Empirical data: When are those conditions met in Hadoop?
Power Work rate
Power efficiency = work rate / power Power Work rate
Power efficiency = work rate / power Power Work rate
Power efficiency = work rate / power Race to sleep? i.e. operate at highest work rate? Power Work rate Yes Time benefit, no energy benefit Yes Increasing efficiency No Decreasing efficiency Somewhat Turning point exists Yes and no Energy benefit , no time benefit Yes and no Energy benefit , no time benefit
Power efficiency = work rate / power Race to sleep? i.e. operate at highest work rate? Power Work rate Yes Time benefit, no energy benefit Yes Increasing efficiency No Decreasing efficiency Somewhat Turning point exists Yes and no Energy benefit , no time benefit Yes and no Energy benefit , no time benefit
Power efficiency = work rate / power Race to sleep? i.e. operate at highest work rate? Power Work rate Go faster if % increase in work rate ≥ % increase in power Go slower otherwise
Power efficiency = work rate / power Race to sleep? i.e. operate at highest work rate? Power Work rate Required condition for race to sleep Go faster if % increase in work rate ≥ % increase in power Go slower otherwise
Power efficiency = work rate / power Race to sleep? i.e. operate at highest work rate? Power Work rate Required condition for race to sleep Go faster if % increase in work rate ≥ % increase in power Go slower otherwise e.g. Old work rate = A New work rate = 1.1A Old power = B New power = 1.05A Old power efficiency = A / B • New power efficiency • = (1.1 / 1.05) × (A / B) • = (1.1 / 1.05) × old power eff.
1. Background: Common system power and work rate behaviors2. Analysis: Required conditions for race to sleep3. Empirical data: When are those conditions met in Hadoop?
Hadoop sort 10GB terasort format HDFS read 10GB HDFS write 10GB Hadoop shuffle 10GB
Work rate Hadoop sort 10GB terasort format HDFS read 10GB HDFS write 10GB Hadoop shuffle 10GB
Power efficiency Work rate Hadoop sort 10GB terasort format HDFS read 10GB HDFS write 10GB Hadoop shuffle 10GB
Race to sleep Power efficiency Work rate Hadoop sort 10GB terasort format No Yes HDFS read 10GB No HDFS write 10GB Hadoop shuffle 10GB Yes
That was multi-node power efficiency Single-node power efficiency is a different picture
Always race to sleep? Maybe the question should beAlways use as much resources as possible?
Always race to sleep? Maybe the question should beAlways use as much resources as possible?Take away: Single node – amortize static power cost (awake nodes should race to sleep) Many nodes – minimize parallelization overhead (as few nodes awake as possible) Increase resource if resulting % work rate increase ≥ % power increase
Power efficiency = work rate / power Race to sleep? i.e. operate at highest work rate? Power Work rate Yes Time benefit, no energy benefit Yes Increasing efficiency No Decreasing efficiency Somewhat Turning point exists Yes and no Energy benefit , no time benefit Yes and no Energy benefit , no time benefit
