Cost- and Energy-Aware Load Distribution Across Data Centers

1. Kien Le, Ricardo Bianchini, Margaret Martonosi, and Thu D. Nguyen (HotPower 2009)Rutgers University and Princeton University Cost- and Energy-Aware Load Distribution Across Data Centers Presented by Shameem Ahmed

3. Motivations • Large org has multiple Data Centers (DC) • Business distribution • High availability • Disaster tolerance • Uniform access times to widely distributed client sites • Problems • Consumes lots of energy • Financial and environmental cost • How can we exploit the geographical distribution of DCs for optimizing energy consumption? • Different & variable electricity prices (hourly pricing) • Exploit DCs in different time zones (peak/off-peak demand price) • Exploit DCs located near sites that produce “green” electricity

4. Assumptions • Multi-DC Internet services (e.g. Google, iTunes) • DCs are behind a set of front-end devices • Each service has single SLA (Service Level Agreement) with customers • SLA (L,P) = At least P% req must complete in <L time • Req can be served by 2 or 3 mirror DC • Further replication increases state-consistency traffic But no meaningful benefit in availability or performance Is it True?

5. Contributions • Framework for optimization-based request distribution policy • What % of client req should be directed to each DC • Front-ends periodically solve optimization problem • After % computation, front-ends abide by them until they are recomputed • A greedy heuristic policy for comparison • Same goal and constraints • First exploits DC with best power efficiency • Then exploits DC with cheapest electricity

6. Prior Research • Energy management on a single data center • A. Qureshi. HotNets 2008 • Shut down entire data centers when electricity costs are relatively high • K. Le et al. Middleware 2007 • Did not address energy issues, time zones, or heuristics

7. Request Distribution Policies

8. Principles and Guidelines • Only minimizing energy cost is not enough • Must also guarantee high performance and availability • Respect these requirements by having the front-ends: • Prevent DC overloads • Monitor response time of DCs and adjust req distribution accordingly • Each DC reconfigures itself by • Leaving as many servers active as necessary + 20% slack for unexpected load increase • Other servers are turned off

9. Optimization Based Distribution (1/4) • Problem Formulation • Policy EPrice: Leveraging time zones & variable electricity prices “base” energy cost (servers are idle) energy cost of processing the client req Doesn’t distinguish DCs based on energy source

10. Optimization Based Distribution (2/4) • Problem Formulation • Policy GreenDC: Leveraging DCs powered by green energy Assumptions: DCs will increasingly be located near green energy source Green energy supply may not be enough to power DC entire period; Need backup (regular electricity) energy cost of processing the client req “base” energy cost that is spent when active servers are idle

11. Optimization Based Distribution (3/4) • Instantiating parameters • Typical approach: front ends communicate & coordinate • Proposed approach: • Each front end independently solves optimization problem • LT(t), LR(t), and offeredi are defined for each front-end • Load capacity (LC) of each DC is divided by # of front-ends • CDFi instantiation • CDFi = Expected % of req that complete within L time • Each Front end • Collects recent history of response time of DCi • Maintains a table of <offered load, %> for each DC • Similar table for BCosti: <offered load, base energy cost> Does this approach satisfy the constraints globally?

12. Optimization Based Distribution (4/4) • Solving Optimization Problem • Electricity price prediction: Ameren • Load intensity prediction: ARMA • CDFi prediction: Current CDFi tables • Can’t use LP solvers • Solving for entire day at once involves non-linear functions (e.g. BCosti, CDFi) • Use Simulated Annealing • Divide the day into six 4-hour epochs • Solution recomputation (e.g. data center becomes unavailable)

13. Heuristic-Based Request Distribution (1/2) • Cost-aware but simple • For each epoch (1 hr), each front-end computes R = P x E • P = % of req must complete within L time (SLA) • E = # of req front-end expects in next epoch (use ARMA) • R = # of req that must complete within L time • Each front-end orders DCs that have CDFi(L, LCi)>= P according to from lowest to highest ratio • Remaining DCs are ordered by same ratio • Concatenate two lists of DC to create final list (MainOrder)

14. Heuristic-Based Request Distribution (2/2) • Request forward policy • Req are forwarded to first DC in MainOrder until its capacity is met • New req is forwarded to next DC on the list and so on • After front-end has served R req within L time, it can disregard MainOrder and start forwarding req to cheapest DC until capacity is met • What will happen if prediction is inaccurate? • Adjusts R for next epoch

15. Optimization-based vs Heuristics-based

16. Evaluation

17. Methodology (1/4) • Implemented a simulator for large Internet service • Simulate only a single front-end (East US) • Front-end distributes req to 3 DC

18. Methodology (2/4) • Request Trace • Day-long trace received by Ask.com • Trace doesn’t include response time • To generate realistic DC response times: • Installed a simple service on 3 PlanetLab machines • Req are made from a machine at Rutgers (front-end) • Assumption: avg processing time of each req = 200 ms • How to mimic effect of load intensity and network congestion • 5% increase in response time for each 25% increase in load ARMA

19. Methodology (3/4) • Electricity Prices, Sources, and time zones • Three price scheme • Constant rate, two rate (on/off peak), hourly prices • How to mimic different brown electricity prices for each DC? • Shift default prices 3 hrs earlier or 6 hrs later • Assumptions • Electricity price for Green DC is constant • Green energy at each green site is enough to process 25% req Ameren

20. Methodology (4/4) • Other parameters • Assumptions • A req consumes 60 J to process by 2 machines (including cooling, conversion, and delivery overheads) • SLA requires 90% of req to complete in 700 ms • Cost-unaware distribution policy • Used for comparison basis • Approach: similar to CA-heuristic • Orders DCs according to performance [CDFi(L, LCi)] from highest to lowest • Req are forwarded to first DC until its capacity is met • New req are forwarded to next DC and so on

21. Result (1/4) • Effect of cost-awareness and pricing scheme (brown electricity) • No cost for base energy • Both cost-aware policies reduce costs even under constant pricing • On/Off and Dynamic schemes reduce cost significantly • EPrice always achieves lowest cost • EPrice: Optimization-based distribution (No green energy) • CA-Heuristic: Cost-Aware Heuristic (consider Costi/CDFi) • CU-Heuristic: Cost-Unaware Heuristic (consider CDFi)

22. Result (2/4) Why does EPrice behave better than CA-Heuristic? Can Compensate DC3’s poor performance during future periods of low load.

23. Result (3/4) • Effect of green DC • Considers only dynamic pricing • Results are normalized against EPrice results w/ dynamic pricing • No cost for base energy Brown energy consumption is reduced by 35% by using green DC (3% cost increase) Why do heuristic policies have higher cost than GreenDC?

24. Result (4/4) • Effect of Base Energy • Assumption: • Server consumes power even when idle • No DC consumes green energy Base energy Cost savings

25. Conclusion • Optimization framework for request distribution in multi-DC • To reduce energy consumption and cost • To respect SLAs • Policies take advantage of time zones, variable electricity prices, and green energy • Propose a heuristic for achieving the same goal • Evaluation using a day-long trace from a commercial service

26. Discussions • Only used 1 Front-end in experiment • More front-ends will satisfy global constraints? • How to ensure end-to-end QoS guarantee • Can we combine SLA guarantee with QoS requirement provided by clients? • How to handle services with session state • Soft state: Only lasts a user’s session with the service • All req of a session must be sent to same DC • Can we apply the similar concept for multi-cloud structure? • Optimize power • Optimize monetary cost for online service provider • In multi-cloud computing, is it good to assume that data will be available in clouds beforehand? • Pros and Cons