290 likes | 435 Views
Cutting the Electric Bill for Internet-Scale Systems. Andreas Andreou Cambridge University, R02 aa773@cam.ac.uk. What’s this all about?. Energy expenses are an increasingly important fraction of data center operating costs Electricity prices show both temporal and geographical variation
E N D
Cutting the Electric Bill for Internet-Scale Systems Andreas Andreou Cambridge University, R02 aa773@cam.ac.uk
What’s this all about? • Energy expenses are an increasingly important fraction of data center operating costs • Electricity prices show both temporal and geographical variation • Exploit variations in electricity prices for economic gain
Key observations • Electricity prices vary • Prices vary on an hourly basis • Often not well correlated at different locations • Substantial variations • Large distributed systems already incorporate request routing and replication • Dynamic request routing to map clients to servers • Mechanisms to replicate data necessary to process requests at multiples sites
Problem Specification • Large system composed of server clusters spread out geographically • Map client requests to clusters such that the total electricity cost is minimized • Assumptions • System fully replicated • Optimize for cost every hour • No knowledge of the future • Rate of change slow enough to be compatible with existing routing mechanisms • Fast enough to respond to electricity market fluctuations • Incorporate bandwidth and performance goals as constraints
Terminology • Energy Elasticity • Degree to which energy consumed by a cluster depends on the load placed on it • Ideally: no load, no power • Worst case: no difference between peak and idle power • State-of-the-art: idle power around 60% of peak • Differential Duration • Number of hours one location is favored over another by more than $5/MWh • PUE • Power usage effectiveness (measure of data center energy efficiency)
Wholesale Electricity Markets (1) • Generation • Government and independent power producers • Coal (~50%), natural gas (~20%), nuclear power (~20%), hydroelectric generation (~6%) • Different regions, different power generation profiles • Transmission • Producers and consumers are connected to an electric grid • 8 reliability regions
Wholesale Electricity Markets (2) • Market Structure • Each region managed by Regional Transmission Organization (RTO) • RTO administer wholesale electricity markets • Auctioning mechanism: • Producers present supply offers • Consumers present demand bids • Coordinating body determines flow and sets prices • Market Types • Day-ahead markets • Real-time markets
Wholesale Electricity Markets (3) • Market Structure • Assumptions • Real-time prices are known and vary hourly • Electric bill is proportional to consumption and indexed to wholesale prices • Request routing behavior induced by our method doesn’t significantly alter prices and market behavior
Different Market Types • Hourly real-time (RT) market is more volatile than day-ahead market
Akamai: Traffic and Bandwidth • Over 2000 content provider customers in the US • 9-region traffic with electricity price data • Data covering 24 days worth of traffic • Traffic data of 5-minute intervals from public clusters • Bandwidth costs are significant • Aggressively optimized to reduce bandwidth costs • 95/5 billing model • Client-Server Distances • Use geographic distance as a coarse proxy for network performance
Cluster Energy Consumption (1) • Roughly linear to its utilization • Pidle : average idle power draw of single server • Ppeak : average peak power draw of single server • r: empirical derived constant • ut : average CPU utilization at time t • what is important in determining savings
Routing Energy • Increased path lengths will not alter energy consumption significantly • Average energy for a packet to pas through is on the order of 2mJ • Incremental energy dissipated by each packet passing through a core router would be as low as 50μJ per medium size packet • New routes may overload existing routers • Additional bandwidth could lead to upgrade • Can ignore by incorporating 95/5 bandwidth constraints
Simulation Strategy • Real-time market prices for 29 different locations • Traffic data for Akamai public clusters in 9 of those • Data set spanning Jan 2006 through Mar 2009 • Workload data set contains 5-minute samples in 25 cities • Period of 24 days and some hours • Discarded 7 and grouped remaining 18 cities to 9 clusters • Akamai’s geographic server distribution • Two routing schemes • Akamai’s original allocation • Distance constrained electricity price optimizer • Energy model as shown before
24 Days of Traffic (1) • Energy Elasticity • Bandwidth Costs
24 Days of Traffic (2) • Distance and savings
39 Months of Prices • Derived from 24-day Akamai workload (US traffic only) • Dynamic beats static
Results • Existing systems can reduce energy costs be at least 2% without any increase in bandwidth costs or significant reduction in client performance • Google-like energy elasticity • Akamai-like server distribution • 95/5 bandwidth constraints • Savings increase with energy elasticity • Fully elastic system with relaxed bandwidth constraints can reduce energy cost be 30% (13% with bandwidth constraints) • Allowing increase of client-server distances leads to increased savings
Considerations (1) • Not reacting immediately to price changes noticebly reduces overall savings
Considerations (2) • Server operators should be able to negotiate contractual arrangements • Distributed systems with energy elastic clusters can be more flexible than traditional consumers • Triggered demand response programs
Future Work • Implementing Joint Optimization • RTO Interaction • Weather Differentials • Environmental Cost