1 / 44

Insights into Google's PUE Results

Insights into Google's PUE Results. Spring 2009. Published Google PUE Results. Insights into Google's PUE Results. Today, more details on... Innovations and best practices that enabled the results Measurement method & accuracy Benefits of measuring PUE

deborahlindsay
Download Presentation

Insights into Google's PUE Results

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Insights into Google's PUE Results Spring 2009

  2. Published Google PUE Results

  3. Insights into Google's PUE Results • Today, more details on... • Innovations and best practices that enabled the results • Measurement method & accuracy • Benefits of measuring PUE • And, a PUE update + a Google server on display

  4. Impact of power & cooling innovations and best practices PUE Components: Typical vs. Google Google Data Center ‘E’ PUE = 1.16 Typical PUE = 2.0 *Reference: Silicon Valley Leadership Group, Data Center Energy Forecast, Final Report July, 2008 Google E Data Center energy-weighted average PUE results from Q2-Q1’09 (to 3/15/09)

  5. Impact of power & cooling innovations and best practices PUE Components: Typical vs. Google Google Data Center ‘E’ PUE = 1.16 Typical PUE = 2.0 85% reduction in cooling energy *Reference: Silicon Valley Leadership Group, Data Center Energy Forecast, Final Report July, 2008 Google E Data Center energy-weighted average PUE results from Q2-Q1’09 (to 3/15/09)

  6. Impact of power & cooling innovations and best practices PUE Components: Typical vs. Google Google Data Center ‘E’ PUE = 1.16 Typical PUE = 2.0 82% reduction in power distribution/backup losses *Reference: Silicon Valley Leadership Group, Data Center Energy Forecast, Final Report July, 2008 Google E Data Center energy-weighted average PUE results from Q2-Q1’09 (to 3/15/09)

  7. PUE Components: Typical vs. Google Result How? Details • Close-coupled cooling • Raise the temperatures • Economizers Cooling energy reduced by 85% This afternoon Power distribution & backup losses reduced by 82% 99.9% efficient UPS Next

  8. Google's On-Board UPS

  9. Facility UPS Grid AC UPS IT AC Battery Pack Rectifier Inverter

  10. Facility UPS Grid AC IT AC

  11. Facility UPS Grid AC IT AC

  12. Typical Server PSU Motherboard Disk

  13. Google Server PSU Motherboard * DCP++ Disk

  14. Google Server PSU Motherboard Disk * Featured in the Climate Savers Computing (initiative)

  15. Google Server PSU Motherboard Disk

  16. Google Server PSU Motherboard Battery Disk

  17. Google Server

  18. Google UPS State Diagram Shutdown Timeout or low battery AC on AC back Recharge Discharge AC loss AC loss Battery full Float

  19. Google UPS State Diagram Shutdown Timeout or low battery AC on AC back Recharge Discharge AC loss AC loss Battery full Float

  20. Float: VBATT ≈ 13V RBATT ≈ 100mΩ VSUPPLY ≈ VLOAD ≈ VBATT > 13V AC live, load running, battery float charging VSUPPLY≈ 13V RLOAD ≈ 0.5Ω RCHARGER ≈ 20Ω Discharge Battery leakage (self-discharge) current <0.6mA, leakage power <8mW, efficiency >99.99%

  21. Design Summary and Achievements Examine the entire cluster for a solution, not just the UPS Eliminate central UPS, distribute to each machine Eliminate AC-DC-AC double conversion  Small battery, just enough to bridge until generators start up, or clean machine shutdown Incremental deployment of UPS capacity - no waste! Single-voltage PSU is the enabler, also made servers more power-efficient (not included in PUE computation) Real-world measured efficiency of >99.99%

  22. PUE Measurement Methods

  23. PUE Definition & Trends Green Grid Data Center Efficiency Metric Total Facility Power PUE = IT Equipment Power EPA 2006-2011 PUE Projections Technology Investment Scenario PUE Typical 2006 Data Center 2.0 Current Trends 1.9 2011-Era Data Center Improved Operations 1.7 Best Practices 1.3 State-of-the-Art 1.2 References: The Green Grid, 2007, “The Green Grid Data Center Power Efficiency Metrics: PUE and DCiE,” Technical Committee White Paper, Environmental Protection Agency, ENERGY STAR Program, 2007, “Report to Congress on Server and Data Energy Efficiency”.

  24. PUE Definition & Trends Green Grid Data Center Efficiency Metric Total Facility Power PUE = IT Equipment Power EPA 2006-2011 PUE Projections Technology Investment Scenario PUE Typical 2006 Data Center 2.0 Current Trends 1.9 2011-Era Data Center Improved Operations 1.7 Best Practices 1.3 Remember this number! State-of-the-Art 1.2 References: The Green Grid, 2007, “The Green Grid Data Center Power Efficiency Metrics: PUE and DCiE,” Technical Committee White Paper, Environmental Protection Agency, ENERGY STAR Program, 2007, “Report to Congress on Server and Data Energy Efficiency”.

  25. Green Grid Measurement Accuracy Classifications Level 1 (Basic) Level 2 (Intermediate) Level 3 (Advanced) IT Equipment Power Measurement From... UPS PDU Servers Total Facility Power Measurement From... Data Center Input Power Data Center Input Less Shared HVAC Data Center Input Less Shared HVAC plus Building, Lighting, Security Minimal Measurement Interval Monthly/Weekly Daily Continuous Reference: The Green Grid, 2008, “The Green Grid Metrics: Data Center Infrastructure Efficiency (DCiE) Detailed Analysis,” Technical Committee White Paper

  26. Green Grid Measurement Accuracy ClassificationsGoogle Data Centers: 2+ Level 1 (Basic) Level 2 (Intermediate) Level 3 (Advanced) IT Equipment Power Measurement From... UPS PDU Servers Enabled on newer servers Total Facility Power Measurement From... Data Center Input Power Data Center Input Less Shared HVAC Data Center Input Less Shared HVAC plus Building, Lighting, Security Minimal Measurement Interval Monthly/Weekly Daily Continuous 4 of 6 reported data centers 2 of 6 reported data centers Reference: The Green Grid, 2008, “The Green Grid Metrics: Data Center Infrastructure Efficiency (DCiE) Detailed Analysis,” Technical Committee White Paper

  27. Google Data Center Power Distribution Schematic Medium Voltage High Voltage Utility Distribution Low Voltage Overhead (Cooling plant...) L3 IT Equipment Power Losses & Overhead Power UPS Network MV Switchgear Substation L2 Overhead (CRACs...) UPS Servers UPS Network 99.99% Efficient UPS

  28. PUE Uncertainty <2%Achieved with Accurate Unit Substation Meters  Σ Unit Substations + Small Terms Total Facility Power PUE = ~ Σ IT Unit Substations - Small Terms IT Equipment Power Total Data Center power extrapolated to here Overhead (Cooling plant...) L3 IT Equipment Power Losses & Overhead Power UPS Network MV Switchgear Substation L2 Overhead (CRACs...) UPS Servers UPS Network IT power extrapolated to here

  29. Primary Power Measurement PointHigh-Accuracy Metering at the Unit Substations  Measurement Uncertainty (CT + PT + Meter) = <±2% Overhead (Cooling plant...) L3 UPS Network MV Switchgear Substation L2 Overhead (CRACs...) UPS Servers UPS Network IT power extrapolated to here

  30. Example: Data Center 'E': PUE = 1.16 ± 1.0%Unit Substation Measurement Uncertainty Unit Substation Measurement 10% of total error Total Data Center power extrapolated to here Overhead (Cooling plant...) L3 IT Equipment Power Losses & Overhead Power UPS Network MV Switchgear Substation L2 Overhead (CRACs...) UPS Servers UPS Network IT power extrapolated to here

  31. Example: Data Center 'E': PUE = 1.16 ± 1.0%Substation-to-Meter Uncertainty Total Data Center power extrapolated to here Overhead (Cooling plant...) M L3 IT Equipment Power Losses & Overhead Power Overhead Unit Substation UPS Network M MV Switchgear Substation L2 Overhead (CRACs...) UPS Servers Substation-to-Meter Losses 2% total power 13% of error UPS Network IT power extrapolated to here

  32. Example: Data Center 'E': PUE = 1.16 ± 1.0%Low Voltage-Side Uncertainty Network 0.2% total power 0.2% of error Total Data Center power extrapolated to here Overhead (Cooling plant...) M L3 IT Equipment Power Losses & Overhead Power Overhead Unit Substation M Network UPS MV Switchgear Substation L2 Overhead (CRACs...) CRACs & Parasitics 3% total power 73% of error UPS Servers M Network UPS Meter-to-IT Losses 1% total power 5% of error IT power extrapolated to here

  33. Example: Data Center 'E': PUE = 1.16 ± 1.0%Summary Uncertainty Component % Unit Substation Energy % Contribution to Total Uncertainty 100% 10% Unit Substation Measurement Substation-to-Meter Estimate 1.9% 13% Network Estimate 0.2% 0.2% CRACs & Parasitics Estimate 3.2% 73% Meter-to-IT Estimate 0.6% 5% • Very accurate measurement of Unit Substation power dominates uncertainty calculation • Estimated components are less accurate, but small in magnitude

  34. PUE Results

  35. January 2009 PUE Results Commissioning & seasonal effects Data published quarterly for all Google data centers with 5+ MW IT load for at least 6 months

  36. Latest PUE Results Continued optimization & beneficial seasonal effects Data published quarterly for all Google data centers with 5+ MW IT load for at least 6 months

  37. PUE ResultsUpdate: Q1'09 to 3/15 Scenario Q1’09 to 3/15 Q4’08 Q3’08 Quarterly energy-weighted average PUE 1.15 1.16 1.23* Trailing twelve-month energy-weighted avg. PUE 1.19 1.20* 1.21 Individual facility minimum quarterly PUE 1.12 1.13* 1.17 Individual facility minimum TTM PUE** 1.15 1.15 1.15 Individual facility maximum quarterly PUE 1.20 1.21 1.30* Individual facility maximum TTM PUE** 1.23 1.24 1.25 *Audit revealed a 1-2% error in previously published results. **Only facilities with at least twelve months of operation are eligible for individual facility TTM PUE reporting

  38. Better than the 2011 EPA State-of-the-Art data center PUE of 1.20 PUE ResultsUpdate: Q1'09 to 3/15 Scenario Q1’09 to 3/15 Q4’08 Q3’08 Quarterly energy-weighted average PUE 1.15 1.16 1.23* Trailing twelve-month energy-weighted avg. PUE 1.19 1.20* 1.21 Individual facility minimum quarterly PUE 1.12 1.13* 1.17 Individual facility minimum TTM PUE** 1.15 1.15 1.15 Individual facility maximum quarterly PUE 1.20 1.21 1.30* Individual facility maximum TTM PUE** 1.23 1.24 1.25 *Audit revealed a 1-2% error in previously published results. **Only facilities with at least twelve months of operation are eligible for individual facility TTM PUE reporting

  39. PUE ComponentsWe Measure by the Minute Daily PUE Overhead – Google ‘E’ Q2’08 Q1’09 Q3’08 Q4’08 Deeper insight into commissioning & seasonal effects

  40. Hourly Average PUE ReportingMinute-by-Minute Measurements Google Data Center ‘E’ Week 1 January 2009 Google Data Center ‘E’ January 2009 Unusual weather impacts chiller runtime

  41. Insights from MeasurementsPUE vs. IT Load Identify commissioning & operational issues, design & deployment plan decisions IT Growth Weather, commissioning & plant scaling

  42. Conclusions • 80+% reduction in power & cooling overhead vs. typical facility • Google UPS solution delivers >99.99% efficiency • Consider options in the market that approach this efficiency • Line-interactive & multi-mode UPS, flywheels, DC distribution • Cooling best practices • Close-coupled cooling, raise the temperatures, & economizers • Continued improvement in latest Google PUE result  • TTM Average PUE Trend: • 1.21 (Q3'08) to 1.20 (Q4'08) to 1.19 (to 3/15/'09) • Know your PUE • What's next?

  43. Best practice: minimize chiller operating hours • Best embodiment: chiller-less data centers • Reduce capital costs • Reduce the PUE spikes • Target TTM average PUE ~ 1.1 • Chiller-less Belgium data center operational The Limit: Chiller-Less Data Center

  44. Thank you! Q&A

More Related