Towards Optimal Sensor Placement for Hot Server Detection in Data Centers

1. Towards Optimal Sensor Placement for Hot Server Detection in Data Centers Xiaodong Wang, XiaoruiWang, GuoliangXing, JinzhuChen, Cheng-Xian Lin. and Yixin Chen.

2. Outline • Introduction • Related work • Hot server detection problem • CFD-guided sensor placement • Evaluation • Summary

3. Introduction • Thermal monitoring is important in data center operation: • Overheating is harmful to data center. • Malfunction of hardware components. • Server shut down. • Excessive cooling energy is consumed. • Operation of cooling systems is not efficient enough. • Excessive energy consumption required by overcooling. • To have precise hot server detection: • Precise hot server detection can guide air conditioning system. • Thermal dynamics in data center need to be better studied. • Place more sensors to increase thermal visibility.

4. Related Work • Studies of thermal profile • [Choi et al. HPCA ‘07 ] studied thermal profile of a rack. • [Patel et al. IPACK ‘01] studied the air temperature specification of a data center in normal condition. • Improve thermal monitoring with sensor networks: • [Liang et al. SenSys ‘09] deployed sensor networks in data center to achieve a high-fidelity thermal monitoring. • [Moore et al. USENIX ‘05] and [Bash et al. USENIX, ’07] proposed to allocate server job and workload based on thermal readings from sensor networks. Not used to guide sensor placement. How to effectively place sensors?

5. Hot Server Detection Problem • Problem to solve: • To intelligently place sensors for a maximum hot server detection probability. • Problem formulation: • Given M locations to monitor and N (N<M) sensors to use: Subject to the constraint: • : Detection probability of overheating at monitored location • : False alarm rate of overheating at monitored location

6. Problem Solving Architecture • Overheating data center analysis. • Analyze the data center in overheating condition. • Obtain the temperature distribution for overheating cases. • Find the sensor placement solution. • Sensor readings usually are corrupted by noise. • Sensors need to collaboratively make hot server detection decision (data fusion) Overheating Analysis Sensor Placement Data fusion & placement algorithm Temperature Interpolation CFD Modeling

7. Overheating Data Center Analysis • Computational Fluid Dynamics (CFD) model for overheating data center • A finite volume method. • Example: Datacenter physical model temperature distribution CFD Modeling Temperature Interpolation Power consumption …… CRAC settings A/C in A/C In/Out A/COut A/C Out A/C Out

8. Overheating Data Center Analysis (cont’d) • Spatial temperature interpolation • Results from CFD are discrete in locations. • Granularity of CFD modeling is a tradeoff between accuracy and computational complexity. • Inverse Distance Weighting (IDW) interpolation: • Weighted average of the available temperature data. • Optimize sensor placement based on the overheating analysis • To achieve a maximized average overheating server detection probability.

9. CFD Guided Sensor Placement • Sensor placement with existing solver: • To decide the x, y, and z variables of each sensor location. • Constrained Simulated Annealing (CSA) • An existing solver with 3N variables. • Computational time increases exponentially. • Lightweight Sensor Placement (LSP): • Only searches placement solution at areas with clustered racks. • A greedy algorithm, which adds sensors one by one. • Search space and computational time are significantly reduced.

10. Simulation Setup • Experiment environment setup • CFD software packages: Gambit and Fluent • Server room size: 32m x 7m x 3m • 13 racks in the server room. • 4 monitored locations each rack (52 locations in total) • 14,400 watts power consumption for each overheating rack. • CRAC settings are collected by external sensor.

11. Simulation Results • Different sensor numbers • Baselines: • Uniformly Random, current practice. • CFD+ proportional. • Using more sensors increases the detection probability. • CFD+LSP (our solution) is closest to the optimal solution

12. Simulation Results (cont’d) • Different temperature threshold • Detection probability decreases when temperature threshold increases • Different fusion range: • A proper fusion range can increase the detection probability.

13. Hardware Experiment in a Server Room • Setup: • A small cluster of two racks is used. • Overheating is created by a heater • Results:

14. Summary • We place sensors intelligently in data centers • To reach a maximum hot server detection probability • Various overheating conditions are studied to guide sensor placement • CFD is used to analyze data centers under overheating condition. • Future consideration: • Integrate with thermal control approaches. • More detail CFD modeling.

15. Q&A Thank You! • Acknowledgement • NSF CAREER Award CNS-0845390 • NSF under Grants CNS-0720663, CNS-0915959, CCF-1017336, and CNS-0954039 • Microsoft Research under a Power-Aware Computing Award

16. Appendix A • Sensor readings usually are corrupted by noise. • Overheating scenario detected when the measured temperature is larger than the threshold. • False alarm happens when the overheating detection is intrigued by noise only.

17. Appendix B • Rack clustering • The closest distance of two monitored locations in two different clusters is larger than 2R. • Inverse Distance Weighting (IDW) interpolation: