SCD Computing Center Expansion

1. SCD Computing Center Expansion Aaron Andersen SCD User Forum 5/18/2005

2. Phase 1: SCD Internal Analysis

3. Growing Model Complexity

4. Power problem: P = C V2 f Frequency (clock) increase driving Moore’s Law Decreasing voltage increases gate delay (hurts performance) Capacitance increases as gate count increases. Chip power consumption will increase 30% over 5 years. Processor costs decreasing at 29%/year. Chips: Faster, Cheaper but Hotter

5. Mesa Lab Power Limit

6. The Problem with the Mesa Lab Facility • NCAR’s machine room was built during the era of low power/high cost computing systems (1976). • 13000 sq ft • 1.2 MW computer power limitation • 80 W/sq ft cooling design • We live in an era in which high power/low cost computing systems are available. • It is not cheap to modify the power infrastructure of Mesa Lab Computing Facility, nor does it make sense without expanding the machine room floor. • Site situated on bedrock. • If we do nothing, the Mesa Lab power and design limitations will become a serious constraint on NCAR’s scientific computing capacity. This is true for both future vector and microprocessor systems.

7. Recap of Phase 1 Conclusions • Three Driving Issues for a Facility Expansion • Geoscience: Exponential demand for supercomputing • Technology: CMOS is getting hotter and cheaper • Engineering: Mesa Lab facility design is obsolescent • Recommended responses on three timescales • Short term (1 year): • Mesa Lab risk mitigation (RMH, Inc.) • Medium term (2 years): • Lease options (Staubach Real Estate) • Colocation (Inflow, Inc., Level 3) • Long term (3-5 years): • Refurbishment of existing structures (RNL Design, Inc.) • New construction (Uptime Institute, Inc.)

8. Phase 2: Analysis of Facility Options

9. Phase 2 Conclusions • Center Green 3 was not as capable as hoped • $26M to create a CG3 data center smaller than ML facility. • No room for expansion • Existing Computer Room Spaces on Market • Most not adequate for anticipated power densities or loads. • One that was acceptable was purchased by another entity. • Strategy Moving Forward • Act to maximize life and capabilities of Mesa Lab facility, • Move forward on building a data center expansion.

10. Preservation of theMesa Lab Facility • Perform a Risk Assessment • Ensure capability of reliably housing ~ 1 – 1.2 MW of computing • Evaluate the capabilities of infrastructure equipment

11. Phase 3: Facility Design Process

12. Land for 20 Years Building Shell for 10 Outfit for 5 Design Concepts Strategy preserves initial capital investment as computing hardware evolves rapidly. Majority of cost in this type of facility in outfitting mechanical and electrical systems.

13. M/E Space Computers Staff/Support Facility Design Concept 20,000 sq. ft. 4MW 20,000 sq. ft. 4 MW 20,000 sq. ft. 4 MW 20,000 sq. ft. 4 MW 20,000 sq. ft. 4 MW 20,000 sq. ft. 4 MW 20,000 sq. ft. 20,000 sq. ft. 20,000 sq. ft. Module 2 Module 3 Initial Module Expand laterally

14. Mechanical /Electrical Data Center Network Operation Center Office Space Architectural Detail

15. Phase 2 Addition Phase 3 Addition Architectural View

16. Next Steps • Conceptual Design & Cost Estimate is due as I present this • Developing full report on findings • Develop funding strategies/options • UCAR Management • Board of Trustees • NSF