Columbia University’s Advanced Concept Data Center Pilot Project

1. Columbia University’s Advanced Concept Data Center Pilot Project May 15, 2009

2. Agenda 1:00pm Introductions 1:15pm Welcome and Opening Remarks Candace Fleming, Vice President, CUIT David Hirsh, Executive Vice President for Research Nilda Mesa AVP, Environmental Stewardship 1:35pm Green DC Program presentation Alan Crosswell, Chief Technologist, CUIT 2:20pm CU Data Center Ian Katz, Manager, CUIT Data Center Current Status 2:35pm NYSERDA Project Overview Richard Hall, Project Manager CUIT 3:00pm Closing Comments & Questions from guests Victoria Hamilton, Research Initiatives Coordinator The “Big Potential” Comments from NYSERDA (tentative) 2

3. Introductions May 15, 2009

4. External Visiting Committee • Laurie Kerr, Senior Policy Advisor for Energy and Green Buildings, NYC Mayor’s Office of Long-Term Planning and Sustainability • Vace Kundakci, Assistant Vice President for Information Technology and Chief Information Officer, City College of New York/City University of New York • Timothy Lance, President and Board Chair, New York State Education and Research Network (NYSERNet) • Marilyn McMillan, Associate Provost and Chief Information Technology Officer, New York University Role: The External Visiting Committee is charged with maintaining skepticism about our plans and our accomplishments from the perspective of external institutions. To the extent we demonstrate feasibility and potential impact, they will serve as powerful ambassadors to other institutions. 4

5. Internal Advisory Group Wilmouth A. Elmes, Associate Vice President of Engineering/Technical Services, Manhattanville Development Project Doug McKean, Director, Capital Project Management Arthur M. Langer, School of Engineering and Applied Science, Senior Director of the Center for Technology, Innovation, and Community Engagement and Faculty & Assoc Director, Executive Masters of Science in Technology Management at the School of Continuing Education Nilda Mesa, Assistant Vice President, Environmental Stewardship Scott W. Norum, Chief Administrative Officer for Arts and Sciences and Vice President, Office of the Vice President for Arts and Sciences Leonard Peters, School of Business, Associate Dean and Chief Information Officer – Information Technology Role: The Internal Advisory Groupserves two primary functions. Not only do we anticipate they will prove a valuable sounding board for the execution of the pilot (implementation guidance/expert judgment), but we expect them to pose the hard questions about issues surrounding scale-up(feedback). 5 5

6. Research Faculty User Group Liam Paninski, Associate Professor, Dept. of Statistics Lei Cong, Staff Associate, Dept. of Statistics Kathryn V. Johnston, Associate Professor, Dept. of Astronomy Mary Putman, Clare Boothe Luce Associate Professor of Astronomy Greg L. Bryan, Assistant Professor , Dept. of Astronomy Role: Responsible for vetting the results of the combined HPC cluster for the Department of Astronomy and the Department of Statistics 6 6

7. Working Group Principal Investigator Alan Crosswell, Associate Vice President and Chief Technologist, CUIT Co-Principal Investigator Victoria Hamilton, Director of the Office for Research Initiatives (ORI) Richard Hall, Project Manager Anthony Cirillo, AVP Systems and Operations Peter Miner, Senior Director, Systems Engr Ian Katz. DC Facilities Manager Melissa Metz, Director of Systems Engineering Fran Ovios, Director of Systems Engineering Stew Feuerstein, Director and Chief Architect Joseph Rini, Director of Network Engineering Megan Andersen, Research Systems Administrator Fran Fitzgerald, CU Facilities Project Manager Role: Day-to-day responsibility for planning, executing and measuring required by the proposal. The Columbia University in-house team will perform much of the work. 7 7

8. Project Structure

9. Welcome & Opening RemarksCandace FlemingVice President and CIO, Columbia University Information TechnologyDavid HirshExecutive Vice President for ResearchNilda MesaAVP, Environmental Stewardship

10. Green DC Program presentationAlan Crosswell, Assoc VP and Chief Technologist, CUIT

11. CU Data Center - Current StatusIan Katz, Facility Manager, CUIT Data Center

12. NYSERDA Project OverviewRichard Hall, ProjectManager, CUIT

13. Project Objectives Become well versed in data center efficiency, design techniques and assessment metrics such as those espoused by Green Grid, ASHRAE and others (PUE, DCeP, etc.). Establish baselines and continuously measure several power and cooling variables to enable the study of historical trends and produce a data set to aid in further analysis of data center electrical and heat loading changes over time. Implement the recommended and advanced best practice improvements and measure the degree of efficiency achieved when these techniques are applied in our data center Validate claims of the degree of power and cooling efficiency improvements that are achievable through best practices and advanced data center techniques. Utilize results to inform subsequent phases of the central data center re-fitting and design of future data center space Provide training and guidance for other NYS data center operators, especially peer higher education institutions 13 13

14. Project Scope Include participation and critique from influential skeptics Update the Data Center’s aging infrastructure (modular approach) Replacement of thirty servers, each of which is at the end of their useful life Implementation of nine high-density equipment racks support current and future research computing clusters consolidating the University’s server needs for High Density Computing (HDC). Address electrical distribution systems to increase efficiency Improve and or replace inefficient cooling systems Installation of instrumentation for power and cooling systems to monitor consumption and performance to establish metrics to measure our progress toward PlaNYC. Realize energy and environmental gains while demonstrating economic and operational feasibility in infrastructure improvements, innovative computational equipment, centralized administrative systems and dispersed research clusters. 14 14

15. Project Plan To achieve the overall project objectives, CUIT has broken down the project into major tasks. Inventory Create detailed physical inventory of existing in-scope server racks Create data center baseline (formulated from above inventoried equip) Instrument server power consumption Install network monitored power monitors for each server Perform data collection at 5-min intervals using one or more of the following methods: Instrument server input air temperature and overall data center chilled water heat load Install server input ambient air temperature for each server Install BTU metering on data center supply and return lines Perform data collection at 5-min intervals Establish overall Data Center profile Utilize equipment load results to establish baselines Develop PUE ratio for entire data center & inventoried servers Implement 9 racks of high power density in-row cooling Install 9 server racks outfitted for high power density Provide in-row cooling subsystem for 9 server racks Implement measurement system for in-row cooling, BTU rejection and electrical consumption 15 15

16. Project Plan Cont’d Replace 30 “old” servers and measure efficiency improvement. Consolidate the replacement servers into high density racks and re-implement the same IT services Take measurements of before-and-after power consumption Document expected and actual efficiency improvement Compare old and new research clusters Benchmark applications on new Astronomy/Statistics HPC cluster “kWh consumed” comparison with old Astronomy server cluster Implement server power management Install server BIOS/high level power management feature upgrades to servers (identified in task 2) Document benchmarks Increase chilled water set point for high power density racks and measure Document measured before-and-after BTUs Communicate results Share results with key stakeholders 16 16

17. Project Schedule Project Approval 4/1/09 Inventory 5/15/09 Instrument Power 7/30/09 Instrument Cooling 9/30/09 Data Center Profile 9/30/09 Nine Racks 1/31/10 Replace 30 Servers 5/30/10 Compare Clusters 10/31/09 Server Power Mgmt 9/30/10 Increase 5-degrees 3/31/10 Project Review 9/30/10 Project close 10/1/10 17 17

18. Closing CommentsVictoria Hamilton, Director of the Office for Research Initiatives (ORI)