NERSC: Current Status and Future Expectations

1. NERSC: Current Status and Future Expectations William T.C. Kramer NERSC/LBNL May 29, 2003 NERSC User Group, Chicago, IL

2. HPPS 12 IBM SP servers 15 TB of cache disk, 8 STK robots, 44,000 tape slots, 20 200 GB drives, 60 20 GB drives,max capacity 5-8 PB Visualization Server – “escher” SGI Onyx 3400 – 6 Processors/ 2 Infinite Reality 4 graphics pipes 24 Gigabyte Memory/4Terabytes Disk ETHERNET 10/100 Megabit HPSS HPSS STK Robots SGI SYMBOLIC MANIPULATION SERVER FC Disk Gigabit EthernetJumbo Gigabit Ethernet OC 48 – 2400 Mbps ESnet Testbeds and servers PDSF400 processors (Peak 375 GFlop/s)/ 360 GB of Memory/ 35 TB of Disk/Gigabit and Fast EthernetRatio = (1,93) IBM SP NERSC-3 – “Seaborg” 6,648 Processors (Peak 10 TFlop/s)/ 7.8 Terabyte Memory/64Terabytes of Disk Ratio = (8,7) LBNL “Alvarez” Cluster 174 processors (Peak 150 GFlop/s)/87 GB of Memory/1.5 terabytes of Disk/ Myrinet 2000 Ratio - (.6,100) Ratio = (RAM Bytes per Flop, Disk Bytes per Flop) NERSC System Architecture NERSC User Group Meeting

3. Short Review • Implemented enhancements to Seaborg – in production one month earlier than planned • Added 29M MPP hours to allocations in FY 03 • Increase in FY 04 due to full year of operation • Scalability program and reimbursement program • Implemented OC-48 link to ESnet • Added 200 GB tape drives to HPSS • Readjusted queues on Seaborg to emphasize large jobs • Progress on GUPFS and Grid Function NERSC User Group Meeting

4. HPSS I/O Activity May 2000 – Total = 10.8 TB mcurie Gseaborg J90s 2TB 1TB 1TB ARCHIVE HPSS 1.8TB Other NERSCdata WAN Traffic 5TB NERSC User Group Meeting

5. HPSS I/O Activity May 2003 – Total 62.9 TB Seaborg escher Other data 850GB 27TB 5TB 6 times more data! ARCHIVE HPSS 10TB PDSF Cluster WAN Traffic 20TB NERSC User Group Meeting

6. Review of the 2001 NERSC Strategic Proposal

7. NERSC Strategic Proposal NERSC User Group Meeting

8. NERSC Strategic Proposal NERSC User Group Meeting

9. NERSC Strategic Proposal NERSC User Group Meeting

10. Proposed and Real Expenditures NERSC User Group Meeting

11. Observations • Average annual amount spent on the computational systems is now ½ of what it was in the past • Ongoing cost increased and will continue to increase • Electrical and space costs due to larger systems (268% and 158% over 5 years) • Media costs for tape storage (462% over 5 years) • Network connections for more data intensive science (from nothing) • Increased efforts for cybersecurity (400% over 5 years) • In the current scenario there are no funds for NERSC-5 or other significant improvements NERSC User Group Meeting

12. Computational Implications • First available funding for additional resources will be FY 2008 rather than FY 2005 • In FY 2008, less than 8% of the NERSC budget will be available for NERSC-5? – instead of 30-40%. • Allocation implications • FY 04 – will increase to ~110,000 MPP hours because N3E will be in service for the entire year • FY05-FY08 – static at 110,000MPP hours • Only incremental improvements after FY 08 NERSC User Group Meeting

13. Storage Implications • No drivers from increased computational resources – thus improvements only for efficiency • No explicit charter to deal with the increased flow of data from the outside • Limit media (tape) costs • Focus on higher density tapes • Repack data from less dense tapes • Continue to use old tapes rather than decommissioning – risk more errors • Less capacity in robots • Quota enforcement • Slower growth in SRUs NERSC User Group Meeting

14. Networking and other Infrastructure Implications • Defering upgrade to OC-192 indefinitely • No planned LAN networking improvement • Lack of ports will impact adding systems • No planned upgrades/replacement to other servers • Limit new software or increasing licenses • Probably add only free software that we feel we can support • Limited on ability to support other special systems NERSC User Group Meeting

15. Service Implications • Continue excellent services and support • Both standard and special services • Focus on optimization and helping people make most effective use of the resources • Scalability will continue to be very, very important • Will look at things systematically as well as working with individual users • Continue software improvements • GUPFS – will save costs as well as increase function once deployed • Grid functionality being deployed • Continue to do what is necessary for cyber security • Continue to advocate for more resources • For the NERSC Base community • For special opportunities NERSC User Group Meeting

16. Advocating “World Class” Funding NERSC User Group Meeting

17. Summary • The level of funding is not sufficient and now NERSC has to revise expectations • This is an issue for all SC program offices – since all programs rely on NERSC to provide resources to meet their program goals • On-going discussions with MICS and SC for some time, but there has not been much indication of improvement • We need NUG’s help to express the impacts to their program offices and to help figure out how to go forward NERSC User Group Meeting