NCAR Computation and Information Systems Laboratory Facilities and Support Overview

1. NCARComputation and Information Systems Laboratory Facilities and Support Overview UCAR Postdoctoral and Graduate Fellows March 16, 2007 Mike Page Consulting Services Group mpage@ucar.edu

2. “SCD will provide a balanced set of services to enable researchers to securely, easily, and effectively utilize community resources” CISL Strategic Plan (2005-2009) CISL’s Mission for User Support

3. Computing systems Allocations Usage Batch Queues Batch Job Submission Interactive Charges Security Data Archival Models User Support CISL Computing Systems At NCAR/CISL you’ll find world-class facilities supporting leading-edge science through high-performance computing. Navigation and usage of the facility requires a basic familiarity with a number of the functional aspects of the facility.

4. IBM Cluster 1600 system - number 39 on Top 500 list 1600 Power4 processors with 1.3-GHz clock Four floating-point operations per cycle 8.32 TFLOPs peak processing 76 8-way nodes and 25 32-way nodes shared memory on each node AIX operating system IBM XL Compiler Suite; TotalView debugger LoadLeveler Batch system 3 GB $HOME quota 120 GB /ptmp quota CISL Computing SystemsBlueskyDecommissioned March 1, 2007

5. IBM e1350 cluster SuSE Linux 128 batch nodes plus 2 interactive nodes Each node is a pair of 2.2 GHz Opteron processors 4 GB memory/ batch node; 8 GB memory/interactive node Myrinet switch (batch nodes only) PGI compiler suite, fully supported 2 GB $HOME quota 20 GB /ptmp quota LSF batch management subsystem CISL Computing SystemsLightning

6. Data analysis/post-processing system SGI Origin 3800 128 500-MHz R14000 processors 64 GB distributed/shared memory IRIX Operating System NQE batch subsystem (LSF in the near future) MIPSPro compiler suite - f77, f90, C, C++; cvd Libraries and utilities Matlab, IDL, Mathematica, NCL, NCO, flint IMSL, NAG, SCSL; netCDF CISL Data Analysis SystemTempest

7. IBM eServer p575 624 Power5 processors with 1.9-GHz clock Four floating-point operations per cycle 4.74 TFLOPS peak processing not on Top 500 list SMT technology 72 8-way batch nodes shared memory on each node AIX operating system IBM XL Compiler Suite; TotalView debugger LSF Batch system 3 GB $HOME quota 240 GB /ptmp quota CISL Computing SystemsBluevista

8. IBM eServer p575+ 1744 Power5+ processors with 1.9-GHz clock Four floating-point operations per cycle 12.16 TFLOPS peak processing 55th on Top 500 list SMT technology 100 16-way batch nodes shared memory on each node (25 @ 64 Gb, 75 @ 32Gb) AIX operating system IBM XL Compiler Suite; TotalView debugger LoadLeveler Batch system 5 GB $HOME quota 250 GB /ptmp quota CISL Computing SystemsBlueice

10. SCD Firewall Enter through roy.ucar.edu ssh only; telnet not allowed ssh, ssh -X , ssh -Y Cryptocard You must have one to access the supers Usage Resynchronization Complete information from CISL Home Page http://www.cisl.ucar.edu File Transfers Security

11. Data Archival System File server? Storing in excess of 3 PetaBytes of data Transferring more than 3 TBs of data per day http://www.cisl.ucar.edu/main/mss.html Use mss from command line or batch job scripts msrcp, msls, etc Access from application codes Asynchronous reads/writes NCAR Mass Store Subsystem(MSS)

12. Will NCAR maintain my data indefinitely? If you don’t retain your account - 1 year If you retain your account - ongoing Using the Mass Store is expensive Consider offloading One Terabyte > 300 DVDs at current density Create your own transportable media (except extreme cases) Use scp/sftp for data accessible from an SCD super Including divisional filesystems Use MSS ftp server http://www.scd.ucar.edu/docs/mss/ftp.html Mass Store Usage

13. CISL’s Research Data Archive (RDA): What, Where and HowJoey ComeauxNCAR/CISL/DSSjoey@ucar.edu

14. What is the RDA? Few examples How to access metadata Where is the RDA located and how do I access it? Community Data Portal: Another source for data Data topics

15. Collection of reference datasets used in atmospheric and related sciences Over 600 datasets currently archived 10-20 new datasets added annually The RDA is maintained by the 9 members of the Data Support Section (DSS) of CISL RDA definition

16. Many categories of data Observations Model Output Climatology Data Long time periods of obs and model output Long-term SST analyses Monthly summary of the day station data Min and max daily temps and preciptiation Satellite derived datasets Outgoing Longwave Radiation, precipitation, … Global topography and ocean depth Surface elevation, vegitation, … Station Library Location and elevation of observing station RDA Content

17. Observations In situ or remotely sensed measurements Includesmeasurements taken by : Global Surface Stations Global Upper Air Profiles (Raobs and Pibals) Global Ships Satellite Aircraft Observations

18. Atmospheric Observations • Surface Obs. • 1901 – Continuing • Global and Regional • Raob + Pibals • 1920 – Continuing • Global and Regional

20. Ocean Observations ICOADS - Synoptic observations and statistics ds540.0 & ds540.1 Observations include : SST Air Temp Specific Humidity Winds Precipitation Sea Level Pressure Psuedo stress Sensible & Latent Heat Evaporation 1960 1990

21. Ocean Analyses • NCEP and Hadley Center • 1850’s – Continuing • In Situ, Satellite, blended

22. Daily Model runs Commonly called Operational Analysis Analyses and Forecasts Global and regional DSS archived from 1970 and Continuing Reanalysis Projects Use current operational model to process observations over a long period of time Generally better than Operational Analyses DSS archive covers 1948-Continuing Model Output

23. Daily Operational Reanalysis

24. Note: all graphics constructed with NCL

25. Climate Model Output

26. Auroral Observations 1536 - 1940 ds836.0 Temperature Observations Manley, England 1659 - 1995 ds825.0 Tropical Cyclone Tracks 1851 - 2005 (Cont) ds824.1 Sunspot Data 1749 - 1993 ds834.0 Global Historical Climate Network 1700 - 2004 (Cont) Tmin,Tmax,Tmean,Precip ds564.0

27. All datasets Described with individual web pages Assigned data consultant http://dss.ucar.edu Catalogued in UCAR Community Data Portal Access All data available from NCAR MSS Accessible with CISL computing accounts Many popular products on the RDA server Direct download Consultants fill requests as needed RDA Metadata and Data

31. Access Options

33. USERS Access to RDA Meta Data DSS Server Supers BlueIce Blue-??? Data Analysis Tempest ???? Divisional Computers ??? msrcp mss:/DSS/file local_file M S S

34. Community Data Portal http://cdp.ucar.edu Metadata for data from several NCAR divisions ATD, ACD, CGD, CISL UNIDATA, CU Another Source for Data

35. Categories of Data Observations, Model Output, climatology data and much more RDA web server : dss.ucar.edu All data available from MSS and some from RDA server Most Important: Have Questions? Please Ask them!! joey@ucar.edu Wrap Up

36. Advanced visualization facilities 3D and 4D data Terascale data sets Remote visualization services Applications and libraries: Vtk, Vis5d, IDL, OpenDX, and more Expert visualization staff Production animation and imaging service dasg@ucar.edu CISL VisLab

37. SCD Home Page: http://www.cisl.ucar.edu/ High End Computing Everything you always wanted to know …. Mass Storage System Data Support Section VisLab Community Data Portal ESMF Documentation

38. DSS is available to provide access to specific datasets CSG is your frontline contact with CISL Facilities such as MSS and VisLab Software consultation What else could you need? Contact the Consulting Services Group Walk in ML 39 (basement) Call (303) 497-1278 E-mail consult1@ucar.edu Conclusion

39. Meet Some of UsCSG

40. Meet Some of UsMSSG

41. Meet Some of UsDSS

42. Questions?