1 / 21

Yellowstone Capability and capacity for the atmospheric and related sciences

Yellowstone Capability and capacity for the atmospheric and related sciences. David Hart dhart@ucar.edu NCAR/CISL User Services SC13 — Denver, CO — Nov. 19-20, 2013. Outline. Yellowstone environment Metrics for management and planning Users and fields of science

renata
Download Presentation

Yellowstone Capability and capacity for the atmospheric and related sciences

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. YellowstoneCapability and capacity for the atmospheric and related sciences David Hartdhart@ucar.eduNCAR/CISL User Services SC13 — Denver, CO — Nov. 19-20, 2013

  2. Outline • Yellowstone environment • Metrics for management and planning • Users and fields of science • Availability, utilization, and efficiency • Job mix: Capacity v. Capability • Wait times and user satisfaction • Disk and archive growth

  3. Yellowstone environment Yellowstone HPC resource, 1.50 PFLOPS peak 4,536 nodes: 2 x 8-core Intel Sandy Bridge, 32 GB GLADE Central disk resource 11 PB, 90 GB/s bandwidth Geyser 16 nodes 4 x 10-core Westmere 1 TB Caldera 16 nodes 2 x 8-core SNB 64 GB 2 NVIDIA GPUs NCAR HPSS Archive 100 PB capacity ~15 PB/yr growth 1Gb/10Gb Ethernet (40Gb+ future) Data Transfer Services Science Gateways RDA, ESG Partner Sites Remote Vis XSEDE Sites High Bandwidth Low Latency HPC and I/O Networks Mellanox FDR InfiniBand fat tree and 10Gb Ethernet

  4. Yellowstone users

  5. Time to start, duration of use

  6. Fields of science

  7. ASD v. Other use

  8. Availability, utilization & efficiency Averages Availability: 97.0% ± 0.9% Utilization: 81.1% ± 1.6% User: 40.9% ± 1.1% FP Efficiency: 1.65% ± 0.1%

  9. Daily utilization pattern

  10. Analysis cluster availability & utilization

  11. Yellowstone job mix David L Hart. 2011. Measuring TeraGrid: workload characterization for a high-performance computing federation. Int. J. High Perform. Comput. Appl. 25, 4 (November 2011), 451-465. DOI=10.1177/1094342010394382

  12. Capability versus capacity Capacity Capability David Hart. 2011. Deep and wide metrics for HPC resource capability and project usage. In State of the Practice Reports (SC '11). ACM, New York, NY, USA, Article 1, 7 pages. DOI=10.1145/2063348.2063350.

  13. Analysis cluster activity

  14. Job wait times

  15. User satisfaction metrics Ernemann, C., Hamscher, V., and Yahyapour, R. (2004). Benefits of Global Grid Computing for Job Scheduling. In Proceedings of the Fifth IEEE/ACM International Workshop on Grid Computing (GRID'04). Washington, DC: IEEE Computer Society, pp. 374-379.

  16. GLADE disk growth

  17. GLADE weekly growth

  18. HPSS daily activity

  19. HPSS weekly growth

  20. Multi-resource tracking

  21. Questions? Additional information • David L. Hart, Pam Gillman, and Erich Thanhardt. 2013. NCAR storage accounting and analysis possibilities. In Proceedings of the Conference on Extreme Science and Engineering Discovery Environment (XSEDE '13). ACM, New York, NY, USA, Article 55, 7 pages. DOI=10.1145/2484762.2484802 • David Hart. 2011. Deep and wide metrics for HPC resource capability and project usage. In State of the Practice Reports (SC '11). ACM, New York, NY, USA, Article 1, 7 pages. DOI=10.1145/2063348.2063350 • David L Hart. 2011. Measuring TeraGrid: workload characterization for a high-performance computing federation. Int. J. High Perform. Comput. Appl. 25, 4 (November 2011), 451-465. DOI=10.1177/1094342010394382 • David L. Hart. 2012. Longitudinal user and usage patterns in the XSEDE user community. In Proceedings of the 1st Conference of the Extreme Science and Engineering Discovery Environment(XSEDE '12). ACM, New York, NY, USA, Article 53, 7 pages. DOI=10.1145/2335755.2335851

More Related