1 / 13

Plans for National HPC Services: UM vn 6.1 Installations and Performance

This document outlines the plans for the National HPC services, including the installations and performance of UM vn 6.1. It provides information on the HECToR and HPCx facilities, UM atmosphere model resolutions, and the availability of the NCAS service on HECToR. It also discusses the differences between HECToR and HPCx, UM issues and compilers, and plans for porting UM to HECToR.

vduane
Download Presentation

Plans for National HPC Services: UM vn 6.1 Installations and Performance

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Plans for the National NERC HPC services UM vn 6.1 installations and performance UM vn 6.6 and NEMO(?) plans

  2. National HPC Facilities HPCx (NERC ~10% share) Phase 3 Phase 4 2007 2008 2009 2010 2011 HECToR (NERC ~20% share) Phase 1 Phase 2 Black Widow (Vector) UKMO Shared (NERC <10% share)

  3. UM atmosphere model resolutions low N48 -> N96 -> N144 -> N216 high 1 node on HPCx = 16 processors UM version 6.1 on HPCx, phase2a IPCC like STASH and with climate meaning

  4. When will the NCAS service on HECToR be available? • HECToR service started on 16th October 2007. • NERC will provide initial HECToR allocation during the NERC HPC steering panel to be held 22nd November 2007 • NCAS service, via the PUMA UMUI, will start with UM versions 4.5 and 6.1. • NCAS service for UM version 6.6 may begin at Easter 2008, depends on Met Office delivery of new versions

  5. What is HECToR phase 1 service? A Cray XT4 with 11,328 cores, each acts as a single CPU, on which NERC has ~20% share of the allocation. The processors are AMD 2.8 Ghz Opterons. HECToR has a total of 32 Tbytes of memory and has a peak speed of 59 Tflops. The machine is run by Edinburgh (EPCC) and Daresbury and so has the same administration process as HPCx using SAFE (Service Administration From EPCC) So it has the same look and feel as HPCx. High level support is provided by NAG, which will cause a significant culture change for NCAS.

  6. What is the HECToR service like compared to HPCx? • it runs SUSE linux (so we may need some script changes) • it uses MPICH2 for the processor interconnect • (so we need to look at the UM scalability issues) • it has a new file system (so we need to explore UM I/O issues) • it doesn’t (yet?) have an archive system • (this is being discussed with NERC, HECToR and EPSRC) • it has 3 different compilers PGI, pathscale and gnu • (there are many UM issues to explore with all these options) • system software is controlled by modules (so we need to make changes to the UM setvars) • job submission using PBS (so we will make changes to UM scripts and the UMUI) • parallel jobs are launched with aprun not mpirun (so we have to change the UM scripts) • no serial queue (yet!) (so we may have to change the way we compile the Um and what about the simple models?

  7. UM Compiler issues Survey from Polyhedron Software Results from a UM version 4.5 code sample

  8. Other NCAS UM issues f77/ ftn module switch We currently testing both compilers. Pathscale compiler • Basic UM PGI options now selected after rounding problems • we need now to look at portability/reproducibility • do some validation runs PGI compiler • UM vn 6.1 • Hadgem -> Hadgem1a • Higem –> Higem2 • NUGAM…… • Weather jobs? • UKCA? • UM vn 4.5 • Hadam3 • + Hadam3P PRECIS, Hadrm3 Hadam4 2) Hadcm3 + preind QUEST? Moses 2.1, 2.2 Famous/QUEST UM vn 6.3, 6.6…….. L64, Stochem

  9. NCAS Plans for Porting UM to HECToR • Set up central UM userid • hum • Install and test UM vn 6.1 and 4.5 • Focus on portability, performance and scalability issues • - there are currently many different queues but we need to • provide advice to users at different resolutions • Work out disk space strategy • - how are we going to manage users personal archives? • - what do we need to do with ECMWF and UKMO data? • Design the FCM build system for HECToR for UM vn 6.3, 6.6 • - timetable of the UK Met Office • - timetable for UKCA, CASCADE, Higem, GSUM, QUESM

  10. 3 Gbyte files 1.6 Gbyte files Time spent (secs) for I/O - UM atmosphere N216 L38  I/O is an issue on different computers hence GSUM will optimise I/O as well as provide a tuneable I/O strategy On HECToR

  11. On HECToR • Current Issues • Robustness of the system • hardware still not that reliable but improving • lustre file system still having teething problems • support rather ‘green’ • No management committee in place to drive improvements • No long term storage solution • UM installation (vn45. and vn6.1) complete • but validation is still not complete • Higem run still running • UKCA, chemistry solvers are taking 31 x HPCx ! • UMCET (ensemble framework) needs re-working - UM vn 6.6 using FCM should be installed by Easter 2008

More Related