120 likes | 132 Views
A W eather climate change I mpact S tudy at E xtreme R esolution (WISER) Alan Gadian , Ralph Burton & James Groves NCAS, Leeds. Computing requirements for WISER. Background.
E N D
JASMIN– forum–2015-02-02 – 1 of 12 A Weather climate change Impact Study at Extreme Resolution (WISER) Alan Gadian, Ralph Burton & James Groves NCAS, Leeds Computing requirements for WISER
JASMIN– forum – 25/9/12 – 2 of 12 Background JASMIN supports for the WISER (Weather climate change Impact Study at Extreme Resolution) project. This project will embed a nested simulation using the NCAR WRF (Weather Research and Forecasting) model within climate reanalyses and predictions from ECMWF and NCAR (US National Center for Atmospheric Research). The nested WRF simulations will cover +/- 680 of latitude, 20 km resolution at 300 latitude, 8km at 680 latitide, and nested down to 4 km resolution over western Europe (d02), capturing almost all important weather scales.
JASMIN– forum – 25/9/12 – 3 of 12 NCAR parallel simulation to study hurricane formation Collaborations with NCAR (National Center for Atmospheric Research) (NESL .. Greg Holland et al. ) and EFAS (European Flooding Awareness System, Thielen et al.) 45S - 45N 36 km resolution 51 levels 10 mb TOA 2000 - 2005 Reynolds SST data Periodic EW boundary
JASMIN– forum–2015-02-02 – 4 of 12 What are we trying to do with WISER? • Results from NCAR simulation, for domains of 12km and 36km • A typical weather system pdf • The Problem: Typical regional climate models do not capture the most extreme events observed. • The pdfs are truncated at high end due to: • resolution – cannot resolve the relevant processes; • • capacity – do not include the relevant processes; • • rarity – cannot be run long enough to sample
JASMIN– forum–2015-02-02 – 5 of 12 Validation runs will include 1970-present day and predictive runs will include present day to 2080. Runs are planned in 10 year blocks, each with several completed simultaneously on 3000-5000 cores. Each 1 year block will use ~300 hours on 4000 cores on Archer. The work is part of an ongoing collaboration between NCAR's Nested Regional Climate Modelling group and NCAS. The project aims to provide new insight into the technology of regional climate modelling (especially from the validation “hindcast” runs) and provide statistical predictions of the European regional impacts of climate change. Typical job pre-processing in three stages ( Jasmin North and at RAL ) running the job ( Cray ) post-processing (Jasmin North and at RAL ) Computing requirements for WISER
JASMIN– forum–2015-02-02 – 6 of 12 Preprocessing: This involves running “geogrid”, “ungrib”“metgrid” and “real”, using initial ECMWF or CCSM data which needs to be transferred in. This is carried out at RAL and Jasmin North. Jasmin is ideally suited with fast disc access and as the preprocessing is on the large domains (i.e. suitable for < a few tens of cores. Main runs: The data is transferred to Archer. Each month takes about 30 hours on ~ 4,000 cores ~171 nodes. Data is dumped every 6 hours ( domains 1 & 2 , each ~ 6 Gb ), in total about 1.5 Tb. (This transfer to RAL is “slow” .. Investigate bbcp /multiple rsyncs to optimise speed of transfer.) Postprocessing: Carried out at Leeds. Data transfer to Jasmin-North is fast. Computing requirements for WISER
JASMIN– forum–2015-02-02 – 7 of 12 Questions please? Computing requirements for WISER
JASMIN– forum–2015-02-02 – 9 of 12 JASMIN structure and links
JASMIN– forum – 25/9/12 – 10 of 12 Jasmin North WISER
JASMIN– forum–2015-02-02 – 11 of 12 Jasmin – North setup
JASMIN– forum – 25/9/12 – 12 of 12 Machine room structure, leading to outside connections 1Gb lightpath to Jasmin - South Gnodal 10Gb University Network R610 Panasas shelf 1 R610 R610 Panasas shelf 2 R815 Panasas shelf 3 ~100Tb Plus several hundred Tb of user discs available for permanent storage
JASMIN– forum – 25/9/12 – 16 of 12 Bottlenecks Issues:- Transfer of CCSM data files from the US to Jasmin Transfer of data files to and from Archer and RAL. RAL – Leeds 1 Gb lightpath is good. Hartree – RAL was slow ( 6 months ago, not sure now) 3. Poor client performance on the R815 (having said that better than university facilities) 4. Network speed issues in and around campus. Care is needed to minimise this (e.g. using quiet times)