1 / 10

REFS application: NOA

REFS application: NOA. PSC0 3 , 1 5- 1 6 January 200 9 , Bucharest. Kotroni V. Floros V., Lagouvardos K. National Observatory of Athens. Objectives. Technical objectives

valerieguerrero
Download Presentation

REFS application: NOA

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. REFS application: NOA PSC03, 15-16 January 2009, Bucharest Kotroni V. Floros V., Lagouvardos K. National Observatory of Athens The SEE-GRID-SCI initiative is co-funded by the European Commission under the FP7 Research Infrastructures contract no. 211338

  2. Objectives Technical objectives • Develop two operational regional ensemble model chains based on BOLAM and MM5, using initial and boundary conditions from the global ensemble model members from NCEP-USA. • Develop with SEWA and the Hydrometeorological Institute of Montenegro the postprocessing that will gather all ensemble members and produce a super-ensemble.

  3. REFS – Progress • Developped a glite UI where we installed MPICH-1.2.7, PGI-5.2, and GANGA 5. • Compiled BOLAM and MM5 model on the UI. • Developped the scripting for the model workflow: • Downloading the initial and boundary conditions from NCEP • Decoding these data and preprocess them (create the initial and boundary conditions for the models) • Model run

  4. REFS – Progress • Developped the REFS execution workflow for Bolam and MM5 models Local Job management: managed by a Python module that uses GANGA to submit a set of jobs to the grid. Prepares a compound grid job comprised of a number of subjobs, one for each ensemble member that should run. (a) prepares the ensemble members, (b) submits the jobs to the grid and (c) periodically monitors the execution until all members complete either successfully or unsuccessfully. Execution on the grid: When the job arrives on the WN it executes a script. LILO (lead-in lead-out) script is responsible (a) to prepare the environment for the model, (b) retrieve the model binaries, (c) unpack them, (d) set the appropriate environment variables, (e) execute the model workflow and (f) finally store the results on the LFC.

  5. REFS – Progress • This procedure has been developped for both Bolam and MM5 models. • Use of the SEE-GRID-SCI infrastructure and namely the hellasgrid sites for the moment. --------------------------------------- • Systematic testing of the response of the SEE-GRID VO infrastructure on daily requests of the whole workflow of BOLAM model regional ensemble prediction system. • IF THE INITIAL AND BOUNDARY CONDITIONS ARE COMPLETELY FTP’ED FROM NCEP-USA ALL 10 ENSEMBLE MEMBERS FINISH SUCCESSFULLY • testing of the response of the SEE-GRID VO infrastructure on daily requests of the whole workflow of MM5 model regional ensemble prediction system. • IF THE INITIAL AND BOUNDARY CONDITIONS ARE COMPLETELY FTP’ED FROM NCEP-USA IT IS QUITE RARE THAT ALL 10 ENSEMBLE MEMBERS FINISH SUCCESSFULLY

  6. REFS – Progress • A user manual “Regional Ensemble Forecast System (REFS) Users Manual” has been prepared and distributed to the partners for commenting 2 days ago. •  A proposal for Meteo applications storage and file schema has been also prepared and discussed during the 1st EVO meeting (with contribution by B. Marovic). • On 12/1/09 the LFC space for REFS applications has been prepared and the update scripts that generate the daily output folders has been activated. These scripts run by a cron job at the UI at NOA

  7. REFS – Progress PROPOSAL FOR METEO APPLICATIONS STORAGE SCHEME The LFC will be used for storing : • Various data files produced by the models: • raw output (output produced daily by each ensemble member of a model), • post-processed output and • intermediate results from failed model executions. • Software packages. Packages containing • the model binary executables, • additional required scripts and auxiliary data. These are fetched from the LFC to the WN before the model execution.

  8. REFS – Progress /grid /meteo.see-grid-sci.eu /REFS /data /<REGION> /rawoutput /<YYYYMMDD> <PRD>-<MOD>-RAW-<YYYYMMMDD>-<NN>.tgz /failed /<YYYYMMDD> <PRD>-<MOD>-FAIL-<YYYYMMMDD>-<NN>.tgz /artifacts /<YYYYMMDD> <PRD>-<MOD>-<YYYYMMMDD>.tgz /shared /software/<PRD>/<APPLICATION-MODULE-NAME>-<VERSION-NUMBER> Example:Output produced by the 7th member of MM5 ensemble forecasting run on 27th November 2008: NOA-MM5-RAW-20081127-07.tgz. The file is stored under directory: /grid/meteo.see-grid-sci.eu/REFS/data/SEEurope/rawdata/20081127.

  9. REFS – Problems (1) IF THE INITIAL AND BOUNDARY CONDITIONS FOR MM5 ARE COMPLETELY FTP’ED FROM NCEP-USA IT IS QUITE RARE THAT ALL 10 ENSEMBLE MEMBERS FINISH SUCCESSFULLY • So we are working on different compilations of the MPI and consequently of MM5 in order to resolve the problem. (2) THE NOMADS SERVERS FROM WHERE WE FTP THE INITIAL AND BOUNDARY CONDITIONS ARE NOT VERY STABLE. SOMETIMES THE DATA ARE NOT AVAILABLE. • A high availability server has been created in NCEP that provides the needed data but in a new format (GRIB2 instead of GRIB1) so some rescripting is needed.

  10. Activities and Time-Plan • Up to M12: • improve the scripts for BOLAM and MM5 • use the new GRIB2 initial data from NCEP • systematically check the pre-operational runs of these two models on the GRID • start working on the post-processing of all four model outputs towards the super-ensemble

More Related