Objectives & generalities about phasing and The phasing roadmap

Objectives & generalities about phasing andThe phasing roadmap Claude Fischer

Content: • Software interaction IFS-Arpège // LAM • General overview and motivation • Rules • Phasing • Validation • Code management • The phasing roadmap • Appendices: • Technical stuff (about VAR) Aladin/LACE/Hirlam Maintenance training workshop

Software interaction IFS-Arpège // LAM • General view & motivation: • IFS evolution forces a continuous, sometimes drastic, code optimisation and cleaning • IFS provides powerful state-of-the-art software, both scientific and technological • In detail, for the LAM: observation operators are IFS, basic configurations, singular vectors, CONGRAD etc… • Sometimes, ideas are first tested in the LAM: NH model, digital filters • Yet, not every development is easily transfered from the IFS to the LAM • This « marriage » creates rights and duties Aladin/LACE/Hirlam Maintenance training workshop

Software interaction IFS-Arpège // LAM • Rules or practice: • IF (LELAM) THEN; CALL ETOTO; ELSE; CALL TOTO; ENDIF • LAM-specific routines go into a separate fortran library, unseen from ECMWF • No LELAM key below the level of gridpoint scan (SCAN2MDM/TL/AD) => LRPLANE • Duplicated code must be avoided !! • The issue(s) for modularity: with respect to a functionality or a desired degree of freedom, not with respect to IFS v/s LAM • Any configuration which does have a sense for both global and LAM should not be coded as a by-product of LELAM ! Aladin/LACE/Hirlam Maintenance training workshop

Software interaction IFS-Arpège // LAM • Scientific « feeling »: apply basic logic ! • Spectral space differences almost everywhere: • Mean wind vector (SPUB, SPVB) since (m=0,n=0) is not defined in bi-Fourier space • Allocations, pointers, do-loop indexes all are to be re-defined for the LAM -> most spectral routines are differentiated (eg: SUJBCOV/SUEJBCOV, SPCM/ESPCM) • Gridpoint space: more subtle: • Existence of lateral boundaries (eg: search of origin point in SL – ELARCHE) • Plane geometry projection and associated map factor (eg: SL interpolation weights – ELASCAW) • Existence of the E-zone (Extension zone for biperiodisation) • Rotation of geographical North/South and East/West with respect to model grid N/S and E/W Aladin/LACE/Hirlam Maintenance training workshop

Phasing (1): practical matters • When and how long ?: • Twice per year (dates pending ECMWF/Météo-France discussions) • 3-6 weeks (visits), but in practice 2-3 months for completion in Toulouse • How many phasers ?: • 4-6 initial phasers (= visitors from partner countries), plus 2-3 additional « late » phasers • a ~ bigger investment of GMAP staff • Phaser stays are financially supported by MF, LACE or HIRLAM • Phasing is not very popular, and depends heavily on the people’s willingness to leave home … for a sacrifice • Separate with/without observations (assimilation usually comes after forecast configurations) Aladin/LACE/Hirlam Maintenance training workshop

Phasing (2) • Principles: • At the very beginning: check and understand the modifications in a new IFS/Arpège cycle • Perform code and scientific analysis, and then adapt to LAM setup and/or LELAM keyed code when needed (sometimes trivial … or not) • Recode some Aladin at identical scientific content for code compliancy with Arpège • Analyze and « uncode » code clashes Aladin/LACE/Hirlam Maintenance training workshop

Phasing (3) • Phasing examples: SUBROUTINE TOTO(ZTAB,K) <<changed into>> SUBROUTINE TOTO(ZTAB,K1,K2) REAL ZTAB(K) <<changed into>> REAL ZTAB(K1,K2) CALL TITI(ZTAB) <<unchanged>> SUBROUTINE ETOTO(ZTAB,K) <<changed into>> SUBROUTINE ETOTO(ZTAB,K1,K2) REAL ZTAB(K) <<changed into>> REAL ZTAB(K1,K2) CALL TITI(ZTAB) Aladin/LACE/Hirlam Maintenance training workshop

Phasing (4) • Phasing examples: N is total wavenumber (Arpège)  function FKSTAR(N,M) (Aladin/LAM) SUBROUTINE TOTO WEIGHT=1.+MIN(1,M) <<inserted>> Y=X/S(N) <<changed into>> Y=WEIGHT*X/S(N) SUBROUTINE ETOTO WEIGHT=2**MIN(1,N) * 2**MIN(1,M) <<inserted>> Y=X/S(FKSTAR(N,M)) <<changed into>> Y=WEIGHT*X/S(FKSTAR(N,M)) Aladin/LACE/Hirlam Maintenance training workshop

Validation • Arpège and LAM sanity-check validation is done for both at the same time : • Systematic validation of all model configurations, which do not use observation handling, is performed by the « mitraillette » tool for LAM’s: automatic launching of a number of configurations (forecast e001, TL and AD tests, fullpos, NH model, 2D plane vertical model, adiabatic or with physics, multi-processor etc…) • Basic validation of screening and 3D-VAR minimisation, ideally at the same time as Arpège 4D-VAR (needs the observation handling to be validated before) • Quality assurance (QA) tests: • After the core phasing period, since this requires again ~2 months of work (run long assimilation experiments, test various operational versions) => both at MF and in partner centers (HIRLAM, LACE) • Not all cycles are QA ! Aladin/LACE/Hirlam Maintenance training workshop

Code management • Source code repository in Toulouse: clearcase • Arpège and LAM cycle numbers: CYxxTy • IFS has its own cycles in Reading (CYxxRz) • Support team for maintenance of cycles and libraries, and interfacing with operations: « GCO » -> 3 persons (2 involved in code and conf management) • User-friendly compilation environment on the high performance platforms (« gmkpack ») • Export versions are defined and made available for the Aladin/Hirlam partners (mostly for QA cycles) Aladin/LACE/Hirlam Maintenance training workshop

The phasing roadmap for a common IFS-Arpège cycle • … is a Word document (check on the Aladin website or ask for it !) • Lists the sequences of tasks during phasing: • Pre-phasing coordination (list the input contributions and fine-tune the dates) • Installation of all codes under SCM and initial source code phasing; possibly some local porting of IFS version • First compilations, validations, debugging • Phasing of LAM code w/r to IFS changes • After first validations are successful, start some porting to other platforms; resume validation • Porting to ECMWF computer (usually with Hirlam) • Send new (pre-)cycle to ECMWF; check numerical performances; prepare some reporting of phasing work Aladin/LACE/Hirlam Maintenance training workshop

Technical stuff Aladin/LACE/Hirlam Maintenance training workshop

Global // LAM code interfacing for VAR • Architecture: obshortl Slightly different code: LELAM key coupling Completely different code Fully shared dataflow between IFS and Aladin (especially in gridpoint space), but quite separate dimensioning and addressing in spectral buffers (spherical versus bi-Fourier). Coupling code is of course only LAM. Aladin/LACE/Hirlam Maintenance training workshop

Global // LAM code interfacing for VAR Jb=; Jb=2 • Simulator and model: obshortl dx=B1/2. Minimisation obsvtl cpgtl Var inner loop (SIM4D) J=Jb+Jo; J ecoupl1 einv_trans H.dx H.dx edir_trans GradJo=H*.R-1. (y-H(x)-H.dx) Jo=B1/2. GradJo espcmtl Aladin/LACE/Hirlam Maintenance training workshop

Global // LAM code interfacing for VAR • Change of variable: cv2spa chavarin cvar2in sqrtbin cvaru3i jgcori ejgvcori ejghcori jgnrsi etransinv_jb cvargptl etransdir_jb ebalstat ebalvert spa1=ylvazx%lamcv*tmeanuver addbgs Aladin/LACE/Hirlam Maintenance training workshop

Phasing of LAM 3D-VAR • ODB & bator need to work (almost same as Arpège, plus LAMFLAG) • At every new cycle with IFS, a careful inspection of B-matrix setup and change of variable routines, SPECTRAL_FIELDS and CONTROL_VECTOR structures, specific observation-related code is needed (~ 1 week of work). • Additionally, TL and AD LAM code may need to be checked. • LAM 3D-VAR does not use multi-incremental I/O prepared for IFS-Arpège: no WRMLPPADM/RDFPINC, no SAVMINI/GETMINI, NUPTRA>1, « traj » recomputation of innovations Aladin/LACE/Hirlam Maintenance training workshop

Objectives & generalities about phasing and The phasing roadmap