GIFS-TIGGE Working Group Autumn 2005 IPO update

1. David Burridge Manager IPO GIFS-TIGGE Working Group Autumn 2005 IPO update

2. Science Plan peer-reviewed; • Implementation Plan 2004-2015 approved; • Approved annual expenditure $ 1.2M through a trust fund; • First Symposium (Montreal,2004) • Regional Committees - RAII,IV,VI • First TIGGE workshop (report issued) • Three TIGGE data Centres - phase 1 developments at CMA, ECMWF & NCAR

3. THORPEX within the WMO structure

4. Boards, working groups and key groups and meetings (continued) • The EB met in Geneva in September 2005 • MEDEX (4 October 2005) • GEOSS (links developed and support for THORPEX included in plans • The GIFS-TIGGE WG is meeting in Boulder on 15 & 16 November 2005 • A technical plan for the first phase of the TIGGE data bases is under development and will be discussed at the Boulder meeting • Southern Hemisphere Planning Meeting in Melbourne (late November 2005) • ICSC 5 meeting in Melbourne (November/December 2005) • SEA WG 1 – January 2006 • There will be a THORPEX/WCRP meeting on the MJO, organised by Julia Slingo, Mel Shapiro and the PDP WG which will be held at ICTP (Trieste – 13-17 March 2006). • There will a “kick off” workshop for all working groups which will be held in the University of Reading (20-22 March 2006; this will be followed by meetings of the SAB and the TAB (23-24 March 2006).

5. THORPEX Interactive Grand Global Ensemble (TIGGE) • Framework for international collaboration in development and testing of ensemble prediction systems • Resource for THORPEX research projects • Component of THORPEX Forecast Demonstration Projects (FDPs) • A prototype future Global Interactive Forecast System • Initially develop database of available ensembles, collected in near-real time • Co-ordinate research using this multi-model ensemble data, including interactive aspects

6. THORPEX Regional Campaigns • ATReC (2003) - many groups are actively working with the data – a summary of current views will be available for the ICSC meeting in Melbourne • European ETreC – D-Phase (MAP), COPS supported by the European regional committee • Stronger links are being developed with AMMA for observing system experiments, modelling & predictability and societal and economic applications • Links with TWP – ICE are being investigated • Pacific Asian Regional Campaign (PARC 2008) – on tropical cyclone tracks, extra-tropical transitions, tropical warm-pool physics and down-stream propagation (link to, Beijing Olympics & IPY) • Winter Olympics (2010) • Tropical convection (2012)

7. TIGGE Implementation Meeting - Report ECMWF 9-10 November 2005

8. List of products: single level • 6_h: accumulated over previous 6 hours • acc_st: accumulated from start of forecast

9. List of products: single level fields • acc_st: accumulated from start of forecast • Orography and Land-sea mask to be provided for the Control for each output step

10. List of products: upper air fields • 5 parameters on 9 pressure levels, i.e. 45 fields. • The 9 levels are 1000, 925, 850, 700, 600, 500, 300, 250 and 200 hPa.

11. TIGGE Phase 1 – A summary • 3 Archive Centres • CMA, NCAR, ECMWF • 8 Data providers (?) • NCEP, ECMWF, UKMO, JMA, BMRC, CPTEC, KMA, MSC • Each Archive Centre will receive data from all the Data Providers • In near realtime • Users will be able to get the same data from any of the Archive Centres • No extra resources! • Use existing infrastructure

13. Definition of a core dataset • A field is uniquely identified within the TIGGE dataset by the following tuple: (base date, base time, time step, origin centre, ensemble number, level, parameter) • When using fields to create a “grand ensemble”, i.e. when considering all members from several origin centres as a super ensemble, we must make sure that they share the same values for the tuple: (base date, base time, time step, level, parameter) • Therefore, a core dataset must be defined in terms of these attributes • All Data Providers must adhere to the core dataset definition.

14. Network Bandwidth • Is a major risk • Bilateral connections need testing • Between Archive Centres • Data Provider to Archive Centres • Requires a lot of tuning/tweaking • Get vs. put • Number of parallel streams • Buffer sizes • TCP window sizes • Target aggregate bandwidth: 100GB per 12 hours • Gives a chance to resend everything on the same day

17. WMO file naming convention • Files to be exchanged must follow the WMO naming convention: Z_TIGGE_C_CCCC_yyyyMMddhhmmss_SSSS_LL_VVVV.bin[.compression] • where: • CCCC Originator • yyyyMMdd base date of the run • hhmmss base time of the run (UTC) • SSSS forecast time-step in hours (0006, …) • VVVV version (0001 for operational) • LL type of level, (sl: single level,…) • compression optional (e.g. .gz for gzip)

18. WMO file naming convention • Example: • time-step 24 • from NCEP • for the run of 1st September 2005 • 12Z • operational version, • single level parameters, • all ensemble members, including the control, • No compression Z_TIGGE_C_KWBC_20050901120000_0024_sl_0001.bin

19. File structure • Files will be exclusively GRIB messages. • No record padding. • No supplementary headers or trailers. • Non-compliant files will not be accepted by the Archive Centres. • It is acknowledged that will add development work at each centre, but: • Will make data management tractable at the Archive Centres • Will be beneficial to the users.

20. Metadata and naming conventions • Used to create of a homogenous catalogue • user data discovery • Used to specify retrieval requests. • Most of the Data Providers have their own ways to name variables, use different abbreviation and GRIB code tables. • The TIGGE partners will have to agree on a common vocabulary of terms. • Use existing names defined by WMO • See work done by the group that created the Climate and Forecast (CF) metadata convention

21. User access : Registration • The TIGGE dataset is available to the research community • A “registration authority” must be set up that should verify each registration request and establish if they are linked to a genuine research activity • Access to valid data is not permitted apart from specific field experiments. • “Valid data” is defined as: base date + base time + time step + 24 hours >= now

22. Data retrieval • ECMWF will utilise the MARS system • NCAR will build upon its Research Data Archive and Community Data Portal. • CMA is still in the development process of their data delivery system. • Over time and with additional project support, it is expected that there will be opportunities to further unify the user interface by leveraging developments from the WMO Information System (WIS) effort.

23. Data retrieval : field order • Fields will be returned to the users in the following order: (outer loop) DateTime Step Origin Ensemble member Level (for multi-level variables) Parameter (inner loop) • This allows users to process fields from different origins as if they were members of the same ensemble

24. Data retrieval • Archive Centres will prioritize and limit requests according to size. • Because users’ requests will generate high data volumes, • This will make sure that no user can monopolise the system by submitting an unreasonable request. • Very large requests may require delivery by tape media.

25. Implementation plan : GRIB2 • ECMWF will consult with NCEP and WMO in order to make sure we agree on the proper encoding of the fields in GRIB2. • ECMWF will provide a sample model output to the Data Providers. • ECMWF will provide a series of example programs to create these files. • These tools may have to be adapted by Data Providers in order to handle their own data and metadata mapping.