1 / 43

Bring i ng together CMIP5 & CORDEX Data management 8 March 2011, SMHI, Norrkoeping

Bring i ng together CMIP5 & CORDEX Data management 8 March 2011, SMHI, Norrkoeping. Topics. Data management in CORDEX : Get access to CMIP5 global forcing data role of IS-ENES ? CMIP5 formatted data not CMIP5 formatted data Provide access to CORDEX results role of IS-ENES ?

lars-valenzuela
Download Presentation

Bring i ng together CMIP5 & CORDEX Data management 8 March 2011, SMHI, Norrkoeping

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Bringing together CMIP5 & CORDEX Data management 8 March 2011, SMHI, Norrkoeping

  2. Topics Data management in CORDEX : • Get access to CMIP5 global forcing data • role of IS-ENES ? • CMIP5 formatted data • not CMIP5 formatted data • Provide access to CORDEX results • role of IS-ENES ? • come as close as possible to CMIP

  3. Provide Access Provide access to CORDEX results • proposed topics at Norrkoeping: • SMHI/NCS offer CORDEX result archiving • move CORDEX DM -> CMIP5 DM • coordination in/with IS-ENES • additional topics to come

  4. Provide Access Issues • versioning ? (OBC) • AFR grid is geographical lat/lon. Why interpolate? • What kind of quality control? • File sizes

  5. CORDEX data amounts Africa 50 km (AFR) • per yr: 3.5 GB Core/Tier1 (530 GB for 56+95 scenario yrs) 16.0 GB Tier2 ( 2.4 TB for …. ) • DMI archive: Core/Tier1 • CCLM (1000 yrs) : 3.5 TB • REMO ( 500 yrs) : 1.7 TB • other models • local archives: Tier2

  6. CORDEX data amounts Africa 50 km (AFR) • mirror of DMI archive at DKRZ (Core/Tier1): • CCLM (1000 yrs) : 3.5 TB • REMO ( 500 yrs) : 1.7 TB • other models ? Is there a list of • DKRZ as local CORDEX archive (Tier2): • CCLM (1000 yrs) : 16 TB • Remo ( 500 yrs) : 8 TB • other model ? • DKRZ as local archive (raw data): • CCLM: 310 GB/yr • REMO: 380 GB/yr

  7. CORDEX data amounts Africa 25 km (AFR-22) • per yr: 14 GB Core/Tier1 (2.1 TB for 56+95 scenario yrs) 64 GB Tier2 ( 9.7 TB for …. ) • DMI archive: Core/Tier1 • CCLM ( 266 yrs): 3.7 TB • other models • local archives: Tier2

  8. CORDEX data amounts Africa 25 km (AFR-22) • mirror of DMI archive at DKRZ (Core/Tier1): • CCLM ( 266 yrs): 3.7 TB • other models ? • DKRZ as local CORDEX archive (Tier2): • CCLM (266 yrs) : 17 TB • other model ? • DKRZ as local archive (raw data): • CCLM: 1.234 TB/yr

  9. CORDEX data amounts Europa 12 km (EUR-11) • per yr: 14 GB Core/Tier1 (2.1 TB for 56+95 scenario yrs) 64 GB Tier2 ( 9.7 TB for …. ) • DMI archive: Core/Tier1 • CCLM (763 yrs): 10.7 TB • REMO (482 yrs): 6.7 TB • other models • local archives: Tier2

  10. CORDEX data amounts Europa 12 km (EUR-11) • mirror of DMI archive at DKRZ (Core/Tier1): • CCLM (763 yrs): 10.7 TB • REMO (482 yrs): 6.7 TB • other models ? • DKRZ as local CORDEX archive (Tier2): • CCLM (763 yrs): 49 TB • REMO (482 yrs): 31 TB • other model ? • DKRZ as local archive (raw data): • CCLM: 1.543 TB/yr • REMO: 1.366 TB/yr

  11. CMIP5/CORDEX file names CMOR: tas_Amon_MPI-ESM-HR_rcp45_r2i1p1_190001-192012.nc CMIP5: tas_Amon_MPI-ESM-HR_rcp45_r2i1p1_190001-192012.nc CORDEX:tas_AFR_MPI-ESM-LR_rcp45_r1i1p1_V-C5_REMO_V-RCM_mon_190001-102012.nc

  12. CMIP5/CMOR directory structure

  13. CMIP5 ESGF data node directory structure

  14. CMIP5/CORDEX directory structure CMIP5/ESGF: /CMIP5/output[1,2]/MPI-M/MPI-ESM-LR/ rcp45/mon/atmos/Amon/r2i1p1/v20110613/tas/...nc CMIP5 data set ID(publication level): CMIP5.output.MPI-M.MPI-ESM-LR.rcp45.mon.atmos.Amon.r2i1p1 CORDEX: /AFR/MPI-M/MPI-ESM-LR/rcp45/r2i1p1/REMO/V-RCM/mon/tas/...nc

  15. CORDEX Data and ESGF • no 2nd ESGF server at DKRZ: • CORDEX needs to have the same directory structure as CMIP5 • publication level can not be eliminated other solution at DKRZ: • via FTP • searchable in Cera UI if metadata provided

  16. Access to CMIP5 RCM Forcing Issues / Concerns: • CMIP5 format (time series of single variables) for RCM forcing data • CMIP5 requested (ta,ua,va,psl (6hrly)) not all needed variables • SST, … on ocean grid • SST, … not 6hrly • large volume (6hrly, model levels) with NetCDF(3)

  17. Data Volumes [MB] 1 year MPI-ESM-LR(T63L47) requested by CMIP5: • .grb.gz: 153 + 189 (spectral ta, div., vort., psl; hus) • .grb: 166 + 207 ( “ ) • .grb: 626 + 207 (grid point ta, ua, va, psl, hus) • .nc: 1238 + 409 ( “ ) • .nc.gz: 822 + 248 ( “ ) • 342 versus 1070 MB

  18. MPI-ESM CMIP5 RCM Forcing Data • optimize data transfer by providing • spectral GRIB gnu-zipped data • script to convert into gridded single-variable time series in CMIP5 NetCDF format • conversion wall clock times • 5:03 min (IBM AIX power6) • 4:44 min (Linux cluster) • 5:26 min (PC 2.4 GHz) • optional as CMIP5 NetCDF per USB • additional files with additional parameters • (all on atmosphere grid, 6 hrly)

  19. MPI-ESM CMIP5 RCM Forcing Data Access from DKRZ • CMIP5 RCM forcing do not belong to the replicated data class • non-CMIP5 formatted data: • create DKRZ data project • give accounts for • ftp access (password) incl. some metadata • to CMIP5 requested and additional parameters • CMIP5 formatted data: • via ESGF? • requested parameters only

  20. Other than MPI-ESM RCM Forcing Data Access from DKRZ through IS-ENES portal • funding/position needed

  21. The End

  22. Topics • What is CMIP5 ? • What is CORDEX ? • How will the MPI-M contribute? • The DKRZ project 555 & other funding & related activities • The CMIP5 Archive : Data and Metadata aspects • A critical assessment

  23. Topics • What is CMIP5 ? • What is CORDEX ? • How will the MPI-M contribute? • The DKRZ project 555 & other funding related activities • The CMIP5 Archive : Data and Metadata apects • A critical assessment

  24. What is COordinatedRegionalDownscalingEXperiments ? WCRP sponsored international coordinated framework for regional climate change projections • downscale a common set of CMIP5experiments • priority: 1950-2005 historical 2006-2100 RCP4.5/8.5 multiple GCMs • next: downscaleCMIP5decadal hind/forecast • common domains, resolution, grids (but different RCMs) • evaluate with ERA-interim 20 years • initial 50km for involvement of wider community, initial focus on Africa • model output protocol • WG1 (Physical Basis): Africa simulation (end 2010) • WG2 (Impact, Adaptation, Vulnerability): other experiments (~ +1 year)

  25. CORDEX common domains

  26. Topics • What is CMIP5 ? • What is CORDEX ? • How will the MPI-M contribute? • The DKRZ project 555 & other funding related activities • The CMIP5 Archive : Data and Metadata apects • A critical assessment

  27. How will the MPI-M contribute? • within the DKRZ 555 project • additional experiments on its own ressources • providing its models to other groups for additional CMIP5 / CORDEX experiments

  28. The DKRZ project 555 Titel: CMIP5 Klimasimulationen und deren Regionalisierung Models: ECHAM6/MPIOM, REMO, CCLM Projects: CMIP5, CORDEX Coordination: Stephanie Legutke (M&D)Contact ECHAM6/MPIOM: Marco Giorgetta (MPI-M) Contact REMO: Daniela Jacob (MPI-M) Contact CCLM:Klaus Keuler (BTU)

  29. Compound project 555 Why ? • ensure promptly downscaling of CMIP5 projections • cover multiple aspects of uncertainly: • using different RCMs (REMO, CCLM) • doing different realisations (ECHAM6:REMO) • doing different scenarios (RCP4.5, RCP8.5) • using different ESM CMIP5 forcing (CCLM)

  30. CMIP5 experiments with ECHAM6/MPIOM • 2587 CMIP5 experiment years with MPI-M comp. ressources • on project 555 account: 2207 (Core) + 2157 (others) • Jülich RC (Martin Schulz): • historical + interakt. aerosole + AtmChemie • RCP4.5 (T63L47/TP10L40) • COMBINEpromotes further CMIP5 experiments after 2010 WG1 deadline see proposal with complete experiment list athttp://wiki.zmaw.de/mad/CMIP5

  31. CORDEX domains used in 555 • Auflösung 0.11o • NX/NY/NZ= 440/428/40 • rotierte KoordinatenPol (-162.0, 39.25) Auflösung 0.44o NX/NY/NZ= 214/221/32 rotierte KoordinatenPol (180., 90.) contribution to WGII contribution to WGI 31

  32. Geplante CORDEX Experimente: REMO

  33. CORDEX Simulationen mit COSMO-CLM

  34. Topics • What is CMIP5 ? • What is CORDEX ? • How will the MPI-M contribute? • The DKRZ project 555 & other funding & related activities • The CMIP5 Archive : Data and Metadata apects • A critical assessment

  35. The DKRZ project 555 • 53-54 computing nodes on blizzard in 2010 together with the millennium project (ca. 22% of blizzard) • 2300 TB tape archive • the project will continue in 2011 • ECHAM6/MPIOM with T127L95/TP04L80 grid (reduced from initially planned T156) • running (technically) since yesterday • paleo runs with T63L47/TP10L80

  36. Other funding and projects • Data basis für CSC • Data requests already arriving (e.g. ClimateforCulture, KLIFF, KLIMZUG, EU projects) • BMBF Projects • “Durchführung der CMIP5 Konsortialrechnungen” • 4 positions (3 years) • “Einrichtung und Betrieb des CMIP5 Datenknotens” : • 2 positions (3 / 5 years) • 1,5 Mio. € material funding

  37. Topics • What is CMIP5 ? • What is CORDEX ? • How will the MPI-M contribute? • The DKRZ project 555 & other funding related activities • The CMIP5 Archive: Data and Metadata aspects • A critical assessment

  38. EU FP7 CMIP5/AR5 Data Management Concept Gateway BADC / IPCC DDC Gateway PCMDI Gateway WDCC / IS-ENES Data Nodes DKRZ Hadley Centre / BADC IPSL NCAR GFDL Other Centres BADC: 0.75 PByte CMIP5/AR5 Data Federation WDCC: 0.75 PByte 1.5 PB backup PCMDI: 0.75 PByte • CMIP5 Data Protocol: • Requested Variables • Directory/Filenames/Metadata Mirrors: Australia Japan Others ?

  39. EU FP7 BADC: 0.75 PByte WDCC: 0.75 PByte 1.5 PB backup PCMDI: 0.75 PByte CMIP5/AR5 Data Management Concept Data Catalog & CIM Data Publication (DOI) & Quality Control Data Nodes ESM results: raw data & CMOR2 reformatted & Add. RCM forcing ? (Meta)Data CMIP5Compliancy i.e. Entry Control IPCC AR5 Reference Data Archive: snapshot end of 2010?

  40. Meta Data for CMIP5 Archive

  41. MetaData harvesting during experiment setup Compilation ConfigurationofExperimente Run Experiment experiment.xml Data Node ... historical_r1.xml historical_r2.xml PIcontrol_r1.xml ... CMIP5GUI CMIP5 data node translate experiment.xml -> cmip5.xml

  42. www publish cmip5-x.html translate complete cmip5_*.xml METAFOR CMIP5 GUI translate cmip5_*.xml MD extraction in IMDI User‘snotebook list / diff grep cmip5*.xml ...cmip5-exp.xml ...

  43. What is CMIP5 Archive and Meta Data ?

More Related