210 likes | 419 Views
Ensemble Handling in GrADS. Jennifer M. Adams Brian Doty IGES/COLA. What is GrADS?. GrADS is an interactive tool that integrates data access, analysis, and visualization Handles many data formats: binary, NetCDF, HDF, GRIB1&2, BUFR Two data models for gridded and in situ data
E N D
Ensemble Handlingin GrADS Jennifer M. Adams Brian DotyIGES/COLA
What is GrADS? • GrADS is an interactive tool that integrates data access, analysis, and visualization • Handles many data formats: binary, NetCDF, HDF, GRIB1&2, BUFR • Two data models for gridded and in situ data • Expression handling is flexible, compact, recursive • Programmable interface for scripting • Written in C; code is open source (GPL)
What is the GrADS Data Server? • GDS is a stable, secure, OPeNDAP data server that provides subsetting and server-side analysis services over the internet • GDS can serve any GrADS-readable dataset, and unifies all data formats into a NetCDF framework • Open a data set with http://servername/filenameinstead of /disk/filename
News from GrADS/GDS Team • GrADS has a 5th grid dimension for ensembles • GrADS has a GRIB2 interface • GDS can serve any GrADS data set • GrADS is a client for allOPeNDAP data sets • GrADS will support GIS-compatible outputs
The New Ensemble Dimension in GrADS • A 5th grid dimension for ensemble members‘set X, Y, Z, T, or E’ or‘set lon, lat, lev, time, or ens’ • A virtual dimension for forecast time offset‘display temp(ft=2)’‘display temp(ftime=24hr)’
GrADS Metadata Requirements for Ensemble Members • Unique name / number • Initial time • Length • If GRIB2, some additional octet values • One time axis spans all members • All members must have common X, Y, Z axes
GrADS GRIB2 Descriptor File Wesley’s g2ctl works very well, but doesn’t handle EDEF (yet). DSET /gens/prod/gefs.%iy4%im2%id2/%ih2/pgrb2a/ge%e.t%ih2z.pgrb2af%f2 TDEF 17 linear 00z09oct2008 6hr EDEF 23 avg 17 00z09oct2008 0spr 17 00z09oct2008 2 c00 17 00z09oct2008 1,0 p01 17 00z09oct2008 3,1 p02 17 00z09oct2008 3,2 p03 17 00z09oct2008 3,3 p04 17 00z09oct2008 3,4 . . . p19 17 00z09oct2008 3,19 p20 17 00z09oct2008 3,20 ENDEDEF @ ens String avg Unweighted mean of all members@ ens String spr Standard deviation with respect to ensemble mean @ ens String c00 Control forecast@ ens String p01 Positively perturbed forecast The GRIB2 codes are octets 35 and 36 from Section 4 (PDT # 1, 2, 11, and 12)
Examples of Ensemble Data Sets • NCEP GFS Ensembles (GENS) • NCEP Climate Forecast System (CFS) • NCEP Short Range Ensemble Forecasts (SREF) • ESRL MRF Reforecasting Experiment • WCRP CMIP3 Multi-Model Data (IPCC AR4) • TIGGE
Ensemble Data Sets Behind GDS • Data become more usable and accessible • Subsets over all dimensions • Server-side analysis • File aggregation • Format translation • Ensemble metadata standards:
Ensemble Forecast Time Series(Longitude, Latitude, and Level are fixed) Forecast Time --->
Ensemble Forecast Grid(Longitude, Latitude, and Level are fixed) Ensemble Member Forecast Time --->
Ten Ensemble Forecasts(Longitude, Latitude, and Level are fixed) Ensemble Member Forecast Time --->
CFS Daily Hindcast (Longitude, Latitude, and Level are fixed) Ensemble Member Time Axis ---->
Ensemble Forecast Time Series(Longitude, Latitude, and Level are fixed) Forecast Time --->
Ensemble Mean = tloop(ave(Z,e=2,e=23))Ensemble Min/Max = tloop(min(Z,ens=c00,ens=p20))+/- StdDev of Ensemble Mean = tloop(sqrt(ave(pow(Z-Zave,2),e=1,e=21))) Forecast Time --->
TIGGE Data Behind GDS at NCAR • Perfect testbed for ensemble handling and GRIB2 interface • Boost to usage of TIGGE data • Forecasts sorted by date and by provider • Time series of analyses • Nearly unbearable load on old hardware • 48-hour data embargo • Int’l agreement requires password protection
TIGGE Multi-Member Multi-Model Ensemble500mb Geopotential Height valid August 30, 2008 7-day Lead 5-day Lead 3-day Lead 1-day Lead
TIGGE MME Forecast Error andEnsemble Spread 500mb Geopotential Height valid August 30, 2008 7-day Lead 5-day Lead 3-day Lead 1-day Lead
init: 00z 8 Sep TIGGE Forecasts of Hurricane Ike valid: 12z 9 Sep - 00z 13 Sep init: 12z 8 Sep init: 00z 9 Sep init: 12z 9 Sep