1 / 42

GRIB2 usage at NCEP

GRIB2 usage at NCEP. Dr. V. KRISHNA KUMAR & BOI VUONG NOAA/NWS/NCEP/Central Operations. NCEP WWB GRIB2 Seminar 15 June 2011. Room 707, WWB Camp Springs, MD. Overview. GRIB2 Sections – NCEP GRIB2 Documentation

peers
Download Presentation

GRIB2 usage at NCEP

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. GRIB2 usage at NCEP Dr. V. KRISHNA KUMAR & BOI VUONG NOAA/NWS/NCEP/Central Operations NCEP WWB GRIB2 Seminar 15 June 2011 Room 707, WWB Camp Springs, MD

  2. Overview • GRIB2 Sections – NCEP • GRIB2 Documentation • GRIB2 in NCEP Operations – Encoding & Decoding routines, NCEP Utilities (cnvgrib, degrib2, overdateg2, overmodelg2, copygb2,wgrib2……..) • GRIB2 – Section 7 Data Section • GRIB2 – Packing Options • How to write GRIB2 data directly? – Some specific examples

  3. Overview • GRIB2 Encoding routines • GRIB2 Decoding routines • Declaration of Derived type “gribfield” • How to use routine “getgb2”? • How to use routine “putgb2”? • GRIB2 read and write examples • Summary

  4. GRIB1 Sections 0 – Indicator Section - ‘GRIB’ (octet 1-4 = 32bits), length of message (octet 5-7=24bits), edition number -1 (octet 8=8bits) - 1 octet = 1byte = 8bits - octets are numbered 1, 2, 3, etc., Bit positions within octets are from bit 1 to bit 8. - bit 1 = most significant bit & bit 8 = least significant) 1 – Product Definition Section (PDS) - Length of PDS, parameter table version number, ID of center, generating process ID, grid identification (Table B), Ref time, parameter and units, level, valid time … 2 – Grid Definition Section (GDS) – optional - Length of sec., sec. no, defn of grid surf and geometry of data values within the surface 3 – Bit-Map Section (BMS) – optional - Length of sec., sec. no, indication of presence or absence of data at each of the grid points, as applicable example sst 4 – Binary Data Section (BDS) - Length of sec., sec. no, data values 5 – End Section = ‘7777’

  5. GRIB2 Sections • 0 – Indicator Section - ‘GRIB’, length, version, discipline – (16 octets long) • Section 1 – Identification - Ref time, orig center, table versions, …(length of the section is expressed as the first four octets (32 bits) -the section number in the fifth octet and octets beyond 21 are reserved for future use) • 2 – Local Use Section – optional================== • 3 – Grid Definition Section======= | • 4 – Product Definition Section====| | - parameter, level, valid time | | | • 5 – Data Representation Section|| | - packing algorithm used, info needed to unpack | | | • 6 – Bit Map Section| | | • 7 – Data Section================== -packed data • 8 – End Section = ‘7777’

  6. GRIB2 – Documentation • NCEP GRIB2 Documentation (internet - www) • http://www.nco.ncep.noaa.gov/pmb/docs/grib2/ • NCEP GRIB2 Resources (Detailed Documentation) http://www.nco.ncep.noaa.gov/sib/grib2/ • GRIB2 specification from WMO contains template definitions and table entries: http://www.wmo.ch/web/www/DPS/grib-2.html • Local Tables: • http://www.nco.ncep.noaa.gov/pmb/docs/grib2 • NCEP source files and utilities on the web for decoding and encoding data in GRIB2 format • http://www.nco.ncep.noaa.gov/pmb/codes/GRIB2/ • NCEP wgrib2 resources • http://www.cpc.ncep.noaa.gov/products/wesley/wgrib2/ • See Wesley’s PowerPoint presentation 4/6/2011 at NCEP under Documentation • for all the pertinent details on wgrib2

  7. GRIB2 Significant Features • One GRIB2 message can contain multiple fields. - Sequences of GRIB sections 2 to 7, sections 3 to 7 or sections 4 to 7 may be repeated within a single GRIB message. - Once a section is repeated all the following sections must appear in order. - A section remains in effect until redefined. - Could be used for vector fields such as u & v components, or ensemble members - IEEE single precision floating point representation used instead of standard IBM representation. The representation occupies four octets. • GDS, PDS, DRS, and DS are all template driven. Octet assignments within a section are based on the Template specified. • Parameters are defined by three values: Discipline (specified in Section 0) Category (PDS) Parameter number (PDS).

  8. GRIB2 Significant Features contd. • All longitudes MUST be in degrees East and be between 0 and 360 degrees. No negatives. No degrees West. No change in latitude definitions –900 to 900. • GDS and BMS no longer optional. They must be included. Only Section 2 (Local Use) is optional. • Latitudes, Longitudes are in units of micro-degrees (10 –6 degrees). Includes distance between grid points for Lat/Lon grids (GRIB1 was milli-degrees) • All other grid point increments are in millimeters (GRIB1 was in meters) • Floating point values are now stored in IEEE format (GRIB1 was IBM 360/195 format HDS)

  9. SECTION 0:  0 2 60463

  10. SECTION 1:  7 0 2 1 1 2010 11 9 6 0 0 0 1  Contains  0  Local Sections  and  1  data fields.

  11. SECTION 3:  0 65160 0 0 0  GRID TEMPLATE 3. 0 :  6 0 0 0 0 0 0 360 181 0 0 90000000 0 48 -90000000 359000000 1000000 1000000 0

  12. PRODUCT TEMPLATE 4. 0 :  3 5 2 0 96 0 0 1 36 100 0 100000 255 0 0  TEXT: HGT      1000 mb valid at  36 hr after 2010110906:00:00

  13. DRS TEMPLATE 5. 3 :  -979087360 0 1 11 0 1 0 0 0 5124 0 4 1 1 32 5 1 2

  14. GRIB2 in NCEP operations Encoding Routines GRIB2 message can contain gridded fields for many parameters on a number of different grids. Start a new GRIB2 message with a call to: (1) call GRIBCREATE – encodes sections 0 and 1 at the beginning of message. (2) call ADDLOCAL – adds a Local Use Section (Section 2) optional (3) call ADDGRID – encodes a grid definition into Section 3. Defines the geometry of the data values in the fields that follow it. ADDGRID can be called again to change the grid definition describing subsequent data fields. (4) call ADDFIELD – adds each data field which adds Sections 4, 5, 6, and 7 to the message. (5) call GRIBEND – adds the final section 8 to the message which updates the length of the message. GRIBEND is required for each GRIB2 message. Boi will demonstrate these routines with examples in the latter half of this seminar

  15. GRIB2 in NCEP operations contd. Decoding Routines call GB_INFO – finds out how many Local Use sections and data fields are contained in a given GRIB2 message. Also returns the number of octets of the largest Local Use section in the message. Ensures the dimensionality of the input array of GETLOCAL call GETLOCAL – returns the requested occurrence of Section 2 from a given GRIB2 message. call GF_GETFLD – gets all information pertaining to the nth data field in the message – returns all the unpacked values for each Section and Template in a f90 “type gribfield” with options to unpack the Bit-map (if applicable) and the data values or just return the field description information. call GF_FREE(gfld) – frees up memory held by gfld as many elements in derived type gribfield are dynamically allocated when decoded. Example: use grib_mod type(gribfield) ::gfld … … call gf_getfld(cgrib,lengrib,1,.true.,.false.,gfld,ierr) print*,’Num of grid points =‘,gfld%ngrdpts print*,’First data value=‘,gfld%fld(1) call gf_free(gfld) How to extract GRIB2 fields from a GRIB2 file? GETGB2 – extracts a specified field from a file containing many GRIB2 messages. Boi will demonstrate these with examples in the latter half of this seminar.

  16. GRIB2 in NCEP operations contd. Create GRIB2 Message call PUTGB2 – if a variable of derived type of “gribfield” already exists, this subroutine can be used to pack it up and write it to a file – Boi will demonstrate this NCEP GRIB2 Utilities /grb2index grib2file grib2indexfile - creates a GRIB2 index file /cnvgrib –h (packing options: 1 => 2, 2=>1, 2=>2) -converts all fields in a file between GRIB1 and GRIB2 /degrib2 grib2file -dumps the meta-data for each GRIB2 field in grib2file - basically prints contents of each element in derived type gribfield /wgrib2 grib2file Grads for GRIB2 sh convert_grib2_to_grads.sh #!/bin/sh set -xa # file_list=`ls grib2.t00z.wafs_grbf06.45 gfs.t00z.wafs_grb45f06.grib2` for list in $file_list do # /usrx/local/grads/bin/g2ctl -verf $list > ${list}.ctl /usrx/local/grads/bin/g2ctl $list > ${list}.ctl /usrx/local/grads/bin/2.0.a3/xgribmap -i ${list}.ctl done exit

  17. $ /nwprod/util/exec/cnvgrib -h Usage: cnvgrib [-h] {-g12|-g21|-g22} [-m] [{-p0|-p2|-p31|-p32|-p40|-p41|-p40000|-p40010}] ingribfile outgribfile cnvgrib: version cnvgrib-1.1.1 Must use one of the following options: -g12 converts GRIB1 to GRIB2 -g21 converts GRIB2 to GRIB1 -g22 converts GRIB2 to GRIB2 (used to change packing option) Optional packing options: (for use with -g12 and -g22 only) -p0 simple packing -p2 complex packing -p31 complex pack with 1st order diffs -p32 complex pack with 2nd order diffs -p40 JPEG2000 encoding -p41 PNG encoding Other Optional options: -m Use missing values instead of bitmap (valid with -p2, -p31 or -p32 options only) GRIB2 in NCEP Operations

  18. GRIB2 in NCEP Operations /nwprod/util/sorc/overmodelg2.fd $ more sample.script # THIS SCRIPT READS A FORECAST FILE (UNIT 11), REPLACES MODEL INDENTIFIER IN # PRODUCT TEMPLATE 4.x FROM A ID FILE UNIT 51. # STANDARD INPUT IS A ID (2-DIGIT INTEGER), FOR EXAMPLE: 96 (GFS Model) set -x export XLFRTEOPTS="unit_vars=yes" # Allow overriding default names. export XLFUNIT_11=gfs.t00z.pgrb2f12.2p5deg export XLFUNIT_51=gfs.t00z.pgrb2f12.2p5deg.new overmodelg2 << EOF 98 EOF /nwprod/util/sorc/overdateg2.fd # THIS SCRIPT READS A FORECAST FILE (UNIT 11), MODIFIES REFERENCE TIME YEAR MONTH # DATE AND HOUR IN SECTION 1 AND IN PRODUCT TEMPLATE 4.x TO REPLACE INTERNAL # REFERENCE TIME FROM A DATE IN FILE UNIT 51. # STANDARD INPUT IS A YYYYMMDDHH (10-DIGIT INTEGER), FOR EXAMPLE 2010090112 set -x export XLFRTEOPTS="unit_vars=yes" # Allow overriding default names. export XLFUNIT_11=gribfile export XLFUNIT_31=gribfile_indx export XLFUNIT_51=gribfile.new overdateg2 << EOF 2010090112 EOF

  19. GRIB2 in NCEP Operations /nwprod/util/sorc/copygb2.fd NAME copygb2 - copies GRIB2 files SYNOPSIS copygb2 [-g "kgdtn [kgdt]"] [-i "ip [ipopts]"] [-k "kpdtn kpdt"] [-v "uparms"] [-X] [-B mapgrib [-b mapindex] [-A "<> mapthreshold"] [-K "mapkpds"]] [-M "mask"|mergegrib [-m mergeindex]] [-a] then either: grib2in index2in grib2out or: -x grib2in grib2out DESCRIPTION The command copygb2 copies all or part of one GRIB2 file to another GRIB2 file, interpolating if necessary. Unless otherwise directed (-x option), the GRIB2 index file is also used to speed the reading. The fields are interpolated to an output grid if specified (-g option). The interpolation type defaults to bilinear but may be specified directly (-i option). The copying may be limited to specific fields (-k option). It may also be limited to a specified subgrid of the output grid or to a subrange of the input fields (-B and -b, -A, and -K options). Fields can be identified as scalars or vectors (-v option), which are interpolated differently. The invalid data in the output field can be filled with mask values or merged with a merge field (-M and -m options). The output GRIB2 message can also be appended to a file (-a option). If grib2out is specified as '-', then the output GRIB file is written to standard output.

  20. GRIB2 in NCEP Operations # # Script to convert a RTMA 2p5 grib2 file to a 5p0 grib2 file # # Author: V. Krishna Kumar October 5, 2010 NCO/Systems Integration Branch # set -x # # clean the output file prior to running this script # # I created a single field grib2 file from the original production grib2 file using # wgrib2 -d 1 in_grb2file -grib out_grb2file # rm rtma5p0.t12z.2dvaranl_ndfd.1field.grb2 # # HGT - Conversion from 2p5 to 5 # Input file : rtma2p5.t12z.2dvaranl_ndfd.1field.grb2 # Output file : rtma5p0.t12z.2dvaranl_ndfd.1field.grb2 # # 2p5 Grid template 3.30 : 6 0 0 0 0 0 0 2145 1377 20191999 238445999 8 25000000 265000000 2539703 2539703 0 64 25000000 25000000 -90000000 0 # 5p0 Grid template 3.30 : 6 0 0 0 0 0 0 1073 689 20191999 238445999 8 25000000 265000000 5079406 5079406 0 64 25000000 25000000 -90000000 0 export grid_new="30 6 0 0 0 0 0 0 1073 689 20191999 238445999 8 25000000 265000000 5079406 5079406 0 64 25000000 25000000 -90000000 0 0" export grib_in=rtma2p5.t12z.2dvaranl_ndfd.1field.grb2 # # Create a grib2 index file of the input grib file # /nwprod/util/exec/grb2index $grib_in ${grib_in}_ix export grib_index=${grib_in}_ix /nwprod/util/exec/copygb2 -g "${grid_new}" -i0 ${grib_in} ${grib_index} rtma5p0.t12z.2dvaranl_ndfd.1field.grb2 echo "Max Min value from the original 2p5 grid " /nwprod/util/exec/wgrib2 -grid -max -min rtma2p5.t12z.2dvaranl_ndfd.1field.grb2 echo "Max Min value from the converted 5p0 grid " /nwprod/util/exec/wgrib2 -grid -max -min rtma5p0.t12z.2dvaranl_ndfd.1field.grb2 echo " Max Min from the current operational 5 degree grid " /nwprod/util/exec/wgrib2 -grid -max -min rtma.t12z.2dvaranl_ndfd.1field.grb2 exit

  21. GRIB MESSAGE  1  starts at 1  SECTION 0:  0 2 60463  SECTION 1:  7 0 2 1 1 2010 11 9 6 0 0 0 1  Contains  0  Local Sections  and  1  data fields.FIELD  1  SECTION 0:  0 2  SECTION 1:  7 0 2 1 1 2010 11 9 6 0 0 0 1  SECTION 3:  0 65160 0 0 0  GRID TEMPLATE 3. 0 :  6 0 0 0 0 0 0 360 181 0 0 90000000 0 48 -90000000 359000000 1000000 1000000 0  NO Optional List Defining Number of Data Points.  PRODUCT TEMPLATE 4. 0 :  3 5 2 0 96 0 0 1 36 100 0 100000 255 0 0  TEXT: HGT      1000 mb valid at  36 hr after 2010110906:00:00  NO Optional Vertical Coordinate List.  Num. of Data Points =  65160     NO BIT-MAP  DRS TEMPLATE 5. 3 :  -979087360 0 1 11 0 1 0 0 0 5124 0 4 1 1 32 5 1 2 Data Values:MIN=        -525.79998779 AVE=          70.36888123 MAX=         313.10000610 GRIB MESSAGE  2  starts at 60464  SECTION 0:  0 2 59574  SECTION 1:  7 0 2 1 1 2010 11 9 6 0 0 0 1  Contains  0  Local Sections  and  1  data fields.FIELD  1  SECTION 0:  0 2  SECTION 1:  7 0 2 1 1 2010 11 9 6 0 0 0 1  SECTION 3:  0 65160 0 0 0  GRID TEMPLATE 3. 0 :  6 0 0 0 0 0 0 360 181 0 0 90000000 0 48 -90000000 359000000 1000000 1000000 0  NO Optional List Defining Number of Data Points.  PRODUCT TEMPLATE 4. 0 :  3 5 2 0 96 0 0 1 36 100 0 97500 255 0 0  TEXT: HGT      975 mb valid at  36 hr after 2010110906:00:00  NO Optional Vertical Coordinate List.  Num. of Data Points =  65160     NO BIT-MAP  DRS TEMPLATE 5. 3 :  -984977408 0 1 11 0 1 0 0 0 5130 0 4 1 1 32 5 1 2 Data Values:MIN=        -323.89999390 AVE=         278.48535156 MAX=         514.00000000

  22. GRIB MESSAGE  149  starts at 7050617  SECTION 0:  0 2 93572  SECTION 1:  7 0 2 1 1 2010 11 9 6 0 0 0 1  Contains  0  Local Sections  and  2  data fields.FIELD  1  SECTION 0:  0 2  SECTION 1:  7 0 2 1 1 2010 11 9 6 0 0 0 1  SECTION 3:  0 65160 0 0 0  GRID TEMPLATE 3. 0 :  6 0 0 0 0 0 0 360 181 0 0 90000000 0 48 -90000000 359000000 1000000 1000000 0  NO Optional List Defining Number of Data Points.  PRODUCT TEMPLATE 4. 0 :  2 2 2 0 96 0 0 1 36 100 0 100000 255 0 0TEXT: U GRD    1000 mb valid at  36 hr after 2010110906:00:00  NO Optional Vertical Coordinate List.  Num. of Data Points =  65160     NO BIT-MAP  DRS TEMPLATE 5. 3 :  -1014333440 0 1 9 0 1 0 0 0 4835 0 4 1 1 32 5 1 2 Data Values:MIN=         -27.70000076 AVE=            .24823742 MAX=          27.20000076 FIELD  2  SECTION 3:  0 65160 0 0 0  GRID TEMPLATE 3. 0 :  6 0 0 0 0 0 0 360 181 0 0 90000000 0 48 -90000000 359000000 1000000 1000000 0  NO Optional List Defining Number of Data Points.  PRODUCT TEMPLATE 4. 0 :  2 3 2 0 96 0 0 1 36 100 0 100000 255 0 0TEXT: V GRD    1000 mb valid at  36 hr after 2010110906:00:00  NO Optional Vertical Coordinate List.  Num. of Data Points =  65160     NO BIT-MAP  DRS TEMPLATE 5. 3 :  -1015873536 0 1 9 0 1 0 0 0 4934 0 4 1 1 16 5 1 2 Data Values:MIN=         -24.30000114 AVE=            .03596115 MAX=          30.39999962 GRIB MESSAGE  150  starts at 7144189

  23. GRIB2 Section 7 – Data Section • Data coded with the minimum number of bits necessary to provide the accuracy. • Simple scaling – data multiplied by an appropriate power of 10 before forming the non-negative differences, and then using the binary scaling to select the precision of the transmitted value. • Data coded in the form of non-negative scaled differences from a reference value of the whole field plus, if applicable, a local reference value. (reference value is normally the minimum value of the data set which is represented).

  24. GRIB2 Section 7 – Data Section contd. • Complex packing for grid-point values – split the whole set of scaled data values into groups on which local references (local minima) are removed – pre-processing may be needed on the scaled data. • Complex packing for spectral data. • The original data value Y can be recovered with the formula • Y*10D=R+(X1+X2)*2E • where E=Binary scale factor, D=Decimal scale factor, R=Reference value of the whole field, X1=0, X2=Scaled (encoded) value.

  25. GRIB2 – Packing Methods • GRIB2_com – Group packing – DR Template 5.2 • GRIB2_1od – Group packing –with 1st order differencing Template 5.3 • GRIB2_2od – Group packing – with 2nd order differencing Template 5.3 • Image compression algorithms within the GRIB2 standard. • Both treat grid point data as a grey scale image (single component) • JPEG_2000 (Part-1 standard) (ISO/IEC 15444-1) • http://www.jpeg.org/JPEG2000.html • PNG (Portable Network Graphics image compression format) • http://www.libpng.org/pub/png/.

  26. How to write GRIB2 directly? An example to convert APCP data into GRIB2 The degrib2 diagnostic was generated from APCP GRIB2 output and the source code is in: $ hostname c2n6.ncep.noaa.gov $ pwd /nco/save/sib/wx12kk/degrib2_analysis/sorc Input binary data file: XMRG2417 run script: sh runGRIB2.sh output: GRIB2417_grib2 Usage: HEtoGrib2_exec <input XMRG DIR> <XMRG file name> <output GRIB DIR> <output GRIB2 file name> For full details, see the source code: HEtoGrib2.f90 call write_nesdis_grib2(gnesdis,nx,ny,nflds,& varname,typeproc,isign, & rtime(1),rtime(2),rtime(3),rtime(4),rtime(5),rtime(6), & ityped,g2Path, accumHour, modelId, subCenter) An example of converting an ASCII data to GRIB2 format from scratch http://www.nco.ncep.noaa.gov/sib/grib2/examples/drap2g2/

  27. GRIB2 Encoding Routines The following routines are needed to write a GRIB2 message: Gribcreate (gribcreate.f) This routine must be called first to initialize a new grib2 message. It uses to encode and pack section 0 and section 1 and it encodes at the beginning of GRIB message (first 8-bytes) with “GRIB - - 0 2’, bytes (5 and 6) reserved for future use, byte 7 is for discipline, byte 8 is for GRIB Edition 2 and leaves space for length of GRIB in bytes 9 to 16 call gribcreate(cgrib,maxpts,listsec0,listsec1,ierr) For example: GRIB2 section 0 and section 1 -GRIB2 section 0 - Indicator Section listsec0(1)=0 Discipline (see Code Table 0.0) ) is meteorological products listsec0(2)=2 GRIB Edition Number (currently 2) -GRIB2 section 1 - Identification Section listsec1(1)=7 NWS - id of originating centre (Common Code Table C-1) listsec1(2)=4 EMC - Id of originating sub-centre (local table)/Table C of ON388 listsec1(3)=2 GRIB Master Tables Version Number (Code Table 1.0) listsec1(4)=1 GRIB Local Tables Version Number (Code Table 1.1) listsec1(5)=1 1 is start of forecast - Significance of Reference Time (Code Table 1.2) listsec1(6)=2011 Reference Time - Year (4 digits) listsec1(7)=6 Reference Time – Month listsec1(8)=15 Reference Time – Day listsec1(9)=14 Reference Time – Hour listsec1(10)=30 Reference Time – Minute listsec1(11)=0 Reference Time – Second listsec1(12)=0 0 is operation products - Production status of data (Code Table 1.3) listsec1(13)=1 1 is for forecast products -Type of processed data (Code Table 1.4) -GRIB2 section 2 - Local Use Section (Optional)

  28. GRIB2 Encoding Routines (continued) Addlocal (addlocal.f) Local use section. It is optional is used to add a Local Use Section 2 to GRIB2 message call addlocal (cgrib,lcgrib,csec2,lcsec2,ierr) Addgrid (addgrid.f) is used to add a Grid Definition Section (grid geometry of the data values in the field) to a GRIB2 message. call addgrid (cgrib, lcgrib, igds, igdstmpl, igdstmplen, ideflist, idefnum, ierr) For example: Global Lat/Lon (1.25 degree) 41760 data points (288x145) GRIB2 section 3 - Grid Definition Section 3.0 igds(1)=0 0 is source of grid definition (see Code Table 3.0) igds(2)=288*145 Number of grid points in the defined grid igds(3)=0 Number of octets needed for each additional grid points definition igds(4)=0 Interpolation of list for optional points definition (Code Table 3.11) igds(5)=0 Lat/Lon grid - Grid Definition Template Number 0 (Code Table 3.1) igdstmpl(1) = 6 Earth assumed spherical with radius = 6,371,229.0m igdstmpl(2) = 0 Scale factor of radius igdstmpl(3) = 0 Scale value of radius igdstmpl(4) = 0 Scale factor of major axis igdstmpl(5) = 0 Scale value of major axis igdstmpl(6) = 0 Scale factor of minor axis igdstmpl(7) = 0 Scale value of minor axis igdstmpl(8) = 288 Number of points along x-axis igdstmpl(9) = 145 Number of points along y-axis igdstmpl(10) = 0 Basic angle of the initial production domain igdstmpl(11) = 0 Subdivision of basic angle igdstmpl(12) = 90000000 La1-Latitude of first grid point igdstmpl(13) = 0 Lo1-Longitude of first grid point igdstmpl(14) = 48 Resolution and component flags igdstmpl(15) = -90000000 La2-Latitude of last grid point igdstmpl(16) = 358750000 Lo2-Longitude of last grid point igdstmpl(17) = 1250000 x direction increment igdstmpl(18) = 1250000 y direction increment igdstmpl(19) = 0 Scanning mode ideflist=0 Used if igds(3) .ne. 0. Dummy array otherwise idefnum=1 Used if igds(3) .ne. 0. Dummy array otherwise

  29. GRIB2 Encoding Routines (continued) • Addfield (addfield.f) This routine uses to add and packs up: Production Definition Section (section 4), Data Representation Section (section 5), Bit-Map section (section 6) and Data Section (section 7) to a GRIB2 message. call addfield (cgrib, lcgrib, ipdsnum, ipdstmpl, ipdstmplen, coordlist, numcoord, idrsnum, idrstmpl, idrstmplen, fld, ngrdpts, ibmap, bmap, ierr) For example: GRIB2 section 4 - Product Definition Template Number 4.0 (Analysis of Forecast at a horizontal level or in a horizontal layer at a point in time) ipdsnum=0 0 is Product Definition Template Number 4.0 ( see Code Table 4.0) numcoord=0 Number of values in array “coordlist” coordlist=0.0 Array containing floating point values intended to document the vertical discretization associated to model data on hybrid coordinate vertical levels ipdstmpl(1) = 3 Meteorological products, Mass Category ipdstmpl(2) = 5 Geopotential Height (HGT) parameter number 5 (see Table 4.2-0-3) ipdstmpl(3) = 2 Type of generating process is Forecast (see Code table 4.3) ipdstmpl(4) = 0 Background generating process identifier ipdstmpl(5) = 96 Generating process is GFS model (see Code ON388 Table A) ipdstmpl(6) = 0 Hours of observational data cutoff after reference time ipdstmpl(7) = 0 Minutes of observational data cutoff after reference time ipdstmpl(8) = 1 1 set unit in hour - Indicator of unit of time range ipdstmpl(9) = 6 Forecast time in units defined in ipdstmpl(8) ipdstmpl(10) = 100 Isobaric surface (see Code Table 4.5) ipdstmpl(11) = 0 scale factor of first fixed surface ipdstmpl(12) = 10000 scaled value of first fixed surface (100 mb) ipdstmpl(13) = 255 missing ipdstmpl(14) = 0 scale factor of second fixed surface ipdstmpl(15) = 0 scaled value of second fixed surface

  30. GRIB2 Encoding Routines (continued) -Data Representation Section (section 5) For example: -GRIB2 section 5 - Data Representation Section idrsnum= 40 Data Representation Template Number 40 ( see Code Table 5.0 ) (40 is Grid Point Data - JPEG2000 Compression) idrstmpl(1) = 1236599024 Reference value (R) (IEEE 32-bit floating-point value) idrstmpl(2) = 0 Binary scale factor (E) idrstmpl(3) = 2 Decimal scale factor (D) idrstmpl(4) = 18 Number of bits required to hold the resulting scaled and referenced data values. idrstmpl(5) = 0 Type of original field values idrstmpl(6) = 0 Type of Compression used. (0 for Lossless) idrstmpl(7) = 255 Missing Target compression ratio -GRIB2 section 6 - Bit Map Section ibmap=255 Bitmap indicator ( see Code Table 6.0 ) 0 - A bit map applies to this product and is specified in this section 255 - A bit map does not apply to this product -GRIB2 section 7 - Data Section 1-4 Length of the section in octet (nn) 5 Number of the section 7 6-nn Data in format described by template 7.x where x is the data representation template number given in octets 10-11 of section 5

  31. GRIB2 Encoding Routines (continued) Gribend (gribend.f) This routine finalizes a GRIB2 message after all grids and fields have been added. It adds the End Section (“7777”) to the end of the GRIB2 message and calculates the length of GRIB2 message and stores it in the section 0 in (octet 9 – 16) call gribend (cgrib, lcgrib, lengrib, ierr)

  32. GRIB2 Decoding Routines To Read a GRIB2 message with call routines are following: Skgb (skgb.f) is used to search a file for the next ‘GRIB _ _ _ 2’ (8-btyes) message in GRIB2 file. If found, the length of message is decoded from byte 9 to 16. (see Indicator section 0) call skgb(ifl1,iseek,msk1,lskip,lgrib) Gb_info (gb_info.f) is used to search through a GRIB2 message to find out how many local use sections and data fields found in the a given GRIB2 message call gb_info (cgrib, lcgrib, listsec0, listsec1, numfields, numlocal, maxlocal, ierr) Gf_getfld (gf_getfld.f) is used to get all information of grid definition (section 3), product template (section 4), Bit-map (if applicable), and the unpack data values for a given data field. All information returned is store in a derive type variable “gfld” call gf_getfld (cgrib, lcgrib, ifldnum, unpack, expand, gfld, ierr) Note: gfld is a type of gribfield which is defined in module grib_mod (see grib_mod.f for declaration of derive type gribfield) Users of this routine have to include the line below in your code. Example: use grib_mod parameter (maxsize=200000) integer :: listsec0(3), listsec1(13) type (gribfield) :: gfld . . . See an example of program degrib2.f that how to call routines gb_info.f and gf_getfld.f to get the information and data values in the GRIB2 message.

  33. Declaration of Derived type gribfieldin Module grib_mod • type gribfield          integer :: version,discipline          integer,pointer,dimension(:) :: idsect          integer :: idsectlen          character(len=1),pointer,dimension(:) :: local          integer :: locallen          integer :: ifldnum          integer :: griddef,ngrdpts          integer :: numoct_opt,interp_opt,num_opt          integer,pointer,dimension(:) :: list_opt          integer :: igdtnum,igdtlen          integer,pointer,dimension(:) :: igdtmpl          integer :: ipdtnum,ipdtlen          integer,pointer,dimension(:) :: ipdtmpl          integer :: num_coord          real,pointer,dimension(:) :: coord_list          integer :: ndpts,idrtnum,idrtlen          integer,pointer,dimension(:) :: idrtmpl          logical :: unpacked          logical :: expanded          integer :: ibmap          logical*1,pointer,dimension(:) :: bmap          real,pointer,dimension(:) :: fld      end type gribfield • If variable “gfld” is declared of type gribfield. For example: Type (gribfield) :: gfld The variable “gfld” would have the following components: gfld%version = GRIB edition number ( currently 2 )

  34. Declaration of Derived Type Gribfield (continued) gfld%discipline = Message Discipline ( see Code Table 0.0) gfld%idsect() = Contains the entries in the Identification section 1 gfld%idrtmpl() = Contains the data values for the specified Data Representation Template gfld%idsectlen = Number of elements in gfld%idsect().   gfld%local() = Pointer to character array containing contents                     of Local Section 2, if included gfld%locallen = length of array gfld%local() … etc … See routine (gribmod.f) in documentation block for more components of variable “gfld” is declared of type gribfield

  35. How to use routine getgb2? • Getgb2 (getgb2.f) is used to search for selected fields and unpack from a GRIB2 message Call getgb2 (lugb,lugi,j,jdisc, jids, jpdtn, jpdt, jgdtn,jgdt, unpack, k, gfld, iret) Where: j INTEGER NUMBER OF FIELDS TO SKIP ( J=0 search from beginning) jdisc   GRIB2 DISCIPLINE NUMBER OF REQUESTED FIELD                  ( IF = -1, ACCEPT ANY DISCIPLINE)   jids   INTEGER ARRAY OF VALUES IN THE IDENTIFICATION SECTION                 (=-9999 FOR WILDCARD)  jpdtn     INTEGER PRODUCT DEFINITION TEMPLATE NUMBER (N)                  ( IF = -1, DON'T BOTHER MATCHING PDT - ACCEPT ANY )  jpdt       INTEGER ARRAY OF VALUES DEFINING THE PRODUCT DEFINITION TEMPLATE 4.N OF THE FIELD FOR WHICH TO SEARCH (=-9999 FOR WILDCARD)  jgdtn INTEGER GRID DEFINITION TEMPLATE NUMBER (M)                  ( IF = -1, DON'T BOTHER MATCHING GDT - ACCEPT ANY )  jgdt    INTEGER ARRAY OF VALUES DEFINING THE GRID DEFINITION                  TEMPLATE 3.M OF THE FIELD FOR WHICH TO SEARCH                  (=-9999 FOR WILDCARD) unpack LOGICAL VALUE INDICATING WHETHER TO UNPACK BITMAP/DATA .TRUE. = UNPACK BITMAP AND DATA VALUES .FALSE. = DO NOT UNPACK BITMAP AND DATA VALUES k INTEGER FIELD NUMBER UNPACKED gfld Derived type gribfield ( defined in module grib_mod )

  36. How to Use Getgb2? An example of using getgb2 is used to search for field (Temperature at 100 mb level) by product definition template. Initialize all variables to default -9999 or -1 for wildcard J = 0 Search from the beginning of GRIB2 file DJDSC=-1 Discipline number JIDS=-9999 Identification Section JGDTN=-1 Grid Definition Template Number JGDT=-9999 Grid Definition Template JPDTN=-1 Product Definition Template Number JPDT=-9999 Product Definition Template Search selected field by product definition template number and the values of product definition template 4.n of the field JPDTN = 0 (search for production template 4.0) (if =-1 Do Not bother matching PDT – Accept any) An integer array of values defining the product definition template 4.0 for Temperature at 100mb Level which we want to search as below: JPDT(1:15)=(/ 0,0,2,0,96,0,0,1,6,100,0,10000,255,0,0 /)

  37. How to use Getgb2? (continued) How do we get the information of product template 4.0 for field Temperature at 100mb from GRIB2 message ? We can use utility degrib2 to display the content of GRIB2 message degrib2 grib2file The content of GRIB2 file is displayed below: GRIB MESSAGE 2 starts at 49405 SECTION 0: 0 2 17175 SECTION 1: 7 0 2 1 1 2010 12 13 0 0 0 0 1 Contains 0 Local Sections and 1 data fields. FIELD 1 SECTION 0: 0 2 SECTION 1: 7 0 2 1 1 2010 12 13 0 0 0 0 1 SECTION 3: 0 41760 0 0 0 GRID TEMPLATE 3. 0 : 6 0 0 0 0 0 0 288 145 0 0 90000000 0 48 -90000000 358750000 1250000 1250000 0 NO Optional List Defining Number of Data Points. PRODUCT TEMPLATE 4. 0 : 0 0 2 0 96 0 0 1 6 100 0 10000 255 0 0 FIELD: TMP 100 mb valid 6 hr after 2010121300:00:00 NO Optional Vertical Coordinate List. Num. of Data Points = 41760 NO BIT-MAP DRS TEMPLATE 5. 40 : 1156022272 0 1 9 0 0 255 Data Values: MIN= 185.19999695 AVE= 207.57528687 MAX= 232.00000000

  38. How to use routine putgb2? Putgb2 routine (putgb2.f) is used to pack a single field into a GRIB2 and writes out that message to the file associated with unit number. Call putgb2 (lugb,gfld,iret) NOTE: The information to be packed into the GRIB field is stored in a derived type variable, gfld. Gfld is of type gribfield, which is defined in module grib_mod, so users of this routine will need to include the line "USE GRIB_MOD" in their calling routine. Each component of the gribfield type is described in the subprogram documentation block of routine gribmod.f An example of using routine putgb2.f: This program (g2_putgb2.f) reads an entire GRIB2 file and find selected fields from a file (data card) and uses routine (putgb2.f) to pack into a GRIB2 message and writes them out to a new GRIB2 file. The content of data card is following: &GRIBIDS DESC=' HGT 100 mb ',JPDTN= 0 ,JPDT= 3 5 2 0 96 0 0 1 6 100 0 10000 255 0 0 / &GRIBIDS DESC=' HGT 100 mb ',JPDTN= 0 ,JPDT= 3 5 2 0 96 0 0 1 6 100 0 15000 255 0 0 / &GRIBIDS DESC=' TMP 100 mb ',JPDTN= 0 ,JPDT= 0 0 2 0 96 0 0 1 6 100 0 10000 255 0 0 / &GRIBIDS DESC=' TMP 100 mb ',JPDTN= 0 ,JPDT= 0 0 2 0 96 0 0 1 6 100 0 20000 255 0 0 / &GRIBIDS DESC=' U GRD 100 mb ',JPDTN= 0 ,JPDT= 2 2 2 0 96 0 0 1 6 100 0 10000 255 0 0 / &GRIBIDS DESC=' V GRD 100 mb ',JPDTN= 0 ,JPDT= 2 3 2 0 96 0 0 1 6 100 0 10000 255 0 0 /

  39. GRIB2 in NCEP Operations (continue) /nwprod/util/sorc/copygb2.fd The command copygb2 copies all or part of one GRIB2 file to another GRIB2 file, interpolating if necessary. Unless otherwise directed (-x option), the GRIB2 index file is also used to speed the reading. The fields are interpolated to an output grid if specified (-g option). The interpolation type defaults to bilinear but may be specified directly (-ioption). For example 1: Script to copy all NAM GRIB2 (Regional CONUS - 40.63km - Lambert Conformal) file GRID TEMPLATE 3. 30 : 6 0 0 0 0 0 0 185 129 12190000 226541000 8 25000000 265000000 40635000 40635000 0 64 25000000 25000000 0 0 to NAM GRIB2 (Regional CONUS High Resolution - 15km - Lambert Conformal) GRID TEMPLATE 3. 30 : 6 0 0 0 0 0 0 344 224 19262000 238623000 8 37500000 262000000 i 15000000 15000000 0 64 30000000 60000000 0 0 export grid2="30 6 0 0 0 0 0 0 344 224 19262000 238623000 8 37500000 262000000 15000000 1500000 0 0 64 30000000 60000000 0 0" export g1=nam.t00z.awip3d72.tm00.grib2 export x1=nam.t00z.awip3d72.tm00.grib2i copygb2 -g "$grid2" $g1 $x1 nam.t00z.awip3d72.tm00.grib2.new For example 2: Script to extract selected field "MSLET Mean Sea Level" from NAM GRIB2 (Regional CONUS 212 (40.63km - Lamber Conformal) file by using the information PRODUCT TEMPLATE 4. 0 : 3 192 2 0 84 0 0 1 72 101 0 0 255 0 0 Then, copygb2 interpolates to NAM GRIB2 (Regional CONUS High Resolution - 15km – Lambert Conformal) export grid2="30 6 0 0 0 0 0 0 344 224 19262000 238623000 8 37500000 262000000 15000000 1500000 0 0 64 30000000 60000000 0 0" export g1=nam.t00z.awip3d72.tm00.grib2 copygb2 -k"0 3 192 2 0 84 0 0 1 72 101 0 0 255 0 0" -g "$grid2“ -x $g1 nam.t00z.awip3d72.tm00.grib2_mslet

  40. GRIB2 Reads and Writes Examples • The program readwriteg2.f reads an entire GRIB2 file and extract selected field(s) unpacked from GRIB2 messages and write it out to a new GRIB2 file. • The program readgrib2.f reads an entire GRIB2 file and extract selected field(s) from data card and return packed GRIB2 message(s) to write them out to a new GRIB2 file. • The program degrib2.f reads an entire GRIB2 file and display the content of an entire GRIB2 file. • This program (g2_putgb2.f) reads an entire GRIB2 file and find selected fields from a file (data card) and uses routine (putgb2.f) to pack into a GRIB2 message and writes them out to a new GRIB2 • The program drap2g2.f reads the Global D-region Absorption Prediction ASCII data and write the data in GRIB2 file The examples of these programs are available to download (tar file) from this url: http://www.nco.ncep.noaa.gov/sib/grib2/examples/ • Reads and displays the content of GRIB2 files • Reads and extracts selected fields from data card and write them out to a new file • Reads and extracts selected fields from data card and write them out to a new file using putgb2 • Reads and searches selected fields (by product definition template) and write them out to a new file • Reads a text data and convert to GRIB2 file

  41. SUMMARY • NCEP GRIB2 tools hopefully will provide guidance to all users to generate GRIB2 messages directly • WMO has mandated all operational centers to transition to GRIB2 data for all the operational products • Usage of CNVGRIB type of tools to convert from GRIB1 to GRIB2 should be minimized • Generation of GRIB1 products take heavy storage and are also computationally expensive • NCEP’s parallel GRIB2 generation of NCEP POST using NCO’s g2tmpl library (under development) for NEMS, NMMB and GFS show encouraging results and are very valuable for NCEP's codes to run more efficiently as well as occupy less disk and tape space • NCO will continually improve the GRIB2 Template source code to incorporate all the Product Definition Templates. • NCO will improve the existing GRIB2 tools in the future.

More Related