1 / 21

SPoRT MET Scripts Tutorial

SPoRT MET Scripts Tutorial. Prepared by Bradley zavodsky NASA/MSFC Short-term Prediction research and transition ( SPort ) center. Purpose of SPoRT MET Scripts. Effectively evaluating model performance requires a combination of quantitative metrics and case studies

orinda
Download Presentation

SPoRT MET Scripts Tutorial

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. SPoRT MET Scripts Tutorial Prepared by Bradley zavodsky NASA/MSFC Short-term Prediction research and transition (SPort) center

  2. Purpose of SPoRT MET Scripts • Effectively evaluating model performance requires a combination of quantitative metrics and case studies • SPoRT does not only transition data and products • SPoRT values transition of capabilities that enable NWS partners to perform evaluations that support forecaster-led conference presentations and journal articles SPoRT directly interacts with NWS forecasters

  3. Purpose of SPoRT MET Scripts • Model Evaluation Tools (MET) is a software package developed by NCAR that contains a number of executable programs that will: • Reformat observations • Match model grid to observations • Perform statistical evaluation • Steep learning curve due to missing pieces Dynamic scripts to easily run the software Open source plotting scripts to visualize stats Creating ASCII files from MADIS

  4. Tarball Contents • Once unzipped and untarred, a number of directories, Perl scripts (*.pl), Perl modules (*.pm), and a namelist.met file should appear • Namelist is modified by users to configure what variables and statistics are to be generated for the current run • Scripts run the MET workflow • Modules contain subroutines and code used by multiple scripts • Directories contain documentation, configuration templates used by scripts, or are placeholders where scripts dump data • Users should only modify the namelist.met file to configure • What follows is an overview of the various components that make up the SPoRT MET Scripts

  5. Scripts: runSPoRTMETScripts.pl • Orange dashed circle in workflow that wraps and drives the entire set of scripts and modules • Contains a series of true/false statements that are read from namelist.met to determine which parts of MET will be run • Uses readNamelist.pm to extract the necessary information needed to run the scripts

  6. Scripts: madisFormatAscii2Nc.pl • Orange box, ASCII Point Obs under Input, and ASCII2NC under Reformat in workflow • Creates temporary MET-formatted ASCII file needed to run ASCII2NC • Runs ASCII2NC to create NetCDF files used by Point Stat (output in pointData directory) • sfcobs_YYYYMMDD_HHHH.nc • upperobs_YYYYMMDD_HHHH.nc • Automatically accesses MADIS FTP server to obtain files for each case study date and hour • Users need MADIS account Creating ASCII files from MADIS

  7. Scripts: runPointStat.pl • Green Point Stat circle under Statistics in workflow • Runs Point Stat to match point observations (e.g. METARs and RAOBs) to the nearest grid point in the model output field to generate text files to be read into Stat Analysis (output in pointStatOutput directory) • point_stat_EXP_LEV_FF0000L_YYYYYMMDD_VV0000V_*.stat • EXP = 5 character experiment name • LEV = surface (sfc) or pressure level (PPPmb) • FF = forecast hour time • YYYYMMDD = valid year, month, & day • VV = valid hour

  8. Scripts: runPointStatAnalysis.pl • Green Stat Analysis circle under Analysis in workflow • Uses the *.stat files output from Point Stat to generate statistics comparing the model output to the observations (output in pointStatAnalysisOutput) • LOC_YYYYMMDDFF_VAR_POLY_LEV_EXP_MPR.txt • LOC = surface land (lnd), surface water (wtr), or upper air (upa) • YYYYMMDDFF = year, month, day, and forecast hour • VAR = variable • POLY = verification subdomain • LEV = surface (sfc) or pressure level (PPPmb) • EXP = experiment name

  9. Scripts: runGridStat.pl • Green GenPolyMask and PCP Combine circles under Reformat and green Grid Stat circle under Statistics in workflow • Maps gridded verification data to model forecast grid and produces files of the differences (gridStatOutput directory) • grid_stat_EXP_VAR_FF0000L_YYYYYMMDD_VV0000L_*.stat • EXP = 5 character experiment name • VAR = variable name • FF = forecast initialization time • YYYYMMDD = valid year, month, and day • VV = valid hour • Currently supports grid comparisons between NCEP Stage IV precipitation and NAM-218 analysis (must download manually)

  10. Scripts: runGridStatAnalysis.pl • Green Stat Analysis circle under Statistics in workflow • Uses the *.stat files output from Grid Stat to generate statistics comparing the model output to the gridded verification dataset (output in gridStatAnalysisOutput) • YYYYMMDDFF_EXP_VAR_sfc_NBHOOD_POLY.txt • YYYYMMDDFF = year, month, day, and forecast hour • EXP = experiment name • VAR = variable • NBHOOD = number of surrounding grid points to consider in matching • POLY = verification subdomain

  11. namelist.met: Overview • namelist.met text file is designed for easy modification to perform evaluations on only metrics of interest • Each script begins with a series of calls to the readNamelist.pm module which scours namelist.met for only variables needed by each script and defines these variables as designated by the user • Some variables can be a list and should be separated by a comma (with no spaces) • Blocks separate like variables

  12. Tutorial • Objective: set up data files and namelist.met to verify 0-24h control and SPoRT forecasts output every three hours from 27 April 2011 • Should have already downloaded tarballs containing updated MET scripts and tutorial data • Untar MET file • Create a directory called ‘forecasts’ • Move Forecast file to newly created directory; untar • Move Stage IV file to ‘gridData’ directory; untar • Open namelist.met using your favorite text editor

  13. namelist.met: &DirectoryInfo Block • Gets scripts familiar with user’s directory structure and points script to data to process • RunDir: full pathname of directory where the SPoRT MET script was untarred and will run • NetCDFDir: full pathname of NetCDF directory for running ncdump to process MADIS data • EMSBinDir: full pathname of directory in WRF-EMS software package containing wgrib, wgrib2, etc. utility binaries • METDir: full pathname of highest level MET directory • ForecastDir: full pathname of location of all GRIB forecast files to be processed

  14. namelist.met: &Domain and &Time • &Domain helps to reduce processing time by only extracting observations that are within the model domain • Input the lower left and upper right latitude and longitude (enter the LL and UR coordinates of your local domain) • &Time controls the number of forecast days are going to be evaluated • Enter the first initialization date and time for the “Start” variables (enter 2011, 04, 27, and 00 for Start variables) • Enter the final forecast date and time for the “End” variables (enter 2011, 04, 28, and 00 for End variables) • NumberVerifyDays: number of consecutive case study days (1)

  15. namelist.met: &ForecastInfo • &ForecastInfo provides the needed information to tell the script how many hours each forecast is, how frequently a GRIB file of model output was generated, and the names of the two (or more) experiments being verified • TotalForecastHours: total number of hours per forecast (24) • ForecastVerifyInterval: how often were the forecasts output? Or with what frequency does the user want to verify (3) • ExperimentNames: a list of names that match the five-character experiment appended to the beginning of the GRIBNAME variable in WRF-EMS (SPORT,CNTRL)

  16. namelist.met: &MADISInfo • &MADISInfo allows the user to configure which MADIS observations he/she wants to verify against • Currently, ACARS profiles, profiler, RAOB, METAR, Mesonet, Maritime, and SAO are available • Set the ProcessMADIS variable to true to do point verifcation; set to false if only doing grid verification (true) • Set the UseVAR variables to true to use a dataset; set to false to exclude from verification (set all UseVAR variables to true) • TimeRangeVAR variables tell MET to match up observations that fall within ±n minutes of the forecast valid time (Raob to 30; all others to 10) • Useful for stations that do not always report exactly at the top of the hour when the forecasts are valid

  17. namelist.met: &METInfo • &METInfo allows for configuration of which parts of MET will run and allows for a user-defined verification domain (if a sub-domain of the overall domain is desired) • RunPROGRAM: set to true to run each component of the MET package; set to false to not run selected components (all true) • PressureLevels: upper air pressure levels (in hPa) to be verified by either Point Stat or Grid Stat (850,500) • VerificationRegions: NCEP verification region on which to verify, USER for a user-defined domain, GRID for entire model grid (USER) • UserVerifyCOORDINATES: lower left and upper right corners of user-defined verification grid (enter the LL and UR coordinates of your local domain)

  18. namelist.met: &PointStatInfo • &PointStatInfo provides the needed information to run Point Stat • UseVerifySurfacePoint/UpperPoint: set to true to verify against surface observations and/or upper air observations respectively (true) • Surface/UpperPointVerificationVariables: GRIB table variable name for variables on which to perform verification (Surface: TMP,DPT,PRMSL; Upper: TMP,DPT) • VerticalObsWindow: Vertical pressure range (hPa) over which upper air observations will be accepted for forecast matching (10) • StatsFilter: Easiest to just set this to MPR for now (MPR)

  19. namelist.met: &GridStatInfo • &GridStatInfo provides the needed information to run Grid Stat • NeighborhoodBox: provides the width of the neighboorhood grids over which verification is performed (5) • Must be an odd number • If set to 1 will only do grid point to grid point matching • PercentCoverage: determines percentage of a neighborhood grid has to contain the forecasted value for a hit to register (0.25) • UseVerifyVAR: set to true to verify against precipitation and/or gridded analysis (Precipitation: true; Grids: false) • GriddedPrecipitationVerificationAnalysis: set to STIV as NCEP Stage IV is the only precipitation analysis currently supported (STIV)

  20. namelist.met: &GridStatInfo (cont’d) • &GridStatInfo provides the needed information to run Grid Stat • AccumulatedPrecipitationHours: accumulated precipitation totals to be verified for (must be greater than forecastInterval (03,12) • PrecipitationThresholds: precipitation thresholds to use for binning skill scores (in mm) (1,5,10,25,50) • GriddedVerificationModel: determines which large-scale NCEP analysis will be used for gridded verification of non-precipitation variables (none for tutorial) • Surface/UpperGridVerificationVariables: GRIB table variable name of variable on which to verify (none for tutorial)

  21. namelist.met: &PlottingInfo • &PlottingInfo allows the user to automatically produce plots of the hour-by-hour forecast validation using the Open Source GD::Graph Perl module • MakePlots: set to true to generate plots or set to false to make your own from the ASCII output • Surface/UpperPlotVariables: GRIB table variable name for surface/upper-air variables for which to generate plots • PressurePlotLevels: pressure levels (hPa) on which to generate plots • NCEPPlotRegions: NCEP verification regions to generate plots • PlotStatistics: statistics to plot • PlotColors: color of line for each forecast (in same order as experiments were defined in ExperimentsNames variable under &ForecastInfo

More Related