1 / 64

CHPS Calibration Features

CHPS Calibration Features. Ben Balk (AMEC) and Matthijs Lemans (Deltares USA) December 3, 2013. CHPS Calibration Features Agenda. Introduction Broad overview of initial operating capabilities Highlight OHD documents and templates FEWS demo – general calibration features

christopher-roy
Download Presentation

CHPS Calibration Features

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. CHPS Calibration Features Ben Balk (AMEC) and Matthijs Lemans (Deltares USA) December 3, 2013

  2. CHPS Calibration Features Agenda • Introduction • Broad overview of initial operating capabilities • Highlight OHD documents and templates • FEWS demo – general calibration features • Focus on displays and general layout • Configuration overview • OHD Calibration Configuration Guide • FEWS demo – calibration run • OHD Calibration Reference Manual • Discussion • Some ways to tailor configuration for RFC needs • Summary

  3. Introduction • Overview of CHPS calibration features • Replicate MCP3/ICP capabilities (not OPT3) • Not identical, but comparable functionality • Initial release similar to GraphGen roll-out in previous release • Transformations/Configurations • FEWS transformations (STATQME, WATERBAL, PEAKFLOW) • OHD – FEWS Open API/custom transformations • OHD documents and templates • Calibration Reference Manual • Calibration Configuration Guide • Configuration templates/samples

  4. Introduction • Calibration workflow • Current workflows • Preprocessor • Basin model • <!--<loopTimeStep id=“12Z”>--> • Line 45 or 48 in GA modules • New calibration modules • WATBAL • STATQME • PEAKFLOW • New report module

  5. Introduction • Calibration workflow • Current workflows • Preprocessor • Basin model • <!--<loopTimeStep id="12Z"/>--> • Line 45 or 48 in GA modules • New calibration modules • WATBAL • STATQME • PEAKFLOW • New report module

  6. Introduction • Calibration workflow • Current workflows • Preprocessor • Basin model • <!--<loopTimeStep id="12Z"/>--> • Line 45 or 48 in GA modules • New calibration modules • WATBAL • STATQME • PEAKFLOW • New report module

  7. Introduction • Import Data Cards • Historical MAP, MAT and QME data • Manual Forecast Display • Set cold state start to beginning of calibration period • Running calibration workflow • Manual Forecast Display or Forecast tab • Cold state start set manually or default in Topology.xml

  8. Introduction • Run with Modifiers • Use Modifiers tab for Snow-17, SAC-SMA, Unit-HG • multipleModuleParameterModifier – only add to calb SA • snow17 calibration • sacsma calibration • Preserve ratio/difference between sub-basin parameters • Keeping track of modifiers • New Modifiers tab in Forecast Management Display • Restore deleted modifiers from previous runs • Export to updated ModuleParFile

  9. Introduction • Viewing data within FEWS • Data Viewer (Filters.xml) • Plots (DisplayGroups.xml) • ICP-like displays • Previous simulations • HTML report

  10. FEWS Demo – General Calibration Features • Repeat and highlight initial overview • Use pre-loaded localDataStore • NWRFC basin • Snow-17 and SAC-SMA • Two elevation zones • Focus • Displays • General layout and functionality • Preview of live demo to follow Configuration presentation

  11. Configuration Overview • Calibrations should be done in separate SA (xxrfc_calb) • Calibration Configuration Guide • Installing CHPS calibration components • Thorough step-by-step process for initial implementation • Nicely grouped by directories • Assume basin or segment already configured in existing SA • Affected configuration files • Nice summary image (page 6 – Config Guide) • Details new (green) and modified files (blue) • Details directory structure • First time installation vs. copied/edited files for new basin • No direct upload to OC except: • ModuleParFiles • Refreshed ColdStateFiles

  12. Configuration – Installing Components • Sample templates included in OHD-CORE-CHPS-3.2.a • ConfigSAMPLE_rosn7_12Z (SAC-SMA) • ConfigSAMPLE_slyp1awkAPICONT_12Z (SNOW-17, API-CONT) • ConfigSAMPLE_wgcm8_00Z (SNOW-17, SAC-SMA, two elev zones) • ConfigTEMPLATE • Use appropriate template to set up initial basin • Simple headwater configurations • Global search and replace of sample basin IDs • May need additional templates (e.g. routing, >2 sub-basins) • No scripts provided to convert NWSRFS decks to CHPS • Time issues (local time vs. GMT) • OHD templates in local time – end of calendar day is 00Z • Appendix A: Use of Time Zones in FEWS

  13. Configuration – Workflows • WGCM8_Stats_Calibration • WorkflowFiles/calibration • Copy appropriate workflow • Edit for new basin • Edit forecast group if needed • PreProcessing_UpdateStates • MergeMAP • MergeMAT • ImportDataCard • Flow_UpdateStates • Basin models • Update GA modules • Register new workflow • WorkflowDescriptors

  14. Configuration – Workflows • WGCM8_Stats_Calibration • WorkflowFiles/calibration • Copy appropriate workflow • Edit for new basin • Edit forecast group if needed • PreProcessing_UpdateStates • MergeMAP • MergeMAT • ImportDataCard • Flow_UpdateStates • Basin models • Update GA modules • Register new workflow • WorkflowDescriptors

  15. Configuration – Workflows • WGCM8_Stats_Calibration • WorkflowFiles/calibration • Copy appropriate workflow • Edit for new basin • Edit forecast group if needed • PreProcessing_UpdateStates • MergeMAP • MergeMAT • ImportDataCard • Flow_UpdateStates • Basin models • Update GA modules • Register new workflow • WorkflowDescriptors

  16. Configuration – Workflows • WGCM8_Stats_Calibration • WorkflowFiles/calibration • Copy appropriate workflow • Edit for new basin • Edit forecast group if needed • PreProcessing_UpdateStates • MergeMAP • MergeMAT • ImportDataCard • Flow_UpdateStates • Basin models • Update GA modules • Register new workflow • WorkflowDescriptors

  17. Configuration – PreProcessing Modules • MergeMAP_UpdateStates • ModuleConfigFiles/preprocessing • Variable • Add MAP_cardfiles • Transformation • Add simple expression OR • Add inputVariable to existing merge hierarchy • Edit forecast group name • Repeat for MergeMAT • ImportDataCard module • ModuleConfigFiles/import • Add QME timeSeriesSet • <timeStepunit="day"/> • Correct timeZoneOffset

  18. Configuration – Flow_UpdateStates Modules • Basin models • cd ModuleConfigFiles/xxxxx • rm –f ModuleConfigFiles/xxxxx/*Forecast.xml • Update GA modules • Comment out loopTimeStep so model does not loop daily • Comment out <timeZone> in SNOW17, SACSMA

  19. Configuration – Calibration Modules • Add directory for new basin • ModuleConfigFiles/calibration/xxxxx • Copy calibration modules from similar basin (e.g. wgcm8) to xxxxx directory • Rename appropriately • Edit all modules using global replace for new basin & catchments • Double-check for correct moduleInstanceId’s and locationId’s • WATBAL • STATQME • PEAKFLOW • Copy Report_STATQME module • ModuleConfigFiles/calibration • Only needed for initial installation • Register new modules • ModuleInstanceDescriptors

  20. Configuration – Water Balance Modules • Check WATBAL_SNOW17 modules for: • Correct FGroupId MergeMAT_UpdateStates reference • MAT_6h variable • Correct moduleInstanceId and locationId for RSEL parameter • RSEL_6h variable • RSEL_day variable • TimeSeriesSets used in Displays • Skip if no SNOW17 model used in basin

  21. Configuration – STATQME Modules • Check STATQME_xxxxx_Calibration module for: • Correct moduleInstanceId reference for basin SQIN • SQIN_6hr variable • Could be ADDSUB, LAG/K, UNITHG, etc. • Proper module instances included in moduleInstanceSet • Zone_SQIN_6hr variable • Catchments_SQIN_UpdateStates • Only needed for multiple sub-basins or elevation zones

  22. Configuration – STATQME Modules • STATQME_xxxxx_Calibration_StatEquations • Flow interval values • Copy values from NWSRFS deck • Card 2; column 26-55 if it exists • Add them to xml snippet below:

  23. Configuration – Peakflow Modules • PEAKFLOW_xxxxx_Selection_Calibration • Can set lower and upper limits for peak selections • Can set gap period between peak selections

  24. Configuration – where we stand now • WGCM8_Stats_Calibration • Copy appropriate workflow • Edit for new basin • PreProcessing_UpdateStates • MergeMAP, MergeMAT • ImportDataCard • Flow_UpdateStates • Update GA modules • Calibration modules • WATBAL • STATQME • PEAKFLOW • Registered workflow • Registered modules

  25. Configuration – What is needed for displays • ModuleConfigFiles – OHD templates • Add time series in SNOW17 and SACSMA modules • RegionConfigFiles – OHD templates • Parameters.xml • Filters.xml • ModuleInstanceSets.xml • LocationSets.xml • Qualifiers.xml • TimeSteps.xml • ThresholdValueSets.xml • Thresholds.xml • Topology.xml • SystemConfigFiles – OHD templates • DisplayGroups.xml • TimeSeriesDisplayConfig.xml • UnitConversionsFiles – OHD template • displayEnglishUnits.xml

  26. Display Configuration – ModuleConfigFiles Add time series in SNOW17 and SACSMA modules ModuleConfigFiles/xxxxx <importTimeSeriesActivity> SACSMA UpdateStates LZDEFR DIR_RO IMP_RO INT_RO PRI_RO SUP_RO SUR_RO • SNOW17 UpdateStates • PSFALL • PRAIN • SNTMP • NEGHS • LIQW • PQNET

  27. Display Configuration – RegionConfigFiles • Parameters.xml • New parameters just added to SNOW17 and SACSMA UpdateStates modules • Add new parameters calculated within new calibration modules • SQME.PCTBIAS, SQME.CC, SQME.LOG, etc. • May have to merge with existing Parameters.xml • Some parameters may already be defined for your RFC • parameterGroup, unit, displayUnit • Flows to be displayed in CMSD like ICP • Filters.xml • Copy appropriate template • Default includes SNOW17 • WithoutSNOW17 • withAPICONT

  28. Display Configuration – RegionConfigFiles Example time series in Filters.xml LocationSets.xml • ModuleInstanceSets.xml

  29. Display Configuration – RegionConfigFiles • Qualifiers.xml • Self-explanatory; place at top of file • TimeSteps.xml • Add wateryear, monthly, season and quarterly time steps • Start of aggregation period is exclusive; end is inclusive

  30. Display Configuration – RegionConfigFiles Thresholds.xml • ThresholdValueSets.xml

  31. Display Configuration – RegionConfigFiles • Topology.xml • showRunApprovedForecastButton – true • Usually disabled during segment forecast runs • coldState start • Node is triggered to run calibration workflow

  32. Display Configuration – RegionConfigFiles • Topology.xml – alternative approach • Extra nodes to link to Plot Overview thumbnails • All nodes run the same calibration workflow • Use of filterId to link Forecast node to Data Viewer node

  33. Display Configuration – SystemConfigFiles • TimeSeriesDisplayConfig.xml • Selection of preferred display options for parameters • preferredColor • lineStyle • markerStyle • Check schemas for options • Can configure as you wish • Will be used in Data Viewer • Implications when viewing previous runs

  34. Display Configuration – SystemConfigFiles • DisplayGroups.xml • Copy appropriate template • Default includes SNOW17 • WithoutSNOW17 • withAPICONT • Predetermined plots • Defined color schemes • ModuleInstanceSets • LocationSets • Time series displays from calibration modules • ICP-like displays • Can configure to your needs • Colors • relativeViewPeriods • Addition of plots

  35. Display Configuration – SystemConfigFiles • DisplayGroups.xml • <nodeId> links to “nodeid” in Topology.xml (Forecasts tab) • <plotId> links to “plotid” template within DisplayGroups.xml • locationSet in <timeSeriesSet> filters on <locationId> in displays

  36. Display Configuration – UnitConversionsFiles • displayEnglishUnits.xml • Unit conversions added for calibration • Most notably a log conversion for QME and SQME

  37. Configuration – What is needed for modifiers • RegionConfigFiles/ModifierTypes.xml • multipleModuleParameterModifier – only add to calb SA • snow17 calibration • sacsma calibration • Preserve ratio/difference between sub-basin parameters • Min/max values • Expiry time

  38. Configuration – What is needed for modifiers • DisplayConfigFiles/ModifierDisplay.xml • Create shortcut buttons in Modifiers tab (optional)

  39. Configuration – What is needed for modifiers • ModuleParFiles/xxxxx/SACSMA_*_UpdateStates.xml • Add RUNOFF_COMPONENT_INTERVAL to all SACSMA parameter files • ColdStateFiles/xxxxx/ • Copy updated SACSMA parameter files to ColdStateFiles directory • Refresh the zip file; do separately for each catchment (i.e. LWR/UPR) • cd <ColdStateFiles_dir>/xxxxx • cp ../../ModuleParFiles/xxxxx/SACSMA_*_UpdateStates.xml params_previous.xml • zip –f “SACSMA_*_UpdateStates Default.zip” params_previous.xml • * is a token; expand to your RFC nomenclature • rm params_previous.xml • Skip above two bullet items if interval already exists • Use steps to refresh cold state parameters at end of calibration

  40. Configuration – additional files for local time • RegionConfigFiles/ModuleConfigProperties.xml • Calibrations run in local time assuming midnight is 00Z • STARTLOCALHOUROFDAY set to zero • STARTLOCALHOUROFDAY is referenced in SNOW17 GA module

  41. Configuration – HTML report

  42. Configuration – What is needed for HTML report • SystemConfigFiles/Explorer.xml • Add Calibration Report explorerTask just before closing </explorerTasks> tag • sa_global.properties • Add contents below to set up your web browser

  43. Configuration – What is needed for HTML report • statqme_statistical_summary_template.html • mkdir ReportTemplateFiles in base Config directory • Copy provided file to ReportTemplateFiles directory • .html file can be customized • Reports/statqme • mkdir –p Reports/statqme in base calb SA directory (xxrfc_calb) • statqmeStyles.css • Copy provided file to Reports directory • .css file determines style of report; can be customized • CoefficientSetsFiles/Area_Coefficients.xml • mkdir CoefficientSetsFiles in base Config directory • Copy provided template and edit appropriately • Basin area in km2 • Needed to compute flow interval biases for report

  44. Configuration – DataCards • Copy MAT/MAP/QME datacards to backup directory • Suggest creating a backup cardfiles directory if one does not already exist • mkdir –p Import/backup/cardfiles in base calb SA (xxrfc_calb) • Datacard preparation for calibration • MATs/MAPs • Check column 35 in datacard header line for appropriate catchment name • e.g. WGCM8U, WGCM8L • QME • At column 35 in datacard header line, change USGS id to FEWS location id • e.g. WGCM8 • No zeros are allowed in the months column, so a global search and replace needed • e.g. :%s/ 01/ 1/g will replace all 01 with 1 for January • FogBugz Case 862 • Copy prepared datacards to Import/cardfiles directory for import

  45. FEWS Demo – Calibration Run • Start with empty localDataStore • NWRFC basin • Import Data Cards • Historical MAP, MAT and QME data in Manual Forecast Display • Set cold state start to beginning of calibration period • Running calibration workflow • Manual Forecast Display or Forecast tab • Cold state start set in Topology.xml • T0 set in sa_global.properties • Run with Modifiers • Preserve ratio/difference between sub-basin parameters • Keeping track of modifiers; use of descriptions • Restore deleted modifiers from previous runs • Export to updated ModuleParFile

  46. FEWS Demo – Calibration Run • Viewing data within FEWS • Data Viewer (Filters.xml) • Plots (DisplayGroups.xml) • ICP-like displays • Previous simulations • HTML report • Show some tailored configurations with pre-loaded localDataStore • APRFC basin • Plot displays • Filters • Previous simulations • Suggestion for new plot template in DisplayGroups • QME/SQME with no predefined color; will key off TimeSeriesDisplayConfig • Overlap qualifier • Necessary if missing gaps in QME record

  47. Forecast Management Display • Recap from demo • Modifiers tab • Description – added through “Run options” or Manual Forecast Display • Show details modifier • Copy

  48. Modifiers • Recap from demo • Restored modifier from copy function in Forecast Management Display • Magenta – needs to be rerun • Export button to drop moduleParFile to selected directory • Next to Apply button

  49. Overlap qualifier • Overlap qualifier • Not all QME records will be complete • If QME record has gaps STATQME stats in report will not match ICP summary • Both QME and SQME stats derived for available data • APRFC example (T0 10-01-2006, cold state start 10-01-1970): • QME: 1978-1993: 2000-2006 • SQME: 1970-2006

  50. Overlap qualifier • Overlap qualifier (continued) • Added additional variables and transformations with an overlap qualifier • Report overlap variables in HTML report • Statistics match ICP statistical summary

More Related