DSST Short-Term Ensemble Planning (STEP) Call #2

1. DSST Short-Term Ensemble Planning (STEP) Call #2 RFC Presentations and Discussion DSST STEP Conf Call #2

2. Agenda • Introduction (DJ) • RFC presentations and Q/A • AB • AP • CB • CN • MA • MB • NC • NE • NW • Comments by other RFCs • Discussion (All) • Towards developing (near-) consensus operations concept and identifying overarching needs and issues (service, operations and science) • What next? (DJ) DSST STEP Conf Call #2

3. ABRFC DSST STEP Conf Call #2

4. Operations Concept • I see the ABRFC using short term ensembles (STE) as the "official" way to issue almost all of our forecasts. That includes our 27 daily forecast, as well as our forecasts we issue for the other 200+ flood forecast points we have.  Only our long term water supply forecasts would be different. If I had our way, I would have implemented the STE as the "AWIPS" requirement fulfillment long ago. 98% of our users do not want long term ESP forecasts! • I picture that we would issue the forecasts mainly in a graphical method, much like the examples that this office has provided (see below).   Instead of issuing just deterministic forecasts like we do now, I see STE as a replacement/addition to this.  DSST STEP Conf Call #2

5. Operations Concept (cont.) • Another use of short-term ensemble (STE) forecasts is by the WFOs as a basis for triggering a river flood watch • If we got the procedures efficient enough, we could issue our STE forecasts routinely for all of our forecast points, not just for flood situations. This way, with experience and possibly local WFO procedures, the WFOs could use the STE forecasts as the main part of the decision process in determining when to issue flood watches and outlooks. DSST STEP Conf Call #2

6. Needs and Issues • One thing that NEEDS be to accomplished is for us to determine or define a relationship between probabilistic forecasts and a single deterministic forecast. I think all STE forecasts should also include the office’s best guess of a deterministic forecast, and this deterministic forecast should be located near the 50% probability line. We have to define how deterministic forecasts relate to probabilistic forecasts before we can go any further. • Many of our users just want one forecast, i.e. the deterministic forecast, while other more sophisticated users will enjoy the probabilistic forecast. Issuing a STE with both data would be the best bet I think! • I also think that the STE info could be presented in a text format, but the exact format could be determined later.   DSST STEP Conf Call #2

7. Needs and Issues (cont.) • Operationally, I see MANY significant things that need to be accomplished. Once you get the science down, we need software that will produce the graphics/text output of this data in a user friendly manner (i.e. xsets/hydrograph creation software). • We need training for those producing and using these forecasts. • We need a solid verification set of data proving that the science behind STE is reliable and sound. I have yet to see any verification (including that which I have done) that convinces me that the science behind the STE is producing reliable and accurate short term forecasts. • As far as the DA goes, I still am not convinced it works. The little I have heard from WGRFC users is that they were not satisfied with it. I think we need some extensive verification of it, and HICS/DOHS/RFCS need to be in the decision to buy off on this. At this point, I have not seen any convincing evidence that it works. DSST STEP Conf Call #2

8. Needs and Issues (cont.) • Overall, I am excited about STE, but still am not convinced that the science behind our current configuration works. • I do however foresee a future where the vast majority of forecasts issued from the ABRFC are STE, which will allow for a great advance in the hydrologic program. DSST STEP Conf Call #2

9. APRFC DSST STEP Conf Call #2

10. APRFC HAS ConOps • Ability to select model(s) of choice • Ability to evaluate 6-hour QPF for a set of stations • Ability to apply post processing to get pcpn range for each station • Ability to review/edit pcpn ranges • Ability to apply “Mountain Mapper” functionality to get MAPs • Ability to review/edit MAP value ranges • Ability to repeat process for temperatures DSST STEP Conf Call #2

11. APRFC Hydro ConOps • Ability to review forecast and range of MAP and MAT values • Ability to run IFP and see resulting hydrographs for range of inputs • Ability to adjust MAP/MAT relationships • Ability to run post processing to get most likely and range of hydrographs DSST STEP Conf Call #2

12. APRFC Summary Comments • We think that short term ensembles have equal or greater value than long term ensembles for our users • We have concerns about • the ability to quantify the uncertainty in QPF given its significant spatial and temporal variability and individual forecaster bias • the ability to quantify the uncertainty of the interrelated QPF and QTF variables and the resulting impact on the ensemble of hydrographs • the apparent conflicts of wanting control over the output (tweaking the results) vs. maintaining unbiased statistical ensembles DSST STEP Conf Call #2

13. CBRFC DSST STEP Conf Call #2

14. CBRFC Short-term Ensemble Opportunity: We can improve our short to medium range esp forecasts by using numerical model predictions in place of historical data. DSST STEP Conf Call #2

15. But... We can not directly use output from a weather model, even if it is in ensemble form. Our conceptual model does not necessarily want to see reality. It wants to see its own twisted version of reality. We get that twisted version by determining a relationship between what the weather model predicts and what our river model has been calibrated with. Otherwise known as downscaling. DSST STEP Conf Call #2

16. Project Area: 27 Segments Above Cameo, Colorado River All recently recalibrated and set up For ESP. DSST STEP Conf Call #2

17. Downscaling • MRF Variables: • 2m air temp • Precipitation • 700mb Relative Humidity • Sea Level Pressure • 10m Vector Wind • Total Column Precipitable Water • Basin Scale Variables: • Mean Areal • Temperature • Mean Areal • Precipitation DSST STEP Conf Call #2

18. In order to downscale we need several years, if not decades of forecast data. This requires a re-forecast process of the weather model. In addition, for operational use, we need that weather model output to stay consistent with the relationships we have determined. If it is changed to use better physics, for example, we would then need a re-forecast data set and determine new downscaling equations. DSST STEP Conf Call #2

19. Downscaling Results MRF is colder than normal in this case. DSST STEP Conf Call #2

20. Input into ESP MRF derived MAT/MAPs are attached to historical years (“ensembles”) and ‘fed’ to ESP. DSST STEP Conf Call #2

21. ESP peak flow Smaller peaks because MRF is colder for first 14 days causes less melt. DSST STEP Conf Call #2

22. Operational challenges • Scale – it may be too large for precipitation. In general, smaller scale should be better • Availability – we need commitment to use this operationally • Re-forecast process – any changes in model physics or scale requires a re-forecast and new downscaling equations. The good news is the weather community is starting to see the value in statistical correction. • Presentation – we should keep these forecasts distinct which will require new data storage and handling techniques DSST STEP Conf Call #2

23. CNRFC DSST STEP Conf Call #2

24. CNRFC comments on short term ensemble • Need customer buy-in, “So how good are you guys anyway?” • 50% or median….need a measure of reliability that is easily understood by users • 90% 10% bounds….is the 10% exceedance number really exceeded only 10% of the time? • A reasonable range of expected values should be very useful to the emergency folks • Need to adequately explain how uncertainty increases as you move into the future • Conditional uncertainty….Some storms (strong cold front crossing the state) are much easier to forecast that others (cut-off low just off the coast) We should be able to modulate the uncertainty accordingly. DSST STEP Conf Call #2

25. A Vision for Operational Hydrologic Short-Term Ensemble Forecasting - Rob Hartman (AHPS Theme Team, June 2005) • For many years, NWS customers have benefited from probabilistic long-range water supply forecasts in the Western U.S. The potential benefits of accurate short-term probabilistic flood forecasts are very significant. This becomes obvious when one considers the cost of local emergency management activities. • For several years now, OHD has been working on short-term ensemble prototypes. These efforts have been concentrated on developing ensemble inputs through model downscaling or simulation based on the joint distribution of forecasts and observations. Additional activities such as post processing and data assimilation (DA) have been identified. To date, however, the operational environment for short-term ensemble based probabilistic forecasts has not been described. • The time-line for AHPS implementation of short-term probabilistic forecasts is such that the forecasting environment will certainly still involve OFS and the IFP. As such, operational ensemble forecasts must function in this environment. This will require several changes to the ESP and OFS architecture. Here is a typical scenario: DSST STEP Conf Call #2

26. The RFC is experiencing moderate flooding in several watersheds. A forecast update is due out at 03Z. Data come in after 00Z for the second period of the day, the 5-day QPF is updated by the HAS function. Input data are QC’d and the forecaster starts his/her IFP run to update guidance. • The first segment is a flood forecast point. The output includes the single-value forecast as well as shaded regions that depict the probabilistic forecast with user definable regions (10% EP , 25% EP, Ensemble Mean, 75% EP, 90% EP, etc). The output indicates that there is a 40% chance that the river will rise to 3 feet over flood stage by noon tomorrow (mouse tracker interpolation). Three feet over flood stage is a critical stage for local mitigation. But wait, the simulation appears to be a bit screwy. Upon examination, the forecaster sees that bad precipitation data made it through the QC process. The MAP for the 18-00Z period needs to be increased by 50%. A MOD is made. The forecaster reruns the segment. The ensembles are regenerated and included in the display as before. The single-value forecast and probabilities shift slightly. The guidance looks reasonable. The phone rings. It’s the WFO and their local EM needs a forecast update right now. The forecaster selects “Issue guidance” from the pull-down menu and the system initiates the process that generates and issues the single-value and probabilistic guidance for this location. DSST STEP Conf Call #2

27. This scenario identifies several issues that are not currently supported with OFS and ESP. These include: • 1. The notion of carryover must to be re-examined so that ESP can be run interactively from any point in time and for individual components of a forecast group. Ensemble generation must become interactive rather than a batch process. Performance must support interactive use (rerun small pieces in a few seconds). • 2. The ESP process must thoroughly re-examine the notion of MODs and their impacts on short-term ensembles. Some believe that DA and automatic state updating is the only solution to avoid MODs. In the midst of forecasting, this is unrealistic. There will always be times when a forecaster needs to drive the model to the appropriate outcome. That’s why we have forecasters. • 3. Statistical post-processing techniques must to be fast and interactive. • 4. Visualization tools must to be developed within the IFP framework to support ensemble and probabilistic information. • 5. Ensemble and probabilistic information must be managed to facilitate the generation of products and guidance. • 6. OFS does not write information back to the processed DB until the segment is exited. This prevents product and information generation while looking at the IFP display. • Without doubt, we’ll find lots of other issues as we attempt operational implementation of short-term probabilistic forecasts. As such, it is important to being addressing the issues and developing an operational prototype. DSST STEP Conf Call #2

28. MARFC DSST STEP Conf Call #2

29. Shorter Term Probabilistic Forecasts • 7-day probabilistic river forecasts • PQPF/PQTF • Demonstration of short-term approach • Juniata & Schuylkill Basins • 18 points issued daily DSST STEP Conf Call #2

30. 7-Day Probabilistic River Forecasts • Current Basin Conditions • Short term probabilistic precipitation and temperatures (PQPF/PQTF) • 48-hour PQPF merged with 5 days of climo • QPF scenarios based on comparison of historic forecast and observed MAPs • 3 graphics generated daily for 18 basins in PA-Juniata (CTP) and Schuylkill (PHI) Basins DSST STEP Conf Call #2

31. WFO CTP AHPS Page DSST STEP Conf Call #2

32. Frankstown Br. Juniata River at Williamsburg, PA DSST STEP Conf Call #2

33. Difficult to Find on AHPS Web Page DSST STEP Conf Call #2

34. 7-Day Expected Value Plot DSST STEP Conf Call #2

35. 7-Day PQPF Traces DSST STEP Conf Call #2

36. Lessons Learned • 7-day PQPF/PQTF forecasts are good contingency forecasts • Describe a range of outcomes that help address HSA questions…”What if..” • Wide range of potential river responses is not always pleasing…depicts difficulty in forecasting precipitation DSST STEP Conf Call #2

37. Lessons Learned • Only limited use by WFO’s and cooperators (much less than 30-day ESP forecasts) • Difficult for “non-technical” audience • Software runs well and can generate graphics in a “hands-off” mode • Once rain is on the ground we have considerable modeling uncertainties not incorporated in this method DSST STEP Conf Call #2

38. Lessons Learned • Await results of OHD verification work to assess validity of products • MARFC would like to generate probabilistic forecasts beyond day 2 based on more than climatology • Potential collaboration with WFO CTP and SREF ensembles DSST STEP Conf Call #2

39. MBRFC DSST STEP Conf Call #2

40. Define User Requirements • Have user create own on the fly if capable • Graphics within hydrograph plot if adequate description provided DSST STEP Conf Call #2

41. Vision of Operations • Balance rivers without ensemble forecasts • Switch on ensemble forecasting and look at rivers again • Data assimilation used as guides to mods • Displays of historical precip and streamflow to compare distributions DSST STEP Conf Call #2

42. Concerns • How are segments linked? Is a precip value at one basin linked to a precip value at a basin nearby? Is trace for one location linked to a trace at the downstream location? • Users will think uncertainty distribution in hydrograph is really a set of hydrographs • Recent historical QPF and MAP are limited and have been dynamic with time DSST STEP Conf Call #2

43. Concerns (cont.) • Basin boundary changes and addition of new basins require recomputing MAPs and QPFs • Inconsistency between 14-day short term and 90-day long term • How do you handle forecast locations that are model deficient and require manual override? Blending of regulation modeling? • Use of mods in the future? DSST STEP Conf Call #2

44. Concerns (cont.) • How can distributions be adjusted operationally? • What can forecaster view to get some idea of possible needed adjustments to the distribution plots? Are ensemble data out of bounds? DSST STEP Conf Call #2

45. NCRFC DSST STEP Conf Call #2

46. Operations Concept – Ideas • Need the ability to operate and generate forecasts at the sub-basin level (segment) • Ensemble generation and viewing should be controlled by an option switch (on/off/default) to allow for interaction with the model for basic tuning before introducing complications of ensembles (assumes integration w/ IFP) DSST STEP Conf Call #2

47. Operations Concept – Ideas, cont. • System needs full transparency for the forecaster – Summary views of inputs and model internals but with the capability to drill down and view detailed information • Essential for quality control and debugging • Smart tools – to assist in diagnosis of the ensemble inputs, data assimilation, and results DSST STEP Conf Call #2

48. Operations Concept – Ideas, cont. • Probabilistic hydrograph displays should include an overlay of historical ranges • Historical perspective of Model inputs • Selectable MODS – parameter to designate a MOD for deterministic, ensemble or both • Drawing tool to redraw forecasts – necessary for timeliness and expediency in problem cases DSST STEP Conf Call #2

49. Issues and Challenges • Hydraulic routings – complicates the ability to operate at the segment level • Model BIAS correction – error model • DA – ability to trace what it did and provide capability to undo or nudge differently • Reservoirs – ensembles on autopilot can introduce unrealistic results DSST STEP Conf Call #2

50. DSST STEP Conf Call #2