ZHAO Linna

1. The 4th THORPEX-Asia Science Workshop & 9th ARC Meeting ARC Meeting ARC Meeting Probabilistic flood prediction by TIGGE on upriver of HUAIhe catchment ZHAO Linna Zhao Linna, WU Hao, Liang Li, DI Jingyue, WANG Zhi Public Weather Service Center, China Meteorological Administration Qi Dan , TIAN Fuyou National Meteorological Center, China Meteorological Administration Zhao Linna Public Weather Service Center, China Meteorological Administration 30 October – 3 November 2012, Kunming, China

2. Contents Introduction The data and the test catchment The Experiment of Probabilistic flood prediction The Calibration of Precipitation CMA-EPS and Hydrological Experiments Outlook

3. 1. Introduction Numerical weather forecasts without uncertainties specified are hard to be incorporated into operations and decision making of other downstream applications such as early warning of floods and rainfall induced geological hazards. The ensemble forecasting technique provides an effective way to quantify those uncertainties. The uncertainty of ensemble forecasting can be expressed in terms of forecast probability density function (PDF), based upon which probabilistic forecast is generated. To make full use of all the information available in an ensemble forecast, Bayesian model averaging (BMA) was introduced by Raftery et al.(2005) as a statistical post-processing method for producing probabilistic forecast from an ensemble.

4. 1. Introduction By BMA, the overall forecast PDF of any variable of interest is a weighted average of forecast PDFs based on each of the individual forecasts, where the weights are estimated by posterior probabilities of the models generating the forecasts and reflect the relative forecasting skills of the individual models in the training period. The BMA weights can be used to assess the usefulness of ensemble members, and this can be used as a basis for selecting ensemble members, this can be useful given the cost of running large ensembles. BMA offers the added advantage, by giving a full predictive PDF, of being able to give probabilities of exceeding arbitrary precipitation amounts, rather than having to create a new logistic regression model for each threshold of interest.

5. 1. Introduction The main objective of this work is thus to give a demonstration for users who need to build or to give probabilistic hydro-meteorological forecasts with TIGGE database. In this respect, the results are felt to extend beyond the Dapoling-Wangjiaba basin of China itself and to apply to more similar test catchment.

6. Contents Introduction The data and the test catchment The Experiment of Probabilistic flood prediction The Calibration of Precipitation CMA-EPS and Hydrological Experiments Outlook

7. The ensemble systems used in this work Totally 87 members

8. Test catchment of Dapoling-Wangjiaba 2Data and test area • The Dapoling-Wangjiaba catchment is the origination of the Huaihe River Basin • Sub-catchment locates at the upper reach of Huaihe River Basin • Coverage of about 30,630 km2 (is 16% of Huaihe basin) • Altitude ranging from 200 to 500 meters • Daily Precipitation observations from June 1, 2008 — August 31, 2008 • 19 rain gauges in the test catchment • The hourly accumulative rain gauge data • Hydrologic data is the daily flow of Xixian and Wangjiaba hydrological site in upper stream of Huai river 8 8

9. Contents Introduction The data and the test catchment The Experiment of Probabilistic flood prediction The Calibration of Precipitation CMA-EPS and Hydrological Experiments Outlook

10. Flood event: 23 Jul.-3 Aug. 2008 flood over the upriver of the Huaihe basin The hydrological characteristics of the main hydrological control stations on the upriver have important demonstrative significance. we selected the Huaihe basin as a research area, especially the upriver of the basin. The flood forecasting at Xixian and Wangjiaba hydrological stations with runoff records are investigated. The areal precipitation is obtained by averaging the records of 19 observations or simulated precipitation values. 3. The Experiment of Probabilistic flood prediction by TIGGE database

11. Nash-Sutcliffe coefficient is used to assess the predictive power of hydrological models. It is defined as: 3. The Experiment of Probabilistic flood prediction by TIGGE database : Methodology • Essentially, the closer the model efficiency is to 1, the more accurate the model is. • where Qo is observed discharge, and Qm is modeled discharge. Qot is observed discharge at time t. • Nash–Sutcliffe efficiencies can range from −∞ to 1. An efficiency of 1 (E = 1) corresponds to a perfect match of modeled discharge to the observed data. An efficiency of 0 (E = 0) indicates that the model predictions are as accurate as the mean of the observed data, whereas an efficiency less than zero (E < 0) occurs when the observed mean is a better predictor than the model or, in other words, when the residual variance (described by the numerator in the expression above), is larger than the data variance (described by the denominator). from Nash, J. E. and J. V. Sutcliffe (1970), Journal of Hydrology

12. 3. The Experiment of Probabilistic flood prediction by TIGGE database : Methodology TIGGE-CMA TIGGE-NCEP TIGGE-ECMWF VIC hydrological model Confluence Model Runoff Export section flow Hydrological probabilistic forecast

13. Results _River discharge prediction day-3 Xi Xian • The NCEP-EPS and CMA-EPS can bracket half of the observed discharges. • The 5th–99th percentile distribution of the EC-EPS is large and nearly brackets all discharge observations during the period 23 July–3 August 2008 except for the flood ascending period on 24 and 25 July • The performance of the EC-EPS is the best among the three systems, which is consistent with the precipitation forecast results of the EPSs. • The performance of the Grand-EPS is equal or better than that of EC-EPS. • Both the single EC-EPS and the Grand-EPS can bracket most of the observations between 5th and 99th quantile. 13 13

14. Results _Comparison of the three EPSs and the grand ensemble between 5th,25th,50th,75th,95th and 99th percentile for the 3-day lead time runoff forecasts day-3 Xi Xian • All of the EPSs did not obviously provide indicative significance for the Xixian flooding process, especially in the first rising limb and the next recession limb, because most of the predictions occurred in the extreme area between 95th and 99th quantile. • Generally, the main referenced inter-zone between 25th and 75th quantile is usually considered as a credible range in the EPS. • However, it is clear that the forecast performance in the second rising limb on 1 August 2008 is better than the first rising limb for all of the EPSs. 14 14

15. Wangjiaba station is the outlet of the upper Huaihe River catchment. All of the EPSs predict the flood in good agreement with the observed discharge, which falls in the 5th–99th quantile except the NCEP-EPS Results _River discharge prediction day-3 Wang Jiaba 15

16. Results _Comparison of the three EPSs and the grand ensemble between 5th,25th,50th,75th,95th and 99th percentile for the 3-day lead time runoff forecasts day-3 Wang Jiaba • The result is different from the discharge prediction at Xixian station in that the twice rising discharge thresholdsat Wangjiaba station are captured nicely and the predictions of discharge are all nearly in the 25th–75th quantile • It is a valuable reference for decision-making, but one deficiency is that the prediction in the recession limb is generally quicker than the observation. • For the three EPSs, the NCEP-EPSpredicted discharge is generally smallerthan the observation, while the EC-EPS and the CMA-EPS produce better predictions of discharge in the study period at Wangjiaba station, which provides helpful probabilistic hydrological forecasts for decision-making.

17. Evaluation of the EPSs at Xixian and Wangjiaba stations Characteristics of Nash-Sutcliffe efficiency coefficient of Wangjiaba and Xixian • The greatest NS value appears in the prediction of the EC-EPS, which is 0.97 • for Wangiiaba station and 0.96 for Xixian station. • The lowest NS value -1.47 appears in the prediction of the NCEP-EPS. For • Wangjiaba station • The average value of the Nash-Sutcliffe index in the EC-EPS is0.68, which is • greater than that of other EPSs.

18. Ensemble forecasts of precipitation The CMA-EPS largely underestimated the precipitation amount while the NCEP-EPSmissed many precipitation events especially at Xixian. For most of the forecast days, the EC-EPS produced precipitation forecasts within the range of 5th–95th percentile, which are closest to the observation, so it outperformed the other two EPSs. The Grand-EPS production is the best since almost half of the forecasts are within the range of 25th–75th percentile. These results are in accordance with the results of the river discharge predictions. The performance of the ensemble precipitation forecasts plays an important role in the forecasts of the river discharge. Huaibin CMA Xi Xian CMA Huaibin NCEP Xi Xian NCEP Huaibin ECMWF Xi XianECMWF Huaibin Grand Xi XianGrand

19. Another reason is … The different performances of the EPSs at the two stations may be caused by the differences in the geographic location and topographical distribution. • Xixian station is located in the upriver of the entire basin, where the topography is fairly complex • so there will be no enough time for this station to respond to the heavy rainfall. • But Wangjiaba station, located at the exit cross-section of the catchment, has sufficient time to respond to the flood process

20. Contents Introduction The data and the test catchment The Experiment of Probabilistic flood prediction The Calibration of Precipitation CMA-EPS and Hydrological Experiments Outlook

21. 4. The Calibration of Precipitation Ensemble Forecast Using Bayesian Model Averaging and Hydrological Experiments TIGGE-CMA TIGGE-NCEP TIGGE-ECMWF VIC hydrological model Confluence Model Runoff Export section flow Hydrological probabilistic forecast

22. 4. The Calibration of Precipitation Ensemble Forecast Using Bayesian Model Averaging and Hydrological Experiments • The Calibration of Precipitation Ensemble Forecast on TIGGE-CMA is investigated • VIC (Variable Infiltration Capacity) model is a hydrological model based on spatial distribution grid (Wood, 1992; Liang Xu, etc., 2003) • For research purpose, conveniently • All date are interpolated in 15km×15km spatial resolution

23. 4. The Calibration of Precipitation Ensemble Forecast Using Bayesian Model Averaging and Hydrological Experiments • BMA（Bayesian Model Averaging）is a statistical way of post-processing forecast ensembles to create predictive probability density functions (PDFs) for weather quantities. • It represents the predictive PDF as a weighted average of PDFs centered on the individual bias-corrected forecasts, where the weights are posterior probabilities of the models generating the forecasts and reflect the forecasts’ relative contributions to predictive skill over a training period. • BMA model not only can provide the biggest forecast possibility, but also described the weather forecast uncertainty. • Daily Precipitation observations from June 1 — August 31, 2008 • Verification: • CRPS (continuous ranked probability score )& MAE(mean absolute error, The smaller the score is, the better the calibration result is • The 25th, 75 percentile of hydrological probabilistic forecast are verified by three index, namely certainty coefficient, relative error of flood peak, difference of peak time

24. The contrast of MAE and CRPS score of 24h, 48h and 72h forecast between the original ensemble forecast (CMA) and bayesian average model ensemble forecast (BMA) • Known from the CRPS and MAE, the 24h forecast of BMA model obviously has a better deviation correction effect compared to the CMA ensemble forecast. • The calibration of 48h or 72h forecast also performs well. 24

25. 4 Results (a) The box plot of BMA model and that of before for stations at Xixian before BMA BMA 8/10 obs. are captured 5/10 obs. are captured Ensemble mean is the mean value of 15 ensemble members, so that it can be taken as deterministic forecast. Although its forecast value is probably more close to the true value compared with some individual member, the deviation between them is still obvious. The BMA forecast, the majority of the dates are in the valid interval 25

26. 4 Results (b) (a) The box plot of BMA model and that of before for stations at Wangjiaba BMA 1/10 obs. are captured 7/10 obs. are captured Before BMA The valid interval given by the BMA model contains the observations. However, the valid interval given by CMA ensemble forecast contains few observations relatively. The valid interval forecast given by the BMA model is more likely to contain the true value of observations. Its forecast capacity is better than the deterministic forecast. 26

27. 4 Results The comparison chart between simulated discharges by BMA precipitation forecast and observed flow of Xixian hydrologic station Xixian 48h Xixian 24h Xixian 72h The results of discharge simulation of 24h, 48h and 72h precipitation forecast from 25th to 75th percentile by the BMA model include the possibility of the flood peak occurs and recedes. 27

28. 4 Results The comparison chart between simulated discharges by BMA precipitation forecast and observed flow of Wangjiaba hydrologic station Wangjiaba 48h Wangjiaba 24h Wangjiaba 72h Probability forecast which forced the hydrological model has contained uncertainty component, so it is very difficult to simulate ideal deterministic runoff by using probabilistic results. But the runoff got by probabilistic forecast is relatively effective to capture the trend of the runoff. 28

29. 4 Results Statistics of each verify index of Hydrological probabilistic forecast • Xixian • Certainty coefficient: -2.65~0.7 at 24,48 & 72hr • Relative error of peak: -0.41~-0.36 • Wangjiaba • Certainty coefficient: 0.34~0.82 at 24,48 & 72hr • Relative error of peak: 0.03~0.26

30. 5. Outlook For the predictions of flood discharge by the three single EPSs, the performance of the EC-EPS is the best, while the NCEP-EPS is the worst, and the Grand-EPS is better than any of the single EPS. The result of hydrological simulation which forced by the calibrated precipitation is almost consistent with the runoff tendency of observations. Grand-EPS produces more reliable predictions of a flooding event and therefore brings significantly valuable results for the operational flood forecasting and warning service.

31. 5. Outlook This work gives an encouraging indication that a multi-model ensemble can provide more valuable probability forecasts than a deterministic prediction for extreme flood events. As the probabilistic hydrological forecasting is a hot topic in the hydro-meteorological science, this study provides a good example for how to use the TIGGE achieve data to drive the hydrological model for probabilistic flood forecasts. Probabilistic hydrological forecasting is foreseen as inevitable in the development of hydrological forecasting in future.

32. Thanks for your attention! Ref: Zhao Linna, QI Dan, Tian Fuyou, et al., 2012: Probabilistic flood prediction in the upper Huaihe catchment using TIGGE data. Acta meteor. Sinica,26(1), 62-71, doi: 10.1007/s13351-012-0106-3.  : zhaoln@cma.gov.cn

33. discussions The 24h forecast of BMA model obviously has a better performance compared to the CMA ensemble forecast. The calibration of 48h or 72h forecast also performs well. The robustness of Bayesian method needs more comprehensive research. Such as BMA model parameter estimation process has over-estimated risks. How to determine the deformation index of fk in the logistic regression model, and how to check the correlation between the index and fk . All of these need continued study. Due to the limitation of the time and space resolution of meteorological data, the verify index of hydrological forecast results can't be meticulously analyzed which need collect high-resolution data for further study.

34. This may explain why it performs well for forecasts at high thresholds where the amount of training data is small. This suggests that BMA may be useful both for routine precipitation forecasting and for forecasting the risk of extreme precipitation events. In addition, the application of hydrological model includes many uncertainty factors, such as input, model parameters, structures and so on. In this paper, only the uncertainty of precipitation forecast input is discussed.

35. Weight of BMA member on 22 July

36. 3. Methodology percentile The time lasts from 1 July to 6 August, 2008, for totally 37 days. All precipitation intensities were taken into consideration with the percentile precipitation evaluation for examining the ability of forecasting the extreme rainfall events an 95% large 75% ak 50% (median) 25% small 5% a1 36 36

37. 4 Results The forecast of 15 members of CMA model are integrated as probabilistic forecast to contrast with that of BMA model The box plot for the CMA modeltake the maximum of 15 members for the maximum, the minimum of 15 members for the minimum the 25th and 75th percentile for box plot The box plot for the BMA modeltake five statistics , namely 95 percentile for the maximum, 5th percentile for the minimum , the 25th and 75th percentile Maximum forecast of original model (CMA) 75% 25% 25% 75% 5% 95% BMA valid interval Minimum forecast of original model (CMA) 37

38. Areal percentile precipitation An established percentile method presented by Hyndman and Fan (1996) is adopted for the areal percentile precipitation. The equation is given as 3. Methodology • where j = int (p × n+(1+p)/3) and γ = p × n + (1 + p)/3 − j, • p is the percentile, Qi(p) is the percentile areal precipitation, A • is the array of the forecasted areal precipitation in ascending • order, and n is the number of ensemble members. • The areal precipitation is obtained by averaging the records of 19 observations or simulated precipitation values. 38 38

39. Nash-Sutcliffe coefficient is used to assess the predictive power of hydrological models. It is defined as: 3. Methodology • Essentially, the closer the model efficiency is to 1, the more accurate the model is. • where Qo is observed discharge, and Qm is modeled discharge. Qot is observed discharge at time t. • Nash–Sutcliffe efficiencies can range from −∞ to 1. An efficiency of 1 (E = 1) corresponds to a perfect match of modeled discharge to the observed data. An efficiency of 0 (E = 0) indicates that the model predictions are as accurate as the mean of the observed data, whereas an efficiency less than zero (E < 0) occurs when the observed mean is a better predictor than the model or, in other words, when the residual variance (described by the numerator in the expression above), is larger than the data variance (described by the denominator). from Nash, J. E. and J. V. Sutcliffe (1970), Journal of Hydrology 39 39

40. 2 Methodology BMA methods • BMA（Bayesian Model Averaging）is a statistical way of post-processing forecast ensembles to create predictive probability density functions (PDFs) for weather quantities. • It represents the predictive PDF as a weighted average of PDFs centered on the individual bias-corrected forecasts, where the weights are posterior probabilities of the models generating the forecasts and reflect the forecasts’ relative contributions to predictive skill over a training period. • BMA model not only can provide the biggest forecast possibility, but also described the weather forecast uncertainty. 40 40

41. 2 Methodology BMA methods PDF： In BMA for ensemble forecasting, each ensemble member forecast fk is associated with a conditional PDF k(y| fk), which can be thought of as the PDF of the weather quantity y given fk , conditional on fk being the best forecast in the ensemble. and 41 41

42. 2 Methodology BMA methods The model for hk(y| fk) is in two parts. The first part specifies PoP as a function of the forecast fk. using logistic regression with a power transformation of the forecast as a predictor variable using the cube root of the forecast as a predictor variable. The second part of the model specifies the PDF of the amount of precipitation given that it is not zero . Here fit gamma distributions to precipitation amounts. 42 42

43. 2 Methodology • Parameter estimation of BMA model 43 43

44. 2 Methodology Parameter estimation of BMA model Parameter estimation is based on data from a training period, here take 30 days of forecast and verifying observation data preceding initialization. The training period is a sliding window, and the parameters are re-estimated for each new initialization period. 44 44

45. 2 Methodology Selection of hydrological model and Verification 1220 grids 15km×15km • VIC (Variable Infiltration Capacity) model is a hydrological model based on spatial distribution grid (Wood, 1992; Liang Xu, etc., 2003) • Its output are evaporation, soil moisture content, runoff depth and so on. • The spatial resolution of VIC is 15km×15km 45

46. 2 Methodology Verification of precipitation probabilistic forecast • CRPS (continuous ranked probability score) ：观测和预报的累积分布函数（CDF）的差别 • MAE(mean absolute error)：是能反映预报误差的指标 The smaller the score is, the better the calibration result is 46

47. 2 Methodology Verification of precipitation probabilistic forecast • CRPS (continuous ranked probability score) ：观测和预报的累积分布函数（CDF）的差别 • MAE(mean absolute error)：是能反映预报误差的指标 The smaller the score is, the better the calibration result is 47

48. 2 Methodology Selection of hydrological model and Verification Certainty coefficient • The 25th, 75 percentile of hydrological probabilistic forecast are verified by three index, namely • certainty coefficient • relative error of flood peak • difference of peak time 48