Model uncertainty

1. Model uncertainty How uncertain are your modelling results?

2. This Session (#19): Session 19 divides into two sections: • Overview of uncertainty analysis aims & objectives • Analysis of uncertainty in breach modelling • Findings and conclusions for breach modelling Mark Morris ~ 25 mins • Analysis of uncertainty in flood propagation • Findings and conclusions for propagation modelling • Overall conclusions & observations • Q&A Francisco Alcrudo ~ 25+10 mins 4th IMPACT Workshop - Zaragoza

3. IMPACT: Modelling uncertainty: Breach Formation [WP2] Flood Propagation [WP3] Sediment Movement [WP4] Geophysics / Field Data [WP6] Assessment of modelling uncertainty Implications for End User Applications [WP5] 4th IMPACT Workshop - Zaragoza

4. Why do this? • Dambreak modelling requires a high degree of modelling expertise / experience. Uncertainties in the modelling process are higher than for ‘normal’ river modelling work. • Flood risk management may be performed more effectively if additional information is given supporting ‘best estimates’. 4th IMPACT Workshop - Zaragoza

5. Aims & objectives: • Investigate uncertainty within modelling predictions for breach, flood propagation and sediment transport • Demonstrate how uncertainty within each of these modelling domains contributes towards overall uncertainty in predicting final conditions (e.g. water levels at specific locations) • Consider the implications of the magnitude of uncertainty found in terms of use by ‘end users’ 4th IMPACT Workshop - Zaragoza

6. Scope of work under IMPACT The scope of work under IMPACT: • does not allow for an investigation of uncertainty in the impact of flooding or the assessment / management of flood risk • does not allow for development of extensive, statistical analysis techniques [Simple, practical assessment of likely order of magnitude of uncertainty] 4th IMPACT Workshop - Zaragoza

7. Progress to date (!) • Understanding the problem • Development of approach • Recognition of limitations for sediment modelling • Selection of case studies • Implementation for breach modelling • Implementation for propagation modelling • Analysis and conclusions     .... ? 4th IMPACT Workshop - Zaragoza

8. Development of approach... • A slow process that ended with a simple practicable but perhaps non rigorous approach balancing the need to analyse the problem with limitations in time and budget (the real world!) • The main arguments: • Fully rigorous statistically based analysis model by model and also linking of models • Removal of subjective - expert judgement type stages • Simple, practicable approach allowing indicative answer within acceptable time / budgets • Natural  4th IMPACT Workshop - Zaragoza

9. Approach taken... Sediments: • Recognition that current ability to model sediment movement under dambreak / extreme flood conditions was limited • Progress has been made (Yves Zech / Sandra Soares) but not sufficient to allow assessment of uncertainty 4th IMPACT Workshop - Zaragoza

10. (Simple) Overview of approach 1 Adopt the concept of using upper, mid and lower estimates of modelling results parameter (e.g. flood hydrograph, water level) 2 Assess uncertainty in breach model leading to upper / mid / lower flood hydrographs 3 Assess uncertainty in propagation models using 3 flood hydrographs from breach modelling as input conditions (leading to nine propagation predictions) 4 Select upper, mid, lower estimates for presentation of results to end user 4th IMPACT Workshop - Zaragoza

11. Some issues • Need to balance expert judgement (subjectivity): • What does upper, mid, lower mean? • What model parameters do you vary to create these? • Different models use different parameters - how do you compare these? • Run time and flexibility of models dictates approach: • Differences between similar model types (rel small) • Differences between breach and propagation (significant) 4th IMPACT Workshop - Zaragoza

12. Selection of case studies: • Very difficult to find extensive, reliable data sets… • Ongoing problem…forensics team? • Two case studies selected: • Tous Dam Failure (breach & propagation) • Lake Ha!Ha! (sediments) 4th IMPACT Workshop - Zaragoza

13. Uncertainty in breach modelling Basic process: • Sensitivity analysis to modelling parameters • Selection of top 3-5 parameters; identification of realistic parameter range and distribution • Monte Carlo analysis • Review and selection of upper, mid & lower hydrographs • Modelling undertaken by 3 organisations / 3 different models • Selection of overall upper, mid, lower 4th IMPACT Workshop - Zaragoza

14. Modelling with HR BREACH • 1D flow model - piping and overtopping • Soil erosion through shear stress / sediment transport; no predefinition of breach growth • Integrated soil mechanics for lateral and longitudinal discreet slope failure (undercutting etc) • Surface protection simulation • Composite structure and associated failure mechanisms • Variable sediment equations, adjustable probability distribution for slope failure (uncertainty in soils), • Monte Carlo facility 4th IMPACT Workshop - Zaragoza

15. Sensitivity analysis 4th IMPACT Workshop - Zaragoza

16. Failure distribution Monte Carlo simulations • 5 parameters varied for MC analysis • 1600 MC runs undertaken • < than statistical requirement • little difference in results from 3 parameter MC analysis No failure - only overtopping 4th IMPACT Workshop - Zaragoza

17. All upper-mid-lower results 4th IMPACT Workshop - Zaragoza

18. Modelling with Rupro (Cemagref) • Aim: providing an outflow hydrograph from piping or overtopping • Simplified assumptions: average breach cross-section, sediment transport using Meyer Peter Müller equation with sediment described by only one representative diameter and one friction coefficient • Advantages: • very rapid calculation, few parameters to test • can be integrated in 1-D and 2-D propagation models thus cascade breaching or dike breaching easy to calculate • Disadvantages: difficulty to assess parameters in case of complex structure of the dam 4th IMPACT Workshop - Zaragoza

19. Parameter considered Values of parameter Peak outflow (m3/s) Time of peak outflow Breach bottom elevation (m) 53 / 70 20300 / 15300 21 : 36 / 20 : 53 Breach Strickler friction coefficient 26.5 / 20 17070 / 20300 24 : 19 / 21 : 36 sediment diameter (mm) 250 / 0.08 6196 / 16700 21 : 51 / 25 : 02 Maximum breach width not limited / 36 m 24924 / 13600 18 : 45 /19 : 33 Modelling with Rupro (Cemagref) Uncertainty calculations • Step 1: what are the parameters to which outflow hydrograph is sensitive? • Step 2: select the three main parameters and 5 values for each (very low, low, average, high, very high) to which a probability is associated 4th IMPACT Workshop - Zaragoza

20. Modelling with Rupro (Cemagref) Uncertainty calculations • Step 3: calculation for all combinations (125) and ranking hydrographs by peak outflows to obtain 5, 10, 50, 90 and 95 % occurrences. 4th IMPACT Workshop - Zaragoza

21. Breach Model DEICH_P Breach Formation 4th IMPACT Workshop - Zaragoza

22. Breach Model DEICH_P Breach Formation without core 4th IMPACT Workshop - Zaragoza

23. Breach Model DEICH_P Breach Formation with core 4th IMPACT Workshop - Zaragoza

24. Selected upper-mid-lower results 4th IMPACT Workshop - Zaragoza

25. Some observations… • Qp varies between +50% (~22,500m3/s) and -17% (~12,500m3/s) • BUT, what don’t forget that non failed data has been removed • Timing influenced by inflow data, but upper-lower still shows ~2.75hr difference 4th IMPACT Workshop - Zaragoza

26. Some observations… • How do our best estimate results compare to mid values? • Mid  Best estimate 4th IMPACT Workshop - Zaragoza

27. Some observations… • Choice of sediment transport equation worth a closer look… • analysis was prompted by a bug in our MC routine - looking for way to model in homogeneous rather than composite form • considered 2 sediment transport equations (cohesive / non cohesive); core and outer layer material parameters • highlights the importance of selecting the right sediment relationship • highlights the effect of assuming a homogeneous bank 4th IMPACT Workshop - Zaragoza

28. Some conclusions (breach uncertainty) • Uncertainty range for Qp for Tous study was ~ +50% -17% • Note - this includes uncertainty from 3 models / modellers. Range from 1 model / modeller will appear less. • Modeller best estimate was better than ‘mid’ estimate (for this case). • Skill of modeller will play a significant part • Routine use of mid value not necessarily the best 4th IMPACT Workshop - Zaragoza

29. Some conclusions (breach uncertainty) • Understanding the sensitivity of models to different parameters is essential • different models will include different parameters • model sensitivity to parameters will vary from model to model • All breach models have one or more parameters relating distribution of shear stress / sediment / erosion / breach growth. • This parameter is usually hidden but has a major influence on results • Care is required to validate particular parameter values for each application 4th IMPACT Workshop - Zaragoza

30. Some conclusions (breach uncertainty) • Application of Monte Carlo approach is ‘better’ than sensitivity analysis, but the process requires a quick running model for practicality • Current breach models are on the limit of practicability for MC analysis • Current breach models are not well designed to predict the timing of breach initiation • current models are very sensitive to boundary conditions (ie defined flow, water level etc) rather than actually predicting the onset of breach 4th IMPACT Workshop - Zaragoza

31. Some conclusions (breach uncertainty) • Breach model results are highly dependent upon choice of sediment transport equation • No existing equations are ideal (steady state, long term…) • selection of most appropriate equation for case application should be done (look at conditions that equation was developed for) 4th IMPACT Workshop - Zaragoza

32. Some conclusions (breach uncertainty) • Simulation of composite or homogeneous structures and assumptions of averaged soil properties can significantly affect results • Variations in Qp of >300% can be seen when different assumptions / sed equations are made (compared to +- 50% for earlier uncertainty range) • Therefore use appropriate model (composite or homogeneous) & appropriate equation. Be wary of assuming homogeneous for a composite structure... 4th IMPACT Workshop - Zaragoza

33. Beyond IMPACT… • Awaiting DSIG work with interest • FLOODsite - Task 20 “Development of framework for the influence and impact of uncertainty” “…develop an approach and prototype software for propagating uncertainty through integrated flood risk models…” • University Bristol (UK) & IHE (Delft) under theme lead from HR - > 60 mm research • www.floodsite.net ---------- 4th IMPACT Workshop - Zaragoza