Download
1 / 9
90 likes | 211 Views
Calibration Guidelines. Model development. Model testing. 9. Evaluate model fit 10. Evaluate optimal parameter values 11. Identify new data to improve parameter estimates 12. Identify new data to improve predictions 13. Use deterministic methods 14. Use statistical methods.
E N D
Calibration Guidelines Model development Model testing 9.Evaluate model fit 10.Evaluate optimal parameter values 11. Identify new data to improve parameter estimates 12. Identify new data to improve predictions 13.Use deterministic methods 14.Use statistical methods 1.Start simple, add complexity carefully 2. Use a broad range of information 3. Be well-posed & be comprehensive 4. Include diverse observation data for ‘best fit’ 5.Use prior information carefully 6. Assign weights that reflect ‘observation’ error 7.Encourage convergence by making the model more accurate 8. Consider alternative models Potential new data Prediction uncertainty
Model DevelopmentGuideline 7:Encourage convergence of the regression by making the model more accurate
Nonlinear regression can be difficult Even when composite scaled sensitivities andcorrelation coefficientsindicate the data provide sufficient information to estimate the defined parameters, nonlinear regression may not converge. Substantial insight about the observations, potential model inaccuracies, and model fit can be obtained from values calculated in failed regressions: dimensionless scaled sensitivities composite scaled sensitivities correlation coefficients influence statistics weighted and unweighted residuals parameter updates during the regression. Learn from Failed Regressions!
Working to make the model represent the system more accurately obviously is beneficial to model development, and generally also improves the behavior of the regression. Use the information from failed regressions, or regressions that converge to unrealistic parameter values, to guide changes. The most advantageous modifications for improving model accuracy are usually: Modify the parameter definition Modify other aspects of model construction, such as its physical features or the processes it simulates Estimate fewer parameters Add observations to the regression Scrutinize existing observations for error in interpretation Make the model more accurate
Insensitivity Nonlinearity Inconsistencies The Main Problems that Plague Convergence…
Insensitivity Specify parameters with small css Combine existing parameters Redesign the parameterization Creatively use system information (Guideline 2) Nonlinearity Evaluate results from intermediate iterations Evaluate large weighted residuals, observations omitted because simulated equivalents could not be obtained, whether parameter values are realistic If forward model nonlinearities are a suspected cause of nonconvergence, consider using a linear approximation Inconsistencies Check parameter representation, dominant observations Evaluate observations & prior, and their simulated equivalents …and Possible Solutions to Consider
Calibration Guidelines Model development Model testing 9.Evaluate model fit 10.Evaluate optimal parameter values 11. Identify new data to improve parameter estimates 12. Identify new data to improve predictions 13.Use deterministic methods 14.Use statistical methods 1.Start simple, add complexity carefully 2. Use a broad range of information 3. Be well-posed & be comprehensive 4. Include diverse observation data for ‘best fit’ 5.Use prior information carefully 6. Assign weights that reflect ‘observation’ error 7. Encourage convergence by making the model more accurate 8.Consider alternative models (MMA) Potential new data Prediction uncertainty
Developing alternative models Deterministic methods Alternative conceptual models about depositional environment Alternative theories about rainfall distribution and(or) infiltration dynamics Stochastic methods Alternative realizations of gravel/sand/clay distribution developed using indicator kriging Combined methods Discard some alternative realizations based on deterministic depositional theories Generate stochastic variations within a deterministically derived hydrogeologic framework Guideline 8:Consider alternative modelsbook p. 308-314
Better models have three attributes: • Better fit (But not too good!) • Weighted residualsthat are more randomly distributed • More realistic optimal parameter values Guideline 8:Consider alternative models This graph shows model discrimination criteria for 5 models of the Maggia Valley, southern Switzerland. SSWR: Sum of squared, weighted residuals. AICc, BIC: Model discrimination criteria. Foglia, in press, GW. Book, p. 311.
