Uncertainty Quantification & the PSUADE Software Mahmoud Khademi Supervisor: Prof. Ned Nedialkov

1. Uncertainty Quantification & the PSUADE Software Mahmoud Khademi Supervisor: Prof. Ned Nedialkov Department of Computing and Software McMaster University, Hamilton, Ontario Canada 2012

2. Outline • Introduction to Uncertainty Quantification (UQ) • Identification • Characterization • Propagation • Analysis • Common algorithms and methods • PSUADE: UQ software library and environment https://computation.llnl.gov/casc/uncertainty_quantification/ • Conclusions & future research directions

3. Introduction to UQ • Quantitative characterization and reduction of uncertainty • Estimating probability of certain outcomes when some aspects of system are unknown • Advances of simulation-based scientific discovery caused emergence of verification and validation (V&V) and UQ • Many problems in the natural sciences and engineering have uncertainty

4. Identification • Model structure: models are only approximation to reality • Numerical approximation: methods are not exact • Input and model parameters may only be known approximately • Variations in inputs and model parameters due to differences between instances of same object • Noise, measurement errors and lack of data

5. Characterization • Aleatoric (statistical) uncertainties: differ each time we run same experiment • Monte Carlo methods are used, probability density function (PDF) can be represented by its moments • Epistemic (systematic) uncertainties: due to things we could in principle know but don't in practice • Fuzzy logic or generalization of Bayes theory are used

6. Propagation • How uncertainty evolve? • Analyzing impact parameter uncertainties have on outputs • Finding major sources of uncertainties (sensitivity analysis) • Exploring “interesting” regions in parameter space (model exploration)

7. Analysis • Assessing "anomalous" regions in parameter space (risk analysis) • Creating integrity of a simulation model (validation) • Providing information on which additional physical experiments are needed to improve understanding of system (experimental guidance)

8. Selecting Proper Methods • Is there nonlinear relationship between uncertain and output variables? • Is uncertain parameter space high-dimensional? • There may be some model form uncertainties • How much is computational cost per simulation? • Which experimental data are available?

9. Monte Carlo Algorithms • Based on repeated random sampling to compute their results • Used when it is not feasible to compute an exact result with a deterministic algorithm • Useful for simulating systems with many degrees of freedom, e.g. cellular structures

10. Monte Carlo Method: Outline • Define a domain of possible inputs • Generate inputs randomly from a probability density function over domain • Perform a deterministic computation on inputs • Aggregate results

11. Polynomial Regression • Input data: • Unknown parameters: • ε: random error with mean zero conditioned on x

12. MARS • MARS (multivariate adaptive regression splines) is weighted sum of some bases functions: • Each basis is constant 1, hinge function or product of them as: or • Each step of forward pass finds pair of bases functions that gives maximum reduction in error • Backward pass prunes the model

13. MARS Versus Linear Regression

14. Principal Component Analysis • Consider a set of N points in n-dimensional space: • Principal Component Analysis (PCA) looks for n by m linear transformation matrix W mapping original n-dimensional space into an m-dimensional feature space, where m < n: • High variance is associated with more information

15. Principal Component Analysis • Scatter matrix of transformed feature vectors is: is scatter of input vectors & mean s • Projection is chosen to maximize determinant of total scatter matrix of projected samples: • are set of eigenvectors corresponding to m largest eigenvalues of scatter matrix of input vectors

16. PSUADE: How it works? • Input section allows the users to specify number of inputs, their names, their range, their distributions, etc. • Driver program can be in any language provided that it is executable. • Run PSUADE with: [Linux] psuadepsuade.in • At completion of runs, information will be displayed and data file will also be created for further analysis

17. PSUADE Capabilities • Can study first order sensitivities of individual input parameter (main effect) • Can construct a relationship between some input parameters to model & output (response surface modeling) • Can quantify impact of a subset of parameters on output (global sensitivity analysis) • Can identify subset of parameters accounting for output variability (parameter screening)

18. PSUADE Capabilities • Monte Carlo, quasi-Monte Carlo, Latin hypercube and variants, factorial, Morris method, Fourier Amplitude Sampling Test (FAST), etc • Simulator Execution Environment • Markov Chain Monte Carlo for parameter estimation and basic statistical analysis • Many different types of response surfaces • Many methods for main, second-order, and total-order effect analyses

22. Sensitivity Analysis

23. Sensitivity Analysis

24. Correlation Analysis

25. Main Effect Analysis

29. Future Research Directions • Resolving curse of dimensionality • Representation of uncertainty • Bayesian computation & machine learning techniques e.g. stochastic multi-scale systems for model selection , classification & decision making • Visualization in high-dimensional spaces