160 likes | 402 Views
Sensitivity Analysis in GEM-SA. Jeremy Oakley. Example. ForestETP vegetation model 7 input parameters 120 model runs Objective: conduct a variance-based sensitivity analysis to identify which uncertain inputs are driving the output uncertainty. Exploratory scatter plots.
E N D
Sensitivity Analysis in GEM-SA Jeremy Oakley
Example • ForestETP vegetation model • 7 input parameters • 120 model runs • Objective: conduct a variance-based sensitivity analysis to identify which uncertain inputs are driving the output uncertainty.
Sensitivity Analysis Walkthrough • Project New • Select the Files tab. Click on Browse on the Inputs File row • GEM-SA Demo Data / Model1 / emulator7x120inputs.txt • Click on Browse on the Outputs File row • GEM-SA Demo Data / Model1 / out11.txt • Return to the Options tab
Sensitivity Analysis Walkthrough • Change the Number of Inputs to 7. • Tick the calculate main effects and sum effects boxes only • Leave the other options unchanged • Input uncertainty options: All unknown, uniform • Prior mean options: Linear term for each input • Generate predictions as: function realisations (correlated points) • Click OK • Project Run
Main effect plots Fixing X6 = 18, this point shows the expected value of the output (obtained by averaging over all other inputs). Simply fixing all the other inputs at their central values and comparing X6=10 with X6=40 would underestimate the influence of this input (The thickness of the band shows emulator uncertainty)
Variance of main effects Main effects for each input. Input 6 has the greatest individual contribution to the variance Main effects sum to 66% of the total variance
Interactions and total effects • Main effects explain 2/3 of the variance • Model must contain interactions • Any input can have small main effect, but large interaction effect, so overall still an ‘important’ input • Can ask GEM-SA to compute all pair-wise interaction effects • 435 in total for a 30 input model – can take some time! • Useful to know what to look for
Interactions and total effects • For each input Xi Total effect = main effect for Xi + all interactions involving Xi • Total effect >> main effect implies interactions in the model • NB main effects normalised by variance, total effects normalised by sum of total effects • Look for large total effects relative to main effects
Interactions and total effects Total effects for inputs 4 and 7 much larger than its main effect. Implies presence of interactions
Interaction effects • Project Edit • Tick calculate joint effects • De-select all inputs under inputs to include in joint effects, select 4,5,6,7 • Click OK • Project Run
Interaction effects Note interactions involving inputs 4 and 7 Main effects and selected interactions now sum to 91% of the total variance
Exercise • Set up a new project using SAex1_inputs.txt for the inputs and SAex1_outputs.txt for the output • 8 input parameters (uniform on [0,1]) • 100 model runs • Estimate the main effects only for this model and identify the influential input variables • By comparing main effects with total effects, can you spot any interactions? • Estimate any suspected interactions to test your intuition!