Tea Break!

1. Tea Break!

2. Coming up: • Fixing problems with expected improvement et al. • Noisy data • ‘Noisy’ deterministic data • Multi-fidelity expected improvement • Multi-objective expected improvement

3. Different parameter values have a big effect on expected improvement

4. The Fix

5. A one-stage approach combines the search of the model parameters with that of the infill criterion • Choose a goal value, g, of the objective function • The merit of sampling at a new point x is based on the likelihood of the observed data conditional on passing though x with function value g • At each x theta is chosen to maximize the conditional likelihood

6. g=-5

7. Avoiding underestimating the error • At a given x, Kriging predictor is most likely value • How much lower could the output be, e.g. how much error? • Approach: • Hypothesise that at x the function has a value y • Maximize the likelihood of the data (by varying theta) conditional on passing through the point x,y • Keep reducing y until the change in the likelihood is more than can be accepted by a likelihood ratio test • Difference between Kriging prediction and lowest value is measure of error, which is robust to poor theta estimation

8. Example • For limit=0.975, chi-squared critical = 5.0, lowest value fails likelihood ratio test

9. Use to compute a new one-stage error bound • Should provide better error estimates with sparse sampling/ deceptive functions • Will converge upon standard error estimate for well sampled problems

10. Comparison with standard error estimates

11. New one-stage expected improvement • One-stage error estimate embedded within usual expected improvement formulation • Now a constrained optimization problem with more dimensions (>2k+1) • All the usual benefits of expected improvement, but now better!?

12. EI using robust error estimate

13. EI using robust error:passive vibration isolating truss example

14. Difficult design landscape

15. Deceptive sample E[I(x)] E[I(x,yh,θ)]

16. Lucky sample E[I(x)] E[I(x,yh,θ)]

17. A Quicker Way

18. Problem is when theta is underestimated • Make one adjustment to theta, not at every point • Procedure • Maximize likelihood to find model parameters • Maximize the thetas subject to likelihood not degrading too much (based on likelihood ratio test) • Maximize EI using conservative thetas for standard error calculation

19. Truss problemLuck sample (top) deceptive sample (bottom)

20. 8 variable truss problem

21. 10 runs of 8 variable truss problem

22. Noisy Data

23. ‘Noisy’ data • Many data sets are corrupted by noise • In computational engineering, deterministic ‘noise’ • ‘Noise’ in aerofoil drag data due to discretization of Euler equations

24. Failure of interpolation based infill • Surrogate becomes excessively snaky • Error estimates increase • Search becomes too global

25. Regression, by adding constant λ to diagonal of correlation matrix, improves model

26. A few issues with error estimates • Interpolation error=0 at sample point: • at x=xi • But not for regression:

27. EI is no longer a global search

28. ‘Noisy’ Deterministic Data

29. Want ‘error’=0 at sample points • Answer is to ‘re-interpolate points from the regressing model • Equivalent to using in the interpolating error equation

30. Re-interpolation error estimate • Errors due to noise removed • Only modelling errors included

31. Now EI is global method again

32. Note of caution when calculating EI as:

33. Two variable aerofoil example • Same as missing data problem • Course mesh causes ‘noise’

34. Interpolation – very global

35. Re-interpolation – searches local basins, but finds global optimum

36. Multi-fidelity data

37. Can use partially converged CFD as low fidelity (tunable) model

38. Multi-level convergence wing optimization

39. Co-kriging • Expensive data modelled as scaled cheap data based process plus difference process • So, have covariance matrix:

40. One variable example

41. Multi-fidelity geometry example • 12 geometry variables • 10 full car RANS simulations 15h each • 120 rear wing only RANS simulations 1.5h each

42. Rear wing only Full car

43. Kriging models Visualisation of four most important variables Based on 20 full car simulations correct data, but not enough? Based on 120 rear wing simulations right trends, but incorrect data?

44. Co-Kriging, all data

45. Design improvement

46. Multi-objective EI

47. Pareto optimization • We want to identify a set of non-dominated solutions • These define the Pareto front • We can formulate an expectation of improvement on the current non-dominated solutions

48. Multi-dimensional Gaussian process • Consider a 2 objective problem • The random variables Y1 and Y2 have a 2D probability density function:

49. Probability of improving on one point • Need to integrate the 2D pdf:

50. Integrating under all non-dominated solutions: • The EI is the first moment of this integral about the Pareto front (see book)