Design Optimization Utilizing Gradient/Hessian Enhanced Surrogate Model

1. 28th, June, 2010, 28th AIAA Applied Aerodynamics Conference Design Optimization UtilizingGradient/Hessian Enhanced Surrogate Model Wataru YAMAZAKI, Markus P. RUMPFKEIL, Dimitri J. MAVRIPLIS Dept. of Mechanical Engineering, University of Wyoming, USA

2. Outline *Background - Efficient CFD Gradient/Hessian calculations - Surrogate Model Enhanced by Gradient/Hessian - Uncertainty Analysis *Objectives *Surrogate Model Approaches - Kriging - Direct and Indirect Gradient-enhanced Kriging - Gradient/Hessian-enhanced Kriging Approaches *Results & Discussion - Analytical Function Fitting - Aerodynamic Data Modeling - 2D Airfoil Drag Minimization - Uncertainty Analysis at Optimal Airfoil *Conclusions

3. Background~ Efficient CFD Hessian Calculation An efficient CFD Hessian calculation method by Adjoint method and Automatic Differentiation (AD) For steady flow Solutions for grid deformation / flow residual equations Adjoint solutions for flow / grid deformation equations Ndvlinear solutions each for dx/dDj and dw/dDj Ndv(Ndv+1)/2 cheap evaluations for each Hessian component M.P. Rumpfkeil and D.J. Mavriplis, AIAA-2010-1268 “Efficient Hessian Calculations using Automatic Differentiation and the Adjoint Method”

4. Background~ Efficient CFD Hessian Calculation ...... Flow Adjoint Mesh Adjoint dx/dD1 dw/dD1 dx/dD2 dw/dD2 dx/dDNdv dw/dDNdv An efficient CFD Hessian calculation method by Adjoint method and Automatic Differentiation (AD) Grid Deformation Flow Residual Gradient and Hessian M.P. Rumpfkeil and D.J. Mavriplis, AIAA-2010-1268 “Efficient Hessian Calculations using Automatic Differentiation and the Adjoint Method”

5. Background~ Approximate CFD Hessian For steady flow, a special form of objective function • Last approximation is accurate only nearly optimum • Approximate Hessian only requires the first-order derivatives M.P. Rumpfkeil and D.J. Mavriplis, AIAA-2010-1268 “Efficient Hessian Calculations using Automatic Differentiation and the Adjoint Method”

6. Background~ Uncertainty Analysis • Uncertainty due to manufacturing tolerances in-service wear-and-tear etc • Analysis of mean/variance/PDF of objective function w.r.t. fluctuation of design variables • Full Monte-Carlo Simulation • Thousands/Millions exact function calls • Accurate and easy, but computationally expensive • Moment Method • Taylor series expansion by grad/Hessian at the center • No information about PDF • Inexpensive Monte-Carlo Simulation • Thousands/Millions surrogate model function calls • Much lower computational cost

7. Objectives The efficient adjoint gradient/Hessian calculation methods will be effective… • for more efficient global design optimization with G/H-enhanced surrogate model approach • for more accurate and cheaper uncertainty analysis by inexpensive Monte-Carlo simulation with G/H-enhanced surrogate model Development of gradient/Hessian-enhanced surrogate models Application to design optimization and uncertainty analysis

8. Kriging, Gradient-enhanced Kriging 2D example : Real Sample Point : Additional Sample Point Kriging model approach - originally in geological statistics Two gradient-enhanced Kriging (cokriging or GEK) • Direct Cokriging Gradient information is included in the formulation (correlation between func-grad and grad-grad) • Indirect Cokriging Same formulation as original Kriging Additional samples are created by using the gradient info Kriging model by both real and additional pts

9. Gradient/Hessian-enhanced Kriging 2D example : Real Sample Point : Additional Sample Point Indirect Approach Arrangements to Use Full Hessian / Diagonal Terms Major parameters : distance between real / additional pts number of additional pts per real pt Worse matrix conditioning with smaller distance larger number of additional pts Severe tradeoffs for these parameters

10. Gradient/Hessian-enhanced Kriging Direct Approach Consider a random process model estimating a function value by a linear combination of function/gradient/Hessian components Minimizing Mean-Squared-Error (MSE) between exact/estimated function with an unbiasedness constraint Solving by using the Lagrange multiplier approach

11. Gradient/Hessian-enhanced Kriging Direct Approach Introducing correlation function for covariance terms Correlation is estimated by distance between two pts with radial basis function Unknown parameters are determined by the following system of equations Final form of the gradient/Hessian-enhanced direct Kriging approach is

12. Gradient/Hessian-enhanced Kriging Direct Approach Correlations between F-F, F-G, G-G, F-H, G-H and H-H Up to 4th order derivatives of correlation function Automatic Differentiation by TAPENADE No sensitive parameter Better matrix conditioning than indirect approach

13. Infill Sampling Criteria for Optimization • How to find promising location on surrogate model ? • Maximization of Expected Improvement (EI) value • Potential of being smaller than current minimum (optimal) • Consider both estimated function and uncertainty (RMSE)

14. Results & Discussion

15. 2D Rastrigin Function Fitting Exact Rastrigin Function Gradient-enhanced Gradient/Hessian-enhanced Function-based Kriging 80 samples by Latin Hypercube Sampling Direct Kriging approach

16. 5D Rosenbrock Function Fitting F: Function-based Kriging FG: Gradient-enhanced FGHd: G/diag. Hess-enhanced FGH: G/full Hess-enhanced RMSE .vs. Number of sample points Superiority in direct Kriging approaches thanks to exact enforcement of derivative information better conditioning of correlation matrix

17. Validation on Rosenbrock Func. CDFs of Full-MC and IMC Optimization History Minimization of 20D Rosenbrock 30 initial sample points by LHS EI-based infill sampling criteria Faster convergence in G/H-enhanced direct approach Uncertainty analysis on 2D Rosenbrock 5 sample points for surrogate model (No sample point on the center location) Superior performance in G/H-enhanced Inexpensive MC (IMC)

18. Aerodynamic Data Modeling Unstructured mesh CFD Steady inviscid flow, NACA0012 20,000 triangle elements Mach Number [0.5, 1.5] Angle of Attack[deg] [0.0, 5.0] 21x21=441 validation data Exact Hypersurface of Lift Coefficient Exact Hypersurface of Drag Coefficient

19. Aerodynamic Data Modeling Gradient-enhanced Exact Function-based Kriging Cl Cd Adjoint gradient is helpful to construct accurate surrogate model CFD Hessian is not helpful due to noisy design space

20. 2D Airfoil Shape Optimization Unstructured mesh CFD Steady inviscid flow, M=0.755 NACA0012, 16 DVs for Hicks-Henne function Objective function of inverse design form Exact / Approximate CFD Hessian available Computational time of F : 2 min, FG : 4 min, FGHapprox. : 6 min, FGHexact: 36 min (4 min in parallel) Geometrical constraint for sectional area

21. 2D Airfoil Shape Optimization Start from 16 initial sample points which only have function info Gradient/Hessian evaluations only for new optimal designs Faster convergence in derivative-enhanced surrogate model Best design in gradient/exact Hessian-enhanced model

22. 2D Airfoil Shape Optimization NACA0012 (Baseline) Optimal by G/exact H-enhanced model Towards supercritical airfoils Shock reduction on upper surface

23. 2D Airfoil Shape Uncertainty Analysis Geometrical uncertainty analysis at optimal airfoil Center = optimal obtained by Grad/exact H model Comparison between 2nd order Moment Method (MM2) using gradient/Hessian at the center Inexpensive Monte-Carlo (IMC1) using final surrogate model obtained in optimization Inexpensive Monte-Carlo (IMC2) using different G/H-enhanced model by 11 samples Full Non-Linear Monte-Carlo (NLMC) using 3,000 CFD function calls optimal (center) ±0.1 airfoil

24. 2D Airfoil Shape Uncertainty Analysis MM2 using derivative at the center IMC1 using G/H surrogate model obtained in optimization IMC2 using different G/H model by 11 samples (for st.devi.=0.01) NLMC using 3,000 CFD function calls Mean of objective w.r.t. standard deviation of all design variables IMC showed good agreement with NLMC at smaller st. devi. Necessity of additional sampling criteria for total model accuracy ? Promising IMC with much cheaper computational cost St. Devi. = 0.01 means the possibility within -0.01<dx<0.01 is about 70% Design Space = [0;1]

25. Concluding Remarks / Future Works Development of gradient/Hessian-enhanced Kriging models Application to design optimization and uncertainty analysis Direct Kriging approach is superior to indirect approach More accurate fitting on exact function Faster convergence towards global optimal design Promising inexpensive Monte-Carlo simulation at much lower cost Application to higher-dimensional / complicated design problem Robust design with inexpensive Monte-Carlo simulation Gradient/Hessian vector product-enhanced approach Thank you for your attention !!

26. Appendix

27. Moment Method Taylor series expansion by grad/Hessian at the center No information about PDF 1st order Moment Method 2nd order Moment Method

28. Gradient/Hessian-enhanced Kriging Implementation Details Correlation function of a RBF Estimation of hyper parameters by maximizing likelihood function with GA Correlation matrix inversion by Cholesky decomposition Search of new sample point location by maximizing Expected Improvement (EI) value with GA

29. Infill Sampling Criteria for Optimization • How to find promising location on surrogate model ? • Expected Improvement (EI) value • Potential of being smaller than current minimum (optimal) • Consider both estimated function and uncertainty (RMSE) EI-based criteria have good balance between global/local searching

30. 5D Rosenbrock Function Fitting # of pieces of information = sum of # of F/G/H net components To scatter samples is better than concentration at limited samples Approximated computational time factor G/H-enhanced surrogate model provides better performance with efficient Gradient/Hessian calculation methods

31. 1D Step Function Fitting Much better fit by G/H-enhanced direct Kriging

32. Minimization of 20D Rosenbrock Func. Minimization of 20 dimensional Rosenbrock function No computational cost for Func/Grad/Hess evaluation Expensive for construction - likelihood function maximization - inversion of correlation matrix Parallel computation for the likelihood maximization problem

33. Uncertainty Analysis Uncertainty analysis at (1.0,1.0) on 2D Rosenbrock 5 sample points for surrogate model approaches (No sample point on the center location) 2nd order Moment Method (MM2) by G/H at the center Superior results in G/H-enhanced Inexpensive MC (IMC) CDF at St. Devi.=0.15 St. Devi. = 0.15 means the possibility within -0.15<dx<0.15 is about 70%

34. Aerodynamic Data Modeling • NACA0012 • M=1.4 • AoA=3.5[deg] • Noisy in Mach number direction Cl Cd

35. 2D Airfoil Shape Uncertainty Analysis Cumulative Density Function at St. Devi. of 0.01 Quadratic model only by using gradient/Hessian at optimal Additional sampling criteria to increase total model accuracy