Design of experiment for computer simulations

1. Design of experiment for computer simulations • Let X = (X1,…,Xp)  Rp denote the vector of input values chosen for the computer program • Each Xj is continuously adjustable between a lower and an upper limit, or 0 and 1 after transformation • Let Y = (Y1,…,Yq)  Rq denote the vector of q output quantities • Y = f(X), X [0,1]p • Important considerations: • The number of input p • The number of output q • The speed with which f can be computed • They are deterministic, not stochastic • Why a statistical approach is called for?

2. Design of experiment for computer simulations • Conventional one-factor-at-a-time approach • It may miss good combinations of X because it doesn’t fully explore the design space. • It is slow, especially when p is large • It may be misleading when interactions among the components of X are strong • Randomness is required in order to generate probability or confidence intervals • Introducing randomness by modeling the function f as a realization of a Gaussian process • Introducing randomness by taking random input points

3. Goals in computer experiments • Optimization • Standard optimization methods (e.g. quasi-Newton or conjugate gradients) can be unsatisfactory for computer experiments as they usually require first and possibly second derivatives of f • Standard methods also depend strongly on having good starting values • Computer experimentation is useful in the early stages of optimization where one is searching for a suitable starting value, and for searching for several widely separated regions for the predictor space that might all have good Y values

4. Goals in computer experiments • Visualization – being able to compute a function f at any given X doesn’t necessarily imply that one “understands” the function • Computer simulation results can be used to help identify strong dependencies • Approximation • If the original program f is exceedingly expensive to evaluate, it may be approximated by some very simple function , holding adequately in a region of interest, though not necessarily over the entire domain of f • Optimization may be done using large number of runs of the simple function

5. Approaches to computer experiments • There are two main statistical approaches to computer experiments • One is based on Bayesian statistics • Another is a frequentist one based on sampling techniques • It is essential to introduce randomness in both approaches • Frequentist approach • For a scalar function Y = f(X), consider a regression model • Y = f(X)  Z(X)b • The coefficients b can be determined by least squares method with respect to some distribution F on [0,1]p • bLS = (Z(X)’Z(X)dF)-1Z(X)’f(X)dF • The quality of the approximation may be assessed globally by the integrated mean squared error • (Y – Z(X) b)2dF

6. Frequentist experimental design • Assume the region of interest is the unit cube [0,1]p, p = 5 • Grids (choose k different values for each of X1 through Xp and run all kp combinations) – works well but completely impractical when p is large. In situations where one of the responses Yk depends very strongly on only one or two of the inputs Xj the grid design leads to much wasteful duplication

7. Frequentist experimental design • Good lattice points (based on number theory)

8. Frequentist experimental design • Latin hypercubes

9. Frequentist experimental design • Randomized orthogonal arrays

10. Example – critical specimen size study

11. 19 mm (ANSI/AWS) 25 mm (MIL) 35 mm (ISO) 19 mm (ANSI/AWS) 45 mm (ISO) 25 mm (MIL) 105 mm (ISO) 102 mm (MIL) 76 mm (ANSI/AWS) Specimen size requirements for tensile shear tests of 0.8 mm gauge steel sheets. Wcritical = f(t, h; E, sy, s0, e; k)

12. Peak Load P E Energy Maximum Displacement D

13. 3.60 3.55 3.50 3.45 3.40 3.35 3.30 3.25 Peak Load (kN)

14. Run t (mm) v1 h (mm) t (mm) v2 E (GPa) h (mm) y(MPa) v3 E (MPa) 0 (MPa) v4 k e (%) v5 sy (MPa) k v6 e (%) v7 suts (MPa) Wcritical (mm) 0.5~2.0 0.1~1.5 190~200 205~1725 50~200 2~65 1.0~3.0 1 1 0.54 10 0.88 12 196.76 14 2.59 5 607.35 2 11 15 785.29 22.6 2 4 0.81 5 0.47 2 190.88 1 1.06 8 875.59 6 25 7 982.94 27.5 3 3 0.72 12 1.05 8 194.41 11 2.24 16 1590.88 5 22 1 1645.29 37.3 4 7 1.07 2 0.22 13 197.35 3 1.29 12 1233.24 8 32 14 1402.36 31.7 5 8 1.16 1 0.14 15 198.53 13 2.47 14 1412.06 11 42 6 1510.59 33.8 6 5 0.90 14 1.21 7 193.82 8 1.88 17 1680.29 15 56 16 1867.05 38.7 7 2 0.63 11 0.96 14 197.94 6 1.65 3 428.53 17 63 5 518.24 19.1 8 6 0.99 3 0.31 1 190.29 16 2.82 7 786.18 14 53 10 920.00 27.1 9 9 1.25 9 0.80 9 195.00 9 2.00 9 965.00 9 35 9 1090.00 40.4 10 12 1.51 15 1.29 17 199.71 2 1.18 11 1143.82 4 18 8 1260.00 44.1 11 16 1.87 7 0.64 4 192.06 12 2.35 15 1501.47 1 8 13 1661.76 58.2 12 13 1.60 4 0.39 11 196.18 10 2.12 1 249.71 3 15 2 312.95 24.2 13 10 1.34 17 1.46 3 191.47 5 1.53 4 517.94 7 29 12 669.41 35.7 14 11 1.43 16 1.38 5 192.65 15 2.71 6 696.76 10 39 4 777.64 39.2 15 15 1.78 6 0.55 10 195.59 7 1.76 2 339.12 13 49 17 534.71 35.0 16 14 1.69 13 1.13 16 199.12 17 2.94 10 1054.41 12 46 11 1197.06 49.6 17 17 1.96 8 0.72 6 193.24 4 1.41 13 1322.65 16 60 3 1394.71 58.9 Table 1. Ranges selected for computer simulation. Table 2. Design matrix and simulation results.

15. Coupon Size Determination Simulation • A two level full factorial would require 27 = 128 runs • In the computer experiment, N levels of each variable can be chosen (based on the number of variables n). N is also the total number of runs needed. N = 17 for seven (7) variables in the example • The computer simulation results are used to create the dependence of critical specimen size on the variables by Kriging regression method

16. Wcritical,1 = 13.4044613 + 18.5987839 t • Wcritical,2 = -6.0291481 + 18.5839362 t + 0.0146654 y + 6.6251147 h • Wcritical,3 = 45.6391799 + 18.5849834 t + 0.0146654 y + 21.8791238 h + 28.3945601 e + 0.0811080 (uts - y) - 0.0003401 E - 9.5332611 h2 - 0.2280655 e (uts - y)