Kriging Metamodels for Simulation Optimization

1. Kriging Metamodels for Simulation Optimization Jack P.C. Kleijnen Department of Information Management /Center for Economic Research (CentER) Tilburg School of Economics & Management (TiSEM)Tilburg University, Tilburg, Netherlands NSF Simulation Optimization WorkshopUniversity of Maryland, College Park, May 24-25, 2010

2. Part 1: Expected Improvement (EI)/ Efficient Global Optimization (EGO) Classic reference: Jones, Schonlau, Welch (‘98)Deterministic simulation (CAE), Kriging Problem: Plug-in estimates for predictor variance Solution: Parametric bootstrappingOrigin: Den Hertog, Kleijnen, Siem (2006) Empirical results: four test functions (Hartmann)# dimensions: 1, 2, 3, 6Bootstrap: faster (first 3 functions) or tie Future research: constrained & random outputs

3. EI / EGO: details Local vs. global optimaExploration vs. exploitation; see Fu (2007) Expensive simulation: Kriging metamodel Deterministic vs. random simulation; see Frazier, Powell, Dayanik (2009), Ankenman, Nelson, Staum (2010) Predictor variance depends on unknown correlations θ(j) → estimate θ(j) σ²(x) = 0 if x = x(i) (i =1, …, n) (“old” data)

4. EI / EGO: more details EI / EGO algorithm for unconstrained minimization: • Find wo = min[w(i)] (i = 1, …, n) • EI(x) = E[wo - y(x) | y(x) < wo] with y(x) ~ N(y^, s2) • Find maximizer xo of EI(x) in candidate set • Simulate xo; refit Kriging; return to 1 until EI ≈ 0 Sub 2: Bootstrap estimator s2*: • Original I/O (X, w) gives original θ^ and μ^ • Sample (w*(1), …, w*(n + 1))’ from N(μ^,θ^) • Squared Error SE = [w*(n + 1) - y*{x(n + 1)}]² • Repeat 2 & 3, B times: s²* = ΣbSE(b) / B

5. Part 2: Robust optimization Taguchi’s worldview:Decision inputs (e.g., order quantity)Environmental inputs (e.g., demand rate) Example: EOQ Classic optimization: known demand rate a Robust optimization: a ~ N(μ, σ) Goal: minimize expected cost E(C)Constraint: keep standard dev. σ(C) below T

6. 4 x 10 8.92 8700 Regression Regression 8.9 1-L Kriging 1-L Kriging 8600 2-L Kriging 2-L Kriging 8.88 8500 8.86 8400 8.84 C C s 8.82 8300 8.8 8200 8.78 8100 8.76 8.74 8000 1.5 2 2.5 3 3.5 4 4.5 1.5 2 2.5 3 3.5 4 4.5 Q Q 4 4 x 10 x 10 Methodology for robust opt. 1. Design: CrossSpace-filling design for decision inputs dLHS for environmental inputs e with F(e) 2. Metamodel:a. Regress: y = β0 + β’d + d’Bd + γ’e + d’Δe + ε b. Two Kriging metamodels, for μ resp. σ

7. 4 x 10 8.92 Regression 8.9 1-L Kriging 2-L Kriging 8.88 8.86 8.84 C 8.82 8.8 8.78 8.76 8.74 8150 8200 8250 8300 8350 8400 s C Methodology continued 3. Min μ s.t. σ ≤ T : Mathematical Programming 4. Vary T: Pareto frontier 5. Quantify variability: Bootstrap metamodel Future research: Random (s, S) & constraints