Problems with parameterization (example:keeper usage): average duration: 1.27, min: 0.106, max: 6.46

1. Assignment 4 Problems with parameterization (example:keeper usage): average duration: 1.27, min: 0.106, max: 6.46 possible outcome for keeper and crane queues?

2. For validation, simulate long enough.

3. Half widths Half width determination by Arena: statistical analysis of samples. Arena help file: Half Width (Runtime Confidence Intervals—Within a Replication) Some sections contain a column called "Half Width". This statistic is included to help you determine the reliability of the results from your replication. Three results are possible in the "Half Width" category: Insufficient: The formula used to calculate half width requires the samples to be normally distributed. That assumption may be violated if there is a small number (fewer than 320) of samples. If that is the case, Arena will return the message "Insufficient" for that variable’s half width, indicating there is insufficient data to accurately calculate the half width. Running the simulation for a longer period of time should correct this. Correllated: The formula used to calculate half width requires the samples to be independently distributed. Data that is correlated (the value of one observation strongly influences the value of the next observation) results in an invalid confidence interval calculation. If data is determined to be correlated, the message "Correllated" is returned for that variable’s half width. Running the simulation for a longer period of time should correct this. A value: If a value is returned in the Half Width category, this value may be interpreted by saying "in 95% of repeated trials, the sample mean would be reported as within the interval sample mean ± half width".

4. Half width calculation Arena has to be trusted w.r.t. confidence values. Computations "inside Arena" not clear! - determine average and variance s^2 - test for insufficient/correlated - half width = C.s for some constant C. ? Concepts from probability theory and statistics. Homework: Appendices B,C,D of lecture notes.

5. Randomization Random generator is in fact deterministic! Replaying same model gives same result. Still, it has all characteristics of "true" random generator (no "fairness"). DCT case: 15% increase BF trucks should diminish keeper queues. Short simulation: keeper queues might get longer! (see half width) Increase confidence in simulation. Divide simulation run into subruns. Add initial run (move away from initial state).

6. Replications Make sure that replications/subruns are independent (e.g. no queue length dependencies). When done right, division into subruns allows to computeconfidence intervals. Example computation: page 30 of lecture notes. n = 30 subruns, each with sample of measure x (occupation rate).

7. 1 0.914 11 0.894 21 0.898 2 0.964 12 0.962 22 0.912 3 0.934 13 0.973 23 0.943 4 0.978 14 0.984 24 0.953 5 0.912 15 0.923 25 0.923 6 0.956 16 0.932 26 0.914 7 0.958 17 0.967 27 0.923 8 0.934 18 0.924 28 0.936 9 0.978 19 0.945 29 0.945 10 0.976 20 0.936 30 0.934 Sample average/stddev: x= 0.9408, s = 0.02485. how do you compute s? a - confidence interval:

8. Confidence interval matching Computed 0.95 - confidence interval should match Arena's half width. Arena half width should (for large n) approximate This example: 1.96(0.02485 / 5.51)=0.00451 So with 95% probability, occ.rate in [0.936,0.945] cf. Chapter 6 of lecture notes

9. a - confidence interval: Function z: normal distribution surface (table lookup). Consequences: - You can be 99.99% confident, but not 100%. - Four times longer simulation halves confidence int. - high variance = low confidence

10. Comparisons Many simulation studies (e.g. DCT example) are about relative shortage of resources, leading to queues. Compare possible solutions through simulation. Simulation yields to following reports: Number waiting Average Half Width res1.Queue 4.18 1.20 Sol1: Number waiting Average Half Width res1.Queue 4.58 1.39 Sol2: Sol1 better?

11. Number waiting Average Half Width res1.Queue 4.18 1.20 Number waiting Average Half Width res1.Queue 4.58 1.39 Longer simulation needed to get following result: Number waiting Average Half Width res1.Queue 4.11 0.40 Number waiting Average Half Width res1.Queue 4.69 0.48

12. Variance and confidence Large half widths caused by high subrun variance, require very long simulations for acceptable confidence. For instance, compare 1 0.934 11 0.924 21 0.928 2 0.944 12 0.932 22 0.932 3 0.936 13 0.933 23 0.943 4 0.958 14 0.944 24 0.953 5 0.932 15 0.923 25 0.933 6 0.946 16 0.932 26 0.944 7 0.938 17 0.947 27 0.933 8 0.934 18 0.934 28 0.936 9 0.948 19 0.945 29 0.945 10 0.936 20 0.936 30 0.934 1 0.914 11 0.894 21 0.898 2 0.964 12 0.962 22 0.912 3 0.934 13 0.973 23 0.943 4 0.978 14 0.984 24 0.953 5 0.912 15 0.923 25 0.923 6 0.956 16 0.932 26 0.914 7 0.958 17 0.967 27 0.923 8 0.934 18 0.924 28 0.936 9 0.978 19 0.945 29 0.945 10 0.976 20 0.936 30 0.934 First samples: less variance, higher confidence, shorter simulation run

13. Subrun result depends on sequence of random numbers Variance reduction 1 Large half widths caused by high subrun variance, require very long simulations for acceptable confidence. Various techniques to reduce subrun variance, e.g. antithetic random numbers (see lecture notes) Next subrun: use antithetic sequence: Frequent arrivals, long processing times in a subrun compensated by infrequent arrivals and short times in next subrun.

14. Variance reduction 2 Second approach: sharing of e.g. arrival patterns. Less risk of adopting inferior solution with fewer arrivals. Both approaches introduce dependency in subruns. A third approach: compensate for the number of entities. sorted combined subruns sr mq #ent 1 0.934 3721 2 0.944 3696 3 0.936 3712 4 0.958 3754 5 0.932 3688 6 0.946 3718 7 0.938 3702 8 0.934 3694 9 0.948 3751 10 0.936 3734 sr mq #ent 5 0.932 3688 8 0.934 3694 2 0.944 3696 7 0.938 3702 3 0.936 3712 6 0.946 3718 1 0.934 3721 10 0.936 3734 9 0.948 3751 4 0.958 3754 sr mq #ent 5,4 0.945 7442 8,9 0.941 7445 2,10 0.940 7430 7,1 0.936 7423 3,6 0.941 7439

15. Sensitivity analysis Simulation model based on assumptions; both modeling and parameters. Assess dependency of simulation result on assumptions. Simulate with modified parameters and compare. Sensitive parameters / uncertain assumptions: show various outcomes.