1 / 20

Output Data Analysis

Output Data Analysis. How to analyze simulation data?. simulation computer based statistical sampling experiment estimates are just particular realizations of random variables that may have large variances n independent replications each replication terminated by same event

kasie
Download Presentation

Output Data Analysis

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Output Data Analysis

  2. How to analyze simulation data? • simulation • computer based statistical sampling experiment • estimates are just particular realizations of random variables that may have large variances • n independent replications • each replication terminated by same event • started with same initial conditions • replications are independent by means of using different random variables • single measure of performance one per replication 040669 || WS 2008 || Dr. Verena Schmid || PR KFK PM/SCM/TL Praktikum Simulation I

  3. obtained random numbers • Y1, Y2, … Ym • is an output stochastic process from a single run • generally neither independent nor identically distributed • most formulas assuming IIDs not directly applicable • y11, y12, … y1m • realizations for random variables Y1, Y2, … Ym • resulting from making a simulation run of length m observations • y21, y22, … , y2m • realizations for random variables Y1, Y2, … Ym • if simulation is run again (using different random variables) 040669 || WS 2008 || Dr. Verena Schmid || PR KFK PM/SCM/TL Praktikum Simulation I

  4. obtained random numbers (cont) 040669 || WS 2008 || Dr. Verena Schmid || PR KFK PM/SCM/TL Praktikum Simulation I • if you make n independent replications (runs) • with different random number used • observations from particular run/rownot IID • observations from form ithcolumn are IID observations of random variable Yi (i = 1..m) ! independence across runs y11, y12, … y1i, …. y1m y21, y22, …. y2i, …. y2m … …. …. yn1, yn2, … yni, ….. ynm

  5. Transient and Steady-State Behavior • stochastic output process Y1, Y2, .. • transient condition: Fi( y | I ) = P(Yi· y | I) for i = 1, 2… • y is a real number • I represents initial conditions • densityfYi • specifies how random variable Yi can vary from one replication to another • Fi(y | I ) ! F(y) as i!1 • F(y) steady-state distribution of output process Y1, Y2, … • in theory only obtained at limit • in practice ! finite time index (k+1) ! distributions will be approximately the same 040669 || WS 2008 || Dr. Verena Schmid || PR KFK PM/SCM/TL Praktikum Simulation I

  6. Transient and Steady-State Behavior (cont.) 040669 || WS 2008 || Dr. Verena Schmid || PR KFK PM/SCM/TL Praktikum Simulation I

  7. Types of Simulations we’ll focus on this type only • terminating simulation • non-terminating simulations • steady-state parameters • steady-state cycle parameters • other parameters 040669 || WS 2008 || Dr. Verena Schmid || PR KFK PM/SCM/TL Praktikum Simulation I

  8. Example • bank • 5 tellers, one queue • opens at 9:00 • closes at 17:00 (stays open until all customers in the bank have been served) • terminating simulation • close at/after17:00 (as soon as all customers have left) 040669 || WS 2008 || Dr. Verena Schmid || PR KFK PM/SCM/TL Praktikum Simulation I

  9. Example (cont.) 040669 || WS 2008 || Dr. Verena Schmid || PR KFK PM/SCM/TL Praktikum Simulation I

  10. Estimating Means • point estimate and confidence interval for mean ¹ = E(X) • unbiased point estimator for ¹ • approximate 100(1-®) percent confidence interval for ¹ 040669 || WS 2008 || Dr. Verena Schmid || PR KFK PM/SCM/TL Praktikum Simulation I

  11. Estimating Means (example) • estimate expected delay • = 2.031 • S2(n) = 0.309 • confidence interval with ® = 10% • estimated proportion of customers being delayed < 5 minutes • expected proportion for a given day/run • indicator function • = 0.853 S2(n) = 0.0039 • CI with ® = 10% 040669 || WS 2008 || Dr. Verena Schmid || PR KFK PM/SCM/TL Praktikum Simulation I

  12. Obtaining a desired precision • so far • fixed sample size procedure (based on n replications) • disadvantage: no control over the CI’s half length (i.e. precision of ) • half length depends on population variance S2(n) • 2 ways to measure the error in the estimate • absolute error ¯ • relative error ° • resulting number of replications may be random 040669 || WS 2008 || Dr. Verena Schmid || PR KFK PM/SCM/TL Praktikum Simulation I

  13. Obtaining a desired precision (absolute error ¯) • absolute error ¯ • estimator has an absolute error of at most ¯ with a probability of approximately 1 - ® • approximate expression for total number of replications na*(¯) required to obtain an absolute error of ¯ • assumes that estimate S2(n) will not change (appreciately) as n increases) • na*(¯) will be determined iteratively 040669 || WS 2008 || Dr. Verena Schmid || PR KFK PM/SCM/TL Praktikum Simulation I

  14. Obtaining a desired precision (absolute error ¯) · 0.25 • example (bank) • Q: what’s the number of replications necessary in order to estimate the expected average delay with an absolute error of 0.25 minutes and a confidence level of 90%? 040669 || WS 2008 || Dr. Verena Schmid || PR KFK PM/SCM/TL Praktikum Simulation I

  15. Obtaining a desired precision (relative error °) • relative error ° • estimator as a relative error of at most °/(1 - °) with a probability of approximately 1 - ®. • approximate expression for total number of replications na*(¯) required to obtain a relative error of ° • assumes that estimate S2(n) will not change (appreciately) as n increases) • nr*(°) will be determined iteratively 040669 || WS 2008 || Dr. Verena Schmid || PR KFK PM/SCM/TL Praktikum Simulation I

  16. Obtaining a desired precision (relative error °) · 0.0909 • example (bank) • Q: what’s the number of replications necessary in order to estimate the expected average delay with a relative error of 10% and a confidence level of 90%? 040669 || WS 2008 || Dr. Verena Schmid || PR KFK PM/SCM/TL Praktikum Simulation I

  17. Estimating other Measures of Performance • be careful! • comparing two systems by some sort of mean may result in misleading conclusions • example: 2 bank policies • 5 queues (one in front of every teller) • 1 queue (that feeds all tellers) 040669 || WS 2008 || Dr. Verena Schmid || PR KFK PM/SCM/TL Praktikum Simulation I

  18. Estimating other Measures of Performance still identical? Estimates of expected proportions of delays in interval 040669 || WS 2008 || Dr. Verena Schmid || PR KFK PM/SCM/TL Praktikum Simulation I

  19. Choosing initial conditions • careful! • measures of performance depend explicitly on the state of the system at time 0 • take care when choosing appropriate initial conditions • example: estimate expected average delay at bank between noon and 1pm • bank will probably be quite congested at noon • starting with no customers present -> estimates will be biased low 040669 || WS 2008 || Dr. Verena Schmid || PR KFK PM/SCM/TL Praktikum Simulation I

  20. Choosing initial conditions • careful! • measures of performance depend explicitly on the state of the system at time 0 • take care when choosing appropriate initial conditions • 2 heuristic approaches • use warmup period • collect data to get an idea of state of system and choose it randomly 040669 || WS 2008 || Dr. Verena Schmid || PR KFK PM/SCM/TL Praktikum Simulation I

More Related