1 / 21

Variance Reduction Techniques

Variance Reduction Techniques. Outline. Importance of Variance Reduction Types of Variance Reduction Techniques Common Random Numbers Example: Common Random Numbers Implementing Common Random Numbers in Arena Antithetic Variates Implementing Antithetic Variates in Arena Control Variates.

jana
Download Presentation

Variance Reduction Techniques

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Variance Reduction Techniques

  2. Outline • Importance of Variance Reduction • Types of Variance Reduction Techniques • Common Random Numbers • Example: Common Random Numbers • Implementing Common Random Numbers in Arena • Antithetic Variates • Implementing Antithetic Variates in Arena • Control Variates

  3. Reasons for Importance • Associated with an estimate of a performance measure (e.g., mean time in the emergency room for a patient). • The variance is a measure of the amount of uncertainty in an estimate -- the smaller the variance, the less the uncertainty. • The smaller the variance, the easier it is to distinguish between/among (i.e., rank, or select the best) alternative systems/policies. • To reduce the variance of an estimate, one could: 1. Make more replications. 2. Use one of several variance reduction techniques. (The latter is the preferred approach, especially if computational time is a constraint).

  4. Types of Variance Reduction Techniques 1. Common Random Numbers (CRN) 2. Antithetic Variates (AV) 3. Control Variates (CV) 4. Others: Indirect Estimation, Conditioning

  5. Common Random Numbers • Applies only when comparing two or more alternative systems • Most Popular VRT • Basic idea: compare alternative system configurations using “similar experimental conditions” (i.e., the “same” set of random numbers) • A form of “blocking” • Also called correlated sampling or matched streams • “Synchronization” of random numbers across different alternatives is important • A pilot study may be appropriate, since “backfiring” is a possibility • Formal statistical analysis can be made complicated with CRN

  6. Justification for the Use of Common Random Numbers • Let Xij be the observation for the jth replication of the ith alternative (i=1, 2; j=1, 2, ..., n). • If the two sets of replications are done independently, then Cov(X1j, X2j)=0, but if positive correlation is induced with CRN, then Cov(X1j, X2j) >0.

  7. Common Random Numbers • “Synchronization” is an important aspect in the use of CRN. For example, you would want to dedicate particular streams of random numbers for particular purposes (e.g., for interarrival times) for each alternative.

  8. Example:Common Random Numbers • Suppose we have a queuing situation in which two alternative system configurations are being considered: Alternative 1: 1 server, with a processing time that is exponentially distributed, with a mean of .9 minute. Alternative 2: 2 servers, each with a processing time that is exponentially distributed, with a mean of 1.8 minute. • Assume that the interarrival time for customers is exponentially distributed, with a mean of 27 seconds, and the queuing discipline is FIFO. • Suppose that the performance measure of interest is the average time in the queue of the first 100 customers. (This model will be called the ATM model).

  9. Example: Common Random Numbers (continued) Running each of these models for 10 replications with no variance reduction technique gave the following results for average time in the queue for the first 100 customers. j X1j X2jZj = X1j- X2j _____________________________________________________________ 1 5.04 1.02 4.02 2 6.10 3.03 3.07 3 6.26 1.85 : 4 1.77 2.81 5 2.43 3.40 6 3.20 1.25 7 11.96 2.63 8 4.00 2.87 9 12.58 3.71 10 11.97 2.18 Sample Mean 6.53 2.48 4.06 Sample Variance 17.2 .78 16.56 95% CI Half-Width 2.97 .63 2.91

  10. Example: Common Random Numbers (continued) Now, implementing Common Random Numbers, through the use of the SEEDS module (from the Elements panel), we obtain the following output for average time in the queue for the first 100 customers. (See the following pages for how this implementation was accomplished). j X1j X2jZj = X1j- X2j _____________________________________________________________ 1 5.71 5.17 .54 2 2.70 2.29 .41 3 .53 .41 .12 4 5.78 5.64 .14 5 8.77 7.82 : 6 2.45 2.01 7 4.18 3.46 8 .83 .65 9 5.17 5.00 10 1.65 1.38 Sample Mean 3.78 3.38 .395 Sample Variance 6.81 6.00 .076 95% CI Half-Width 1.87 1.75 .197

  11. Example: Common Random Numbers • Note how the sample variance for the difference was decreased from 16.56 to 0.076 ( a decrease of 99.5%) and how the half-width for the 95% confidence interval for the difference between the two alternatives was decreased from 2.91 to 0.197 ( a decrease of 93%). • In summary, we have: 95% CI without CRN: (1.15, 6.97) 95% CI with CRN: (0.319, 0.592)

  12. Implementing Common Random Numbers in Arena • By default, Arena uses stream 10 to generate all of its random numbers for a series of simulation replications. This is not what you want for CRN. • Use the SEEDS module from the Elements panel to implement CRN. This allows us to name and dedicate different streams for different purposes in a model • Using the Common selection for the Initialize Option spaces the seeds for each replication within each stream 100,000 random numbers apart, so that there will be no overlap of random number usage within a stream across replications. • Generally, do not use stream 10 when using this approach. • The SEEDS module allows you to dedicate the streams 1,2,… in series.

  13. Implementing Common Random Numbers in Arena • Each location in the model which involves generation of random variates needs to be modified. For example, instead of EXPO(5) for interarrival time in a Arrive module, one would use EXPO(5, Stream Interarrivals) where Stream Interarrivals is defined as a particular stream in the SEEDS module. • To implement CRN, the SEEDS module should be employed in all alternative model runs.

  14. Example: Implementing CRN in the Arena model • For each of the two alternative models, 1) attach the ELEMENTS panel, and place the SEEDS module in the model window. 2) Name and dedicate streams for Interarrival Times and Processing Times, by adding the elements: i) for stream 1: Identifier: Interarrival Times Stream Seed Value: (default) Initialize Option: Common ii) for stream 2: Identifier: Processing Times Stream Seed Value: (default) Initialize Option: Common 3) In the Arrive module, replace the EXPO(1) in the Time Between field with EXPO(1, Interarrival Time Stream)

  15. Example: Implementing CRN in the Arena model 4) In the Server module, replace the Process Time in the single server model with EXPO(0.9, Processing Time Stream) and in the two server model with EXPO(1.8, Processing Time Stream). 5) Save the observations for the average value of the time in queue for each model to one filefor the single server model and to another file for the two-server model, by modifying the Outputs section of the Statistics module.

  16. Example: Implementing CRN in the Arena model • After running each of the models, we can use the output analyzer to compare the alternatives: 1) Select Tools/Output Analyzer. 2) Select File/New. 3) Add Data from Files 4) Select Analyze/Compare Means. 5) Select Data File A Replications: Lumped Data File B Replications: Lumped • We obtain the following results for a 95% paired-t confidence interval: Std Dev = 0.275 CI: [0.198, 0.592] Reject Ho: means are not equal at the 0.05 level.

  17. Antithetic Variates (AV) • Useful in reducing the variance of an estimate for a single alternative. • Basic idea: make pairs of runs for a single alternative so that a “small” observation for one run is offset by a large observation for the second run of the pair. • Operationally, rn’s for run 1 of a pair: r1, r2, r3, … rn’s for run 2 of a pair: 1-r1, 1-r2, 1-r3,… • Note that both streams are i.i.d., u(0,1) rn’s, hence everything is “valid” with respect to each run. • Synchronization must be used.

  18. Implementing Antithetic Variates in Arena • This is much like implementing CRN in Arena, the only difference is that you need to specify Antithetic in the Initialize Option field if the SEEDS module. This will give you Antithetic pairs in your replications. • For example, implementing the antithetic option in the single-server ATM model used for the CRN example would give us the following results for average queuing time (the AV model is contained in the file av1).

  19. Implementing Antithetic Variates in Arena ReplicationNo Variance ReductionAntithetic Variates 1 5.04 6.83 2 6.10 3.15 3 6.26 7.16 4 1.77 2.51 5 2.43 3.41 6 3.20 2.68 7 11.96 0.58 8 4.00 7.25 9 12.58 2.43 10 11.97 1.99 Sample Mean 6.53 3.8 Sample Variance 17.2 5.7 95% CI Half Width: 2.97 1.71 • Note how the sample variance was reduced by 67% (from 17.2 to 5.7) and how the confidence interval half-length was reduced by 42% (from 2.97 to 1.71) by using antithetic variates.

  20. Control Variates • The basic idea is to take advantage of correlation between certain random variates, in order to reduce the variance of the output. For example, suppose we have a queuing system with: -- Interarrival Time Mean (specified) = 3 min. -- Interarrival Time Mean (actual, from model run) = 3.21 min. -- Mean Time in system(estimated, from model) = 3.4 min. • Since the mean interarrival time from the model (3.21) is greater than it should be (3.), one might expect that the sample mean time in the system (3.4 minutes) is less than it should be. We should adjust this time (3.4 minutes) upwards; the question is, by how much?

  21. Choosing Control Variates • Important factors to consider include: • The correlation between the control variate and x should be high, • The control variate itself should have a low variance.

More Related