1 / 19

APPENDIX D R ANDOM N UMBER G ENERATION

Slides for Introduction to Stochastic Search and Optimization ( ISSO ) by J. C. Spall. APPENDIX D R ANDOM N UMBER G ENERATION. Organization of chapter in ISSO* General description and linear congruential generators Criteria for “good” random number generator

rianne
Download Presentation

APPENDIX D R ANDOM N UMBER G ENERATION

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Slides for Introduction to Stochastic Search and Optimization (ISSO)by J. C. Spall APPENDIX DRANDOM NUMBER GENERATION Organization of chapter in ISSO* General description and linear congruential generators Criteria for “good” random number generator Random variates with general distribution Different types of random number generators *Note: These slides cover some topics not included in ISSO

  2. Uniform Random Number Generators • Want a sequence of independent, identically distributed U(0, 1) random variables • However, random number generators (RNGs) produce a deterministic and periodic sequence of numbers • What qualities should the generators have?

  3. Criteria for ‘Good’ Random Number Generators • Long period • Good distribution of the points (low discrepancy) • Able to pass some statistical tests • Speed/efficiency • Portability – can be implemented easily using different languages and computers • Repeatability – should be able to generate the same sequence over again

  4. Generating Random Numbers • Given a transition function, f, the state at step n is given by • The output function, g, produces the outputs as • The output sequence is • Want the sequence period to be close to 2b, where b corresponds to the number of bits

  5. Types of Random Number Generators • Linear – most commonly used • Combined – can increase period and improve statistical properties • Non-linear – structure is less regular than linear generators but more difficult to implement

  6. Linear Congruential Generators • U(0,1) numbers via linear congruential generators (LCG) are calculated by • These are the most widely used and studied random number generators • The values a, c, and m should be carefully chosen

  7. Linear Congruential Generators • Some values for a and m (assuming c = 0) • a = 23, m = 108+1 (original implementation) • a = 65534, m = 229 (poor because of high order correlations) • a = 515, m = 247 (long period, good distribution, but lower order bits should not be trusted) • a = 16807, m = 231 –1 (this has been discussed as the minimum standard for RNGs)

  8. 0.58 0.56 0.54 m = 231 – 1, a = 4, c = 1 0.52 Empirical Mean 0.5 m = 482, a = 13, c = 14 0.48 m = 27, a = 26, c = 5 0.46 0.44 m = 9, a = 4, c = 1 0.42 0 500 1000 1500 2000 Number of Samples

  9. 30 points 96 points 1 1 0.8 0.8 Uk–-1 Uk–-1 0.6 0.6 0.4 0.4 0.2 0.2 0 0 0 0.2 0.4 0.6 0.8 1 0 0.2 0.4 0.6 0.8 1 Uk Uk Lattice Structure (Exercise D.2)

  10. Fibonacci Generators • These are generators where the current value is the sum (or difference, or XOR) or the two preceding elements • Lagged Fibonacci generators use two numbers earlier in the sequence

  11. Multiple Recursive Generators • Multiple recursive generators (MRGs)are defined by where the ai belong to {0,1,…,m – 1} and • For prime m and properly chosen ai’s, the maximal period is mk-1

  12. Combining Generators • Used to increase period length and improve statistical properties • Shuffling: uses the second generator to choose a random order for the numbers produced by the final generator • Bit mixing: combines the numbers in the two sequences using some logical or arithmetic operation (addition and subtraction are preferred)

  13. Nonlinear Generators • Nonlinearity can be introduced by using a linear transition function with a nonlinear output function • An example is the explicit inversive generator where

  14. Random Number Generators Used in Common Software Packages • Important to understand the types of generators used in statistical software packages and their limitations • MATLAB: • Versions earlier than 5: a linear congruential generator with • Versions 5 & 6: a lagged Fibonacci generator combined with a shift register random integer generator with period • EXCEL: un = fractional part (9821×un –1+ 0.211327); period • SAS (v6): LCG with period

  15. Inverse-Transform Method for Generating Non-U(0,1) Random Numbers • Let F(x) be the distribution function of X • Define the inverse function of F by • Generate X by • Example: exponential distribution

  16. AcceptReject Method • Let pX(x) be the density function of X • Find a function f(x) that majorizes pX(x) • , q is a density function • Generate X by • Generate U from U(0,1) (*) • Generate Y from q(y), independent of U • If , then set X=Y. Otherwise, go back to (*) • Probability of acceptance (efficiency) = 1/c • Related to Markov chain Monte Carlo (MCMC) method (see Exercise 16.4)

  17. 2.5 2.0 pX(x) 1.5 q(x) = U(0,1) 1.0 0.5 0 0.2 0.4 1.0 0.6 0.8 1.2

  18. U ~ U(0,1): 0.9501, 0.2311, 0.6068, 0.4860, 0.8913,  Y ~ q(y) U(0,1): 0.7621, 0.4565, 0.0185, 0.8214, 0.4447,  : 0.7249, 0.8131,  X ~ PX(x): 0.7621, 0.4565,  accept reject

  19. References for Further Study • L’Ecuyer, P. (1998), “Random Number Generation,”in Handbook of Simulation: Principles, Methodology, Advances, Applications, and Practice (J. Banks, ed.), Wiley, New York, Chapter 4. • Neiderreiter, H. (1992), Random Number Generation and Quasi-Monte Carlo Methods, SIAM, Philadelphia.

More Related