Random Number Generation

1. Random Number Generation • Plan: • Introduce basics of RN generation • Define concepts and terminology • Introduce RNG methods • Linear Congruential Generator (LCG) • Explicit Inversive Congruential Generator • Non-linear Additive Feedback • Transform-Expand-Sample (TES) • Technical issues for RNGs • Testing RNGs

2. Motivation • Many simulations rely on use of random number generators (RNG) • Want to assess the “typical” behaviour of a system, and its average case perf • Single simulation run: one answer • Multiple runs: lots of (different) answers! • Central Limit Theorem: convergence to Normal distribution, with well-defined mean and variance (confidence intervals)

3. RNG Details • Truly random RNGs are hard to find • We settle for pseudo-random RNG • Assumes we can find “good” pseudo-random number generators • Assumes proper seeding of pseudo-RNG • Basic building block: U(0,1) • All other distributions can be generated from this using inverse mapping of CDF

4. Inverse Mapping (1 of 2) • If you have y = U(0,1), then you can generate x from any other distribution using inverse transformation (mapping) • Uses Cumulative Distribution Function (CDF) of the desired distribution • y = F(x) • x = F-1(y) • Can be analytic formula or empirical data

5. Inverse Mapping (2 of 2) y 1 y = F(x) y2 F(x) y1 x1 x2 x 0

6. Uniformly distributed Uncorrelated Reproducible Portable Long Period Analyzable Computationally fast Parallelizable Scales suitably to any number of processors Uncorrelated streams Independent generation Easy to use (API) Gives right results! RNG Requirements

7. LCG Basics • The most prevalent type of pRNG is the Linear Congruential Generator (LCG) • Formally a 4-tuple: LCG(m, a, b, y0) with modulus m, multiplier a, offset b, and initial seed y0 • Formula: yn+1 = (a yn + b) mod m • Generates pRN series in {0…m-1} • Can normalize to U(0,1) by dividing by m

8. LCG: Toy Example • Example: a=12, b=5, m=57, y0 = 34 • Sequence: • 34, 14, 2, 29, 11, 23, 53, 14, 2, 29, 11, ... • Note the short cycle with period < m • In practice, want m large, and a, b, y0 carefully chosen so that period is large • Some very good values are known

9. EICG • Another type is the Explicit Inversive Congruential Generator (EICG) • Also a 4-tuple: EICG(m, a, b, n0) • Formula: yn = (an + b) mod m • Note the inversion modulo m

10. Some Typical RNGs • rand() 32-bit multiplicative CG • random() 32-bit non-linear additive FB • drand48() 48-bit LCG • erand48() 48-bit LCG • Some are better than others • Don’t forget to seed them properly!

11. RNG Testing • How to test the goodness of a RNG? • Large samples • Bucket analysis (uniformity test) • Auto-correlation analysis (second-order) • d-dimensional plots • Spectral tests • Discrepancy tests • Kolmogorov-Smirnov (KS) tests

12. RNG Issues • Size of stream used vs. generation period • General safety rule: square root • Want independent streams for each RV • How to generate for parallel/distributed • Splitting into non-overlapping blocks • Leap-frog technique p1 p2 p3 p4 pi pj

13. Transform-Expand-Sample (TES) • What if you want correlated RNs? • Use TES! • Map U(0,1) to unit circle (modulo 1) • Constrain location of subsequent points • Multiple constraints 0 0 1

14. Summary • Random number generation is at the heart of almost any computer simulation • Quality of the pRNG is very important (and often overlooked!) • Using RNG properly is important • Seeding them properly is important too • Statistical inference from simulation results only makes sense if RNG works