Numerical Methods

Numerical Methods Lecture 5 – Random Simulation Methods Dr Andy Phillips School of Electrical and Electronic Engineering University of Nottingham

Today’s Topics • Random numbers • Classifications of Random Numbers • How to generate them • Monte Carlo Simulations • Calculating  • Dice Throws • Common Birthdays

Classification of random numbers • Random numbers for use in computer programs can be classified into 3 different categories: • Truly random numbers • Pseudorandom numbers • Quasi-random numbers

Truly random numbers • Truly random numbers obviously cannot be produced by computer programs • They must be supplied by an external source like radioactive decay • Such sequences are available (e.g. on magnetic tape), but clumsy to use and often not sufficient in terms of speed and number

Truly Random Numbers • Although the casinos at Monte Carlo are (hopefully!) based on random phenomena, true random numbers are rarely used in computing • Not only are such numbers difficult to generate reliably, but often the lack of reproducibility renders the validation of programs that use them extremely difficult

Pseudorandom numbers • A sequence of numbers is generated by an algorithm in a way that the resulting numbers look statistically independent and uniformly distributed • This is the prevailing method used in random number generators

y y x x Pseudorandom numbers Representing pairs of PR numbers as (x,y) co-ordinates then get (with hopefully many more points!) a distribution like the left hand picture, with ‘clusters’ and ‘voids’. This is a disadvantage for some applications where it is vital that something even more uniform is obtained. In the right hand picture there are less ‘clusters’ and ‘voids’ (but we do not go as far as using a regular grid). To get this there is a price to pay…

Quasi-random numbers • For some applications PR numbers are “a little too random”! • We need quasi-random numbers • Plotted in pairs as on previous slide, on the right • These are generated by algorithms tuned to optimize the sequence’s uniform distribution, which can improve the accuracy of Monte-Carlo integration. • These numbers are not statistically independent and thus cannot be used generally

Random Numbers • Random numbers are used in computing to simulate apparently random processes in the external world • Well-known techniques exist for generating, in a deterministic fashion, number sequences largely indistinguishable from true random sequences. • The deterministic nature of these techniques is important because it provides for reproducibility in computations

Random Numbers • Computers invariably use pseudo-random numbers (for built in random number generators): • finite sequences generated by a deterministic process but indistinguishable, by some set of statistical tests, from a random sequence. • In the following, we use the term random to mean pseudo-random • The statistical methods used to validate random sequences are an important topic of research, but beyond the scope of this course

Random Numbers • Methods for generating a sequence of random numbers have been extensively studied and are well understood • A function called a generator is defined that, when applied to a number, yields the next number in the sequence • For example, the linear congruential generators have the general form:

Random Number Generation • Linear Congruential Generator • Where • Xk is the kth element • Xo, a, c & m define the generator • a is the multiplier • c is the increment • m is the modulus • For numbers in the range [0,1] we divide by m Xk+1 = ( aXk + c ) mod m

Random Number Generation • Such a sequence will however repeat, the length of the repeated cycle being referred to as the period • A good generator has • A long period & • No discernible correlation between elements of the sequence • Compiler RNGs may use a combination of linear congruential generators

In its use • The parameters • Xo, a , c , and m • These values are chosen to make the sequence look as random as possible. • Common choices for these values are given on the next slide

Random Number Generation • Typical Values • Xo - Any positive integer • a = 16807 • m = 231 – 1 (i.e. 2147483647) • c = 0 • Which has a period of • m-1 (ie 231-2) • Called the minimal standard RNG (sometimes the RNG in compilers and other software) • Poor value choice can be a real problem • The Marsaglia effect (3D plot shows linear structure)

Time for an example (Hopefully!)

Random Distributions • Distributions other than uniform can be generated by suitable transformations of the basic uniformly distributed sequence. • Numerical libraries often offer a rich set of distributions • They provide robust methods for RNG

Generating these RNs • Whilst we could write our own RNG… • Compilers come with RNGs which we can make use of • We pass them a seed (starting point of sequence – often the time of day ) • In C this is the srand function • We then obtain our values with the appropriate function • In C this is rand

On to simulation… • Numerical methods that are known as Monte Carlo methods can be loosely described as statistical simulation methods, • Where statistical simulation is defined in quite general terms to be any method that utilizes sequences of random numbers to perform the simulation.

Monte Carlo methods • Monte Carlo methods have been used for centuries, but only recently (in the last seven decades) has the technique gained the status of a full-fledged numerical method capable of addressing the most complex applications.

Monte Carlo Method • The name ‘Monte Carlo’ was coined by Nicholas Metropolis (inspired by Stanislav Ulam's interest in poker) during the Manhattan Project of World War II, • Due to the the similarity of statistical simulation to games of chance, • and because the capital of Monaco was a center for gambling and similar pursuits

They are used for… • Serious problems • Nuclear Reactor Design • Traffic Flow • Economic Forecasting • And not so serious problems • Bingo • Etc.

Monte Carlo Method • The Monte Carlo method depends on us being able to define the Probability Density Function of a Problem R[0,1] f(x) Value f(x) = pdf describing e.g. traffic flow NB Use the fact that integral of f(x) is 1 over full range of x

Monte Carlo Method • The simulation proceeds by performing random sampling from the ‘pdf’ • Find the x value at which the cumulative probability is equal to the RN generated • This process is repeated (“trials”) and the desired result is taken to be the average over the number of observations

We can even integrate • Consider Area = Average height * base i.e. Average value of y(x) over the range [a,b] Use MC to find average height Select n points in range [a,b] Average = 1  y(i) y A x a n b i=1 n n => Area = (b-a) *  y(i) n i=1

y 1 x2 + y2 = 1 x 1 And of course calc. areas… • Using the above method… • Though such methods are fairly slow to converge Area = ¼ circle of radius 1 Using RN [0,1], we have Area = (1 - 0)  (1-[x(i)]2) y(i) n 1/2 i=1 n 1st lab will test the accuracy of this method

(-1,1) (1, 1) (-1,-1) (1,-1) And we can even calculate  Area of circle  Area of Square 4 = Loop: Pick two random numbers [0,1] n=n+1 If sqrt(x2+y2)<=1 m=m +1 [i.e. increment counter n every point and increment counter m if point in circle quarter]

 = 4m n And we can even calculate  Area of ¼ circle (/4) Area of ¼ Square 1 = (1, 1) After enough goes this resolves to m = (/4) n 1

There are many more • Birthday Problem • Probability of at least two of ‘n’ people in a room sharing the same birthday • Define array [0…365] • Generate random numbers rn in range (1,365) • See if rn ‘seen’ before (check array), inc as appropriate • Set array [rn] to 1 to flag this value as ‘seen’ • Repeat for ‘n’ people • Repeat the whole thing many times • for 23 people, probability is 50% • For simplicity ignore leap years!!

There are many more • Dice Problem • Probability of throwing an 8 with 2 dice • Calculate two random numbers each in the range 0…6 • Convert to whole number values 1,2,…6 by rounding up • Sum them, if get 8 then increment a counter • Increment ‘total throw’ counter • Repeat many times • Probability = no of 8’s / total throws

To sum up • The MC examples shown have been for simple examples where we can easily define the pdf • In ‘real life’ the pdf will be more complex, but the general approach is always the same • So the method remains a simple and effective tool

Numerical Methods