Basic simulation methodology

1. Basic simulation methodology • Simulating multivariate distributions • Simulating random sequences • Importance sampling • Antithetic sampling • Quasi random numbers Computational statistics, lecture 2

2. Methods for simulating multivariate distributions • Transforming pseudo random numbers (PRNs) having a multivariate distribution that is easy to simulate • Using factorization of the multivariate density into univariate density functions • Using envelope-rejection techniques Computational statistics, lecture 2

3. Illustrations of independent and dependent normal distributions http://stat.sm.u-tokai.ac.jp/~yama/graphics/bnormE.html Computational statistics, lecture 2

4. Theory of transforming a normal distribution to another normal distribution • Let X be a random vector having a zero mean m-dimensional normal distribution with covariance matrix C • Let Y = B X where B is an arbitrary k x m matrix • Then Y has a k-dimensional zero mean normal distribution with covariance matrix B C BT Computational statistics, lecture 2

5. Generating a bivariate normal distribution with a given covariance matrix: method 1 • Let Y be a random vector having a bivariate normal distribution with covariance matrix C • Then, C can be decomposed into a product C = B BT • Furthermore, the random vector B X, where X has a standard bivariate normal distribution, has a bivariate normal distribution with covariance matrix C Example: Computational statistics, lecture 2

6. Generating a bivariate normal distribution with a given covariance matrix: method 2 Let Y be a zero mean bivariate normal distribution with density Decompose the probability density into Note that the conditional distribution is normal with Example: Computational statistics, lecture 2

7. Random number generation:method 3 - the envelope-rejection method • Generate x from a probability density g(x) such that cg(x)  f(x) where c is a constant • Draw u from a uniform distribution on (0,1) • Accept x if u < f(x)/cg(x) *************************** Justification: Let X denote a random number from the probability density g. Then • How can we generate normally distributed random numbers? Computational statistics, lecture 2

8. Simulation of random sequences Example 1: Random walk Example 2: Autoregressive process Note: A burn-in period is needed Computational statistics, lecture 2

9. Simulating rare events by shifting the probability massto the event region • Assume that we would like to estimate pt= P(X > t) where X is a random variable with density f(x) • Let f* be an alternate probability density • Then and we can estimate pt by computing where Xi has density f* Computational statistics, lecture 2

10. Simulating rare events by scaling or translation • Assume that we would like to estimate p = P(X > t) where X has probability density f(x) • Scaling: • Translation: Computational statistics, lecture 2

11. Simulating rare events by scaling: a simple example • Assume that we would like to estimate p = P(X > 4) where X has a standard normal distribution. • Let f* be the probability density of 10X • Then and we can estimate pt by computing where Xi is normal with mean zero and standard deviation 10 Computational statistics, lecture 2

12. Antithetic sampling Use the same sequence of underlying random variates to generate a second sample in such a way that the estimate of the quantity of interest from the second sample will be negatively correlated with the estimate from the original sample. Computational statistics, lecture 2

13. Antithetic sampling – a simple example Use Monte-Carlo simulation to estimate the integral How can we apply the principle of antithetic sampling? Computational statistics, lecture 2

14. Quasi random numbers(minimal discrepancy sequences) • Quasi-random numbers give up serial independence of subsequently generated values in order to obtain as uniform as possible coverage of the domain • This avoids clusters and voids in the pattern of a finite set of selected points http://www.puc-rio.br/marco.ind/quasi_mc.html Computational statistics, lecture 2

15. Pseudo and quasi random numbers Pseudo random numbers Quasi random numbers Computational statistics, lecture 2