1 / 27

General Principle of Monte Carlo Fall 2013

This article provides an overview of the principles of Monte Carlo method, discussing its mathematical theory, strengths, and weaknesses. It also explores examples of its application in solving deterministic and probabilistic problems.

dixonn
Download Presentation

General Principle of Monte Carlo Fall 2013

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. General Principle of Monte CarloFall 2013 By Yaohang Li, Ph.D.

  2. Review • Last Class • Short Resume of Statistics • Probability • Random Variables • Important Distributions • Uniform Distribution • Exponential Distribution • Binomial Distribution • Poison Distribution • Normal Distribution • Estimation • Sample • Estimand • Parent Distribution • Sampling Distribution • Estimator • Important estimators • This Class • Nature of Monte Carlo • Direct Simulation • Monte Carlo Integration • Next Class • Analysis of Monte Carlo Integration

  3. Mathematical Theory and Experiment • Theoretical Mathematics • Deduce conclusions from postulates • Experimental Mathematics • Infer conclusions from observations • Complicated Mathematical Experiments require computers • Difference • Between deduction and induction

  4. Problems Handled by Monte Carlo • Probabilistic Problems • Directly concern with the behavior and outcome of random processes • Approach (Direct Simulation) • Choose random numbers in a certain way • Directly simulate the physical random process of the original problem • Infer the desired solution from the behavior of these random numbers • Usually possess little theoretical interest • Deterministic • Not directly concern with the behavior of random processes • Usually called Sophisticated Monte Carlo

  5. Strength and Weakness of Theoretical Mathematics • Theoretical Mathematics • Strength • Abstraction and Generality • Abstract the essence of a problem and reveal its underlying structure • Inherent weakness • The more general and formal the theory • The harder to provide a numerical solution in a particular application • Main idea behind Monte Carlo approach to deterministic problems • Exploit the strength of theoretical mathematics • Avoid its associated weakness by replacing theory by experiment whenever the theory falters

  6. Example of Sophisticated Monte Carlo • Electromagnetic Theory • Requires solution of Laplace’s equation subject to certain boundary conditions • The boundary conditions may be complicated and no analytical techniques are available • Monte Carlo Method • Study of particles • Diffuse randomly in a region bounded by absorbing barriers • Guide the particles by means of random numbers • Until they are absorbed on barriers • Barriers are chosen to represent the prescribed boundary conditions • The distribution of particles absorbed on barriers gives the solution

  7. Monte Carlo Solution • Monte Carlo Solution • Uncertain • Arise from raw observational data consisting of random numbers • If we can manage to make the uncertainty fairly negligible, Monte Carlo can serve a useful purpose • Good experiments • Ensure the sample should be more rather than less representative in the Monte Carlo computation • Good representation of conclusions • Indicates how likely the solution can be wrong

  8. Reducing Uncertainty • Reducing Uncertainty • Collect more observations • Not very economic • Variance Reduction • Manipulate the mathematical problem • Reduce the inherent variance

  9. Direct Simulation • Probabilistic Problem • Direct simulate the physical random processes of the original problem • Direct observation of the random numbers • Examples • Control of the floodwater and the construction of dams on the Niles • The quantity of water in the river varies randomly from season to season • Use data records of weather, rainfall, and water levels extending over 48 years • To see what will happen to the water if certain dams are built and certain possible policies of water control exercised • Telenetworking • Growth of an insect population on the basis of certain assumed vital statistics of survival and reproduction • Service rates • Operational research • Computer games

  10. Direct Simulation of the Lifetime of Comets • A Long-period Comet • Described as a sequence of elliptic orbits • Sun at one focus • Energy of a comet is inversely proportional to the length of the semimajor axis of the ellipse

  11. Direct Simulation of the Lifetime of Comets • Behavior of the Comet • Most of the time • Moves at a great distance from the sun • A relatively short time (instantaneous) • Passes through the immediate vicinity of the sun and the planets • At this instant, the the gravitational field of the planet perturbs the cometary energy by a random component • Successive energy perturbations (in suitable units of energy) may be taken as independent normally distributed random variables 1, 2, 3,…, n • 1, 2, 3,…, n can be normalized to a standard normal distribution

  12. Direct Simulation of the Lifetime of Comets • Comets under perturbation • A comet, starting with an energy –z0, has accordingly energies -z0, -z1=-z0+1, -z2=-z1+2, … • The process continues until the first occasion on which z changes sign • Once z changes sign, the comet departs on a hyperbolic orbit and is lost from the solar system • Kepler’s Third Law • the time taken to describe an orbit with energy –z is z-3/2 • the total lifetime of the comet is • zT is the first negative quantity in the sequence of z0, z1, …

  13. Direct Simulation of the Lifetime of Comets • Solution • Very difficult problem in theoretical mathematics • Easy to simulate • f(g)=P(Gg) is the required distribution solution

  14. Direct Simulation Examples • Computational Financing • http://www.mathfinance.wagner.com/PRODUCTS/RETIREMENT/RSP/Applets/BernApplet.html

  15. Numerical Integration • Methods for approximating definite integrals • Rectangle Rule • Trapezoidal Rule • Divide the curve into N strips of thickness h=(b-a)/N • Sum the area of each trip • Approximate to that of a trapezium • Simpson’s Rule • Calculate quadratic instead

  16. Monte Carlo Method • General Principles • Every Monte Carlo computation that leads to quantitative results may be regarded as estimating the value of a multiple integral • Efficiency • Definition • Suppose there are two Monte Carlo methods • Method 1: n1 units of computing time, 12 • Method 2: n2 units of computing time, 22 • Methods comparison

  17. Monte Carlo Integration • Consider a simple integral • Definition of expectation of a function on random variable  • If  is uniformly distributed, then

  18. Crude Monte Carlo • Crude Monte Carlo • If 1, …, n are independent random numbers • Uniformly distributed • then fi=f(i) are random variates with expectation  • is an unbiased estimator of  • The variance is • The standard error is • /n1/2

  19. Hit-or-Miss Monte Carlo Revisit • Suppose 0  f(x)  1 when 0  x  1 • Main idea • draw a curve y=f(x) in the unit square 0x,y1 • is the proportion of the are of the square beneath the curve • or we can write

  20. Analysis of Hit-or-Miss Monte Carlo •  can be estimated as the a double integral • The estimator of hit-or-miss Monte Carlo

  21. Hit-or-Miss Monte Carlo • Hit-or-Miss Monte Carlo • We take n points at random in the unit square, and count the proportion of them which lie below the curve y=f(x) • The points are either in or out of the area below the curve • The probability that a point lies under the curve is  • The Hit-or-Miss Monte Carlo is a Bernoulli trial • the estimator of Hit-or-Miss Monte Carlo is binomial distributed

  22. Binomial Distribution Revisit = = • Binomial Distribution • Discrete probability distribution Pp(n|N) of obtaining exactly n successes out of N Bernoulli trials • Each Bernoulli trial is true with probability p and false with probability q=1-p • variance N(1-p)p

  23. Comparison of Hit-or-Miss Monte Carlo and Crude Monte Carlo • Standard error of Hit-or-Miss Monte Carlo • Standard error of Crude Monte Carlo • Hit-or-Miss Monte Carlo is always worse than Crude Monte Carlo • Why?

  24. Why Hit-or-Miss Monte Carlo is worse? • Fact • The hit-or-miss to crude sampling is equivalent to replacing g(x, ) by its expectation f(x) • The y variable in g(x,y) is a random variable • Leads uncertainty • Leads extra uncertainty in the final results • Can be replace by exact value

  25. General Principle of Monte Carlo If, at any point of a Monte Carlo calculation, we can replace an estimate by an exact value, we shall replace an estimate by an exact value, we shall reduce the sampling error in the final result

  26. Summary • Nature of Monte Carlo • Experimental Mathematics and Theoretical Mathematics • Problems that Monte Carlo can handle • Probabilistic • Direct Simulation • Deterministic • Sophisticated Monte Carlo • Monte Carlo Solution • Good Experiment • Good Conclusion • Direct Simulation • Comet Lifetime Problem • Monte Carlo Integration • Hit or Miss Monte Carlo • Crude Monte Carlo

  27. What I want you to do? • Review Slides • Review basic probability/statistics concepts • Work on your Assignment 1

More Related