Lesson 8: Basic Monte Carlo integration

1. Lesson 8: Basic Monte Carlo integration • We begin the 2nd phase of our course: Study of general mathematics of MC • Consists of a progression: • Monte Carlo evaluation of integrals (4 ways) • Basic numerical analysis framework (to explain the 4 ways) • MC evaluation of integral equations • Generalization of this technique to solve general differential equation sets

2. Monte Carlo Integration • Next set of mathematical tools: MC integration • Our study so far of sampling from distributions has provided us with the tools for MC simulation • MC integration will provide: • More rigorous ideas of keeping score • Basic mathematical underpinnings of variance reduction. • “Abstract” approach to MC problem: ALMOST ALL MC PROBLEMS ARE INTEGRATIONS • Development of four particular methods using the framework.

3. Four particular integration methods • We will now go over four particular variations on this theme: • Rejection method • Averaging method • Control variates method • Importance sampling method

4. Rejection method • This is a similar approach to the use of rejection methods in picking from a distribution. • It is a "dart board" method in which we estimate the area under a functional curve by containing the curve in a rectangular "box", picking a point randomly in the box, and scoring 0 if it misses (i.e., is above the curve) or the full rectangular area if it hits (i.e., is below the curve). • As before, we have to specify an upper bound of the function, , and then proceed by:

5. Rejection method (2) 1. Choose a value of uniformly between a and b. 2. Choose a value of uniformly between 0 and 3. Score if and score otherwise.

6. Rejection method example • Find using a rejection method. • Answer: The maximum value of this function in the range is 4, so our procedure is: • Choose a value of uniformly between 0 and 2. • Choose a value of uniformly between 0 and 4. • Score 8 if is less than ; otherwise score 0. • Find first two moments of this method and calculate the expected mean and SD of mean.

7. Averaging method • This is a much more straight-forward approach to the problem because it uses the function directly. The procedure for this method is to: • Choose a value of uniformly between a and b. • Score

8. Averaging Example • Again find using an averaging method. • Answer: The procedure is to: • Choose a value of uniformly between 0 and 2. • Score • Find first two moments of this method and calculate the expected mean and SD of mean. (Compare to previous method.)

9. Control variates method • This method is the first of two methods that utilize a user-supplied second function, , which is chosen to be a "well behaved" approximation to • What makes these methods so powerful is that they allow the user to take use of a priori knowledge about the function. • In the control variates method, the integral solution "begins" as the integral of the known function: • and uses the Monte Carlo approach to find an additive correction to this user-supplied guess.

10. Control variates method (2) • The procedure for this method is to: • Choose a value of uniformly between a and b. • Score • Notice that there is NO variance introduced through the part of the score. • Obviously, then a good guess will result in a small difference and, therefore a small variance. • In the limit of a perfect guess, , there is no correction and no therefore no variance. • Not quite as obvious is the fact that if h(x) and f(x) differ by a CONSTANT, we also have a 0 variance method.

11. Control variates example • Again find , this time using a control variates method with • Answer: Note the integral of h(x) over (0,2) is 2. With this value known, the procedure is to: • Choose a value of uniformly between 0 and 2. • Score • Find first two moments of this method and calculate the expected mean and SD of mean. (Compare to previous methods.)

12. Importance sampling method • The final method is the importance sampling method. This technique is similar to the control variates method, in that it takes advantage of a priori knowledge about the function , but differs from it in that its correction is multiplicative rather than additive. • The importance sampling method uses the approximate function as the probability distribution with which the variables are drawn:

13. Importance sampling (2) • The resulting score is: • As with control variates, a "perfect" guess of would result in a zero variance solution, this time because, again, every score would be exactly correct. • (Note that, because of the normalization, a guess equal to a MULTIPLE of f(x) will also work.)

14. Importance sampling example • Again find , this time using an importance sampling method with • Answer: Since the integral of h(x) over the range (0,2) is 2, the resulting probability distribution from which to pick the x’s will be: • Following the direct procedure for choosing from this distribution, we first determine the c.d.f, which is:

15. Importance sampling example • We then set this c.d.f. to the uniform deviate: • and invert to get the formula: • Score is now: • Find first two moments of this method and calculate the expected mean and SD of mean. (Compare to previous methods.)

16. 2nd pass at integration: more rigor • Theoretical underpinning is the Law of Large Numbers • In one of our early lectures, we defined the mean of a continuous function as: • And later worked out a Monte Carlo algorithm with the same expectation:

17. Law of Large Numbers (2) • Remember that the Law of Large Number takes this a step further by replacing the x with a function f(x) and speaking of the average value of the function, : • This relates the result of a continuous integration with the result of a discrete sampling. All MC comes from this.

18. Using the Law of Large Numbers • Putting our “goal” integration in this form requires that we multiply and divide by the probability distribution,p(x) • Following the previous “rules” we have divided the integrand into two “pieces”: the score and the PDF • There is an implicit requirement that p(x)>0 for all x for which f(x) is not 0 so that f(x)p(x)/p(x) is defined

19. Averaging method • The easiest of our four methods to put in this form is the averaging method (which we previously discussed second) • Recall that the procedure for this method is to: • Choose a value of uniformly between a and b. • Score • In terms of our mathematical framework, this is equivalent to again using: • and scoring with a direct use of

20. Rejection method Backing up to the rejection method, the procedure was: 1. Choose a value of uniformly between a and b. 2. Choose a value of uniformly between 0 and 3. Score if and score otherwise. • In terms of our mathematical framework, this is equivalent to using: (for a uniform distribution between a and b) and …

21. Rejection method (2) scoring with a probability mixing strategy of: with probability or scoring 0 with probability This mixed scoring strategy obviously has the desired expected value of

22. Control variates method • The procedure for this method is to: • Choose a value of uniformly between a and b. • Score • where, h(x) is chosen as an easily integrated approximation of f(x)+constant

23. Control variates method (2) • In terms of our mathematical framework, this again uses a flat distribution and score with:

24. Importance sampling method • The procedure for this method is to: • Choose a value of between a and b using a probability distribution h(x) that is “shaped like” f(x). • Score • In terms of our mathematical framework, this is a simple replacement of the flat distribution of the averaging method with the “better” distribution h(x) (with allowance for the fact that h(x) is probably unnormalized):

25. Importance sampling method(2) • Giving us:

26. Solution of Integral Equations Application of our integration techniques to integral equations • Introduction of Dirac notation • Conversion of differential equations to integral equations • Solution of integral equations • Solution of linked equations

27. Dirac notation • In our integrations so far, I have simplified the mathematics a bit by always choosing x between a and b. • I was careful to always choose x between a and b. What if I had not done this?

28. Dirac notation (2) A more rigorous way to approach this is to look at the Monte Carlo attack of the integral in TWO steps: (1) an approximation of f(x) itself using: (2) a substitution of this functional approximation into the integral:

29. Dirac notation (3) This is the approach we will take from now on. The notation: has the advantage of giving us not only the “weight” but also reminding us of the selected point. This way we can think of a “sample” as having these two pieces:a “weight” and a “location”

30. Averaging Example with Dirac • For the third time, find , this time using Dirac approximation • Answer: The Dirac approximation is:

31. Averaging Example with Dirac (2) • If we use: , then we are guaranteed that , giving us: • which is equivalent to the averaging method

32. Averaging Example with Dirac (3) • If we use: • then plugging in gives us the importance sampling result:

33. Developing integral equations from differential equations: Simple • We now know how to attack integrals with Monte Carlo • We desire to be able to “solve” differential equations = estimate functionals (usually integrals or point values) of the function that solves a given equation • Traditional solution: Convert them into integral equations and apply the MC integration rules to them • Example: Find the value of f(4), given the differential equation and boundary condition:

34. Simple integral equations (2) • Answer: We can integrate from 0 (the known value) to the desired value to get: • Now we apply one of the four integration methods to the integral in the equation:

35. Simple integral equations (2) • NOTE: From now on, I will skip the summation and division by N and just write:

36. Simple integral equations (3) • The normal procedure for this method is to: • Choose a value of between a and b using a probability distribution p(x) (of YOUR choosing). • Score • So, let’s do it. • What PDF should we use? • Lazy man’s PDF: uniform • Optimum PDF: ? (You tell me…)

37. Linked equations • When you are faced with linked equation sets, the principles are the same, put you have to be more careful: • Putting in multiple boundary conditions • Keeping up with multiple sampled variables (each equation will have one) • Most tricky: Realizing and adapting to CHANGING LIMITS on the integrals (after the first) • MUCH more difficult to optimize the choice of the PDFs used

38. Linked equation example • Example: Find f(2) for the second order differential equation: • In order to make it fit the category, we will start be re-writing as the linked set:

39. Linked equation example (2) • Applying our tools to the second equation first, we begin by transforming it into an integral equation for the value at x=2: • Using our MC integration approximation, we get: • How do we get the ? Answer: We estimate it from the other equation.

40. Linked equation example (3) • Applying our tools to the first equation first, we begin by transforming it into an integral equation for the value at : • The resulting procedure is: • Choose a value of using • Choose a value of using • Score:

41. Linked equation example (4) • Now let’s do it. • What PDF’s to use? • Flat • Better than flat

42. Homework