Chap 5 Sums of Random Variables and Long-Term Averages

1. Chap 5Sums of Random Variables and Long-Term Averages • Many problems involve the counting of number of occurrences of events, computation of arithmetic averages in a series of measurements. • These problems can be reduced to the problem of finding the distribution of a random variable that consists of sum of ni.i.d. random variables.

2. 5.1 Sums of Random Variables

3. Ex. 5.2

4. The characteristic function of Sn

5. Ex. 5.3 Sum of n iid Gaussian r.v. with parameters and . Ex. 5.5 Sum of n iid exponential r.v. with parameter p.102 n-Erlang

6. If are integer-valued r.v.s. , it is preferable to use the prob. generating function (z-transform). Ex. Find the generating function for a sum of n iid geometrically distributed r.v. p.100, negative binomial

7. Sum of a random number of random variables N is a r.v., independent of Ex. 5.7

9. 5.2 Sample Mean and Laws of Large Numbers

10. Using Chebyshev inequality

12. Weak Law of Larger Numbers Mn Fig. 5.1 n Sample mean will be close to the true mean with high probability Strong Law of Larger Numbers With probability 1, every sequence of sample mean calculations will eventually approach and stay close to E[X].

14. 5.3 The central Limit Theorem CLT: as n becomes large, cdf of Sn approach that of a Gaussian. then

16. characteristic function of a zero-mean, unit-variance Gaussian r.v. Fig 5.2-5.4 show approx.

19. 5.4 Confidence Intervals If is small, Xj’s are tightly clustered about Mn. and we can be confident that Mn is close to E[X ].

21. Case 1. Xj’s Gaussian with unknown Mean and known Variance Mn is Gaussian with mean and variance

22. Table5.1

23. Case2: Gaussian; Mean and Variance unknown use sample variance as replacement of variance the confidence interval becomes Zero-mean unit-variance Gaussian Indep. W is a student’s t-distribution with n-1 degrees of freedom.

24. (Ex. 4.38)

25. Table 5.2

26. Case 3: non-Gaussian; Mean and Variance unknown. Use method of batch mean. Performing a series of M independent experiments in which sample mean (from a large number of observations) is computed.

27. 5.4 Convergence of Sequences of Random Variables a sequence of functions of

28. Ex.5.18. 1 1 1 a sequence of real number for a given . 0 n 2 3 4 5 1

29. n N

30. n Ex: Strong Law of Large numbers

33. Mean-Square Convergence Convergence in Probability n n0 Ex: weak law of large numbers.

34. Ex. Central limit theorem Ex. 5.21: Bernoulli iid sequence

35. m.s. a.s. s prob dist