Chapter 1 Probability and Distributions

1. Chapter 1Probability and Distributions Math 6203 Fall 2009 Instructor: AyonaChatterjee

2. 1.1 INTRODUCTION • Every experiment has one or more outcomes. • Random experiment > single outcome cannot be predicted but outcomes over a long period of time follow a certain rule. • Sample Space > collection of every possible outcome. • Denoted by C. • Let c denote an element in C and let C represent a collection of elements of C.

3. Examples: Toss a coin, toss 2 coins, throw a die. Write sample spaces. • We define f/N as the relative frequency of event C in N performances. • As N increases, f/N -> p • Here p is the number which, in future performances of the experiment the relative frequency of the event C will approximately equal. • ‘p’ is called the probability of event C.

4. Note: • Above idea works only if experiment can be repeated in identical conditions. • But sometimes p is personal or subjective. • For both mathematical theory is the same.

5. 1.2 SET THEORY • Set -> Collection of particular objects. • Notation: c is an element if set C, • One-one correspondence – Every member of A is paired with exactly one member of B. No two ordered pairs have the same first element or the same second element.

6. Definitions • If each element C1 is also an element of C2, the set C1 is called subset of set C2. • Example C1 ={x: 0≤x ≤1} and C2 ={x: -1 ≤x ≤2} then C1 is the subset of C2.

7. Null set: If C has no elements, C = Φ. • Union: Set of all elements that belong to at least one of the sets C1 and C2 is called union of C1 and C2. Notation C1 U C2. • Intersection: Set of all elements that belong to each of the sets C1 and C2 is called intersection of C1 and C2. Denoted by C1 C2 .

8. Complement: The set that consists of all elements of C that are not elements of C is called the complement of C. We will denote complement of C as Cc . • De Morgan’s Laws • Let us prove the laws and do some examples for union, intersection, complement with • C1 ={x: 0≤x ≤1} and C2 ={x: -1 ≤x ≤2}

9. Let Cbe a set in one-dimensional space and let Q(C) be equal to the number of points in C which correspond to positive integers. Then Q(C) is a function of the set C. • Example C = {x, 0 < x< 5}, then Q(C)=4 • C = {-1, -2}, Q(C) = 0 • Let C be a set in one-dimensional space and let Q(C)=∑f(x) over C or • Note: f(x) has to be chosen with care or else the integral may fail to exist.

10. Examples

11. 1.3 THE PROBABILITY SET FUNCTION • σ – Field: Let B bea collection of subsets of C. We say B is a σ-field if • B is not empty. • If • If the sequence of sets {C1, C2, ….}is in B then • Example B = {C, Cc , Φ, C} is a σ-field. • σ-field is also called the Borelσ-field .

12. Probability definition • Let C be a sample space and let B be a σ-field on C. Let P be a real valued function defined on B. Then P is a probability set function if P satisfies the following three conditions. • P(C)≥ 0 • P(C) = 1 • If {Cn} is a sequence of sets in B and

13. Theorems-work on proofs

14. Inequalities • Booles’s inequality • Bonferoni’s Inequality

15. Definitions • Mutually exclusive events: when two events have no elements in common. • Equilikely case: If there are k events each event has probability 1/k. • Permutation and Combination. • Examples from Poker hand, page 17

16. Theorems

17. 1.4 CONDITIONAL PROBABILITY AND INDEPENDENCE • Let C1 and C2 both be subsets of C. We want to define the probability of C2 relative to C1. This we call conditional probability, denoted by P(C2 | C1 ). • Since C1 is the sample space now, the only elements of interest are the ones in both C1 and C2.

18. Properties of P(C2 | C1 ) • Note P(C2 | C1 )≥0 • Note • P(C1 | C1 ) = 1

19. Examples • Example: A hand of 5 cards is to be dealt at random without replacement from an ordinary deck of cards. Find the probability of an all-spade hand given that there are at least 4 spades in the hand. • From an ordinary deck of playing cards, cards are to be drawn successively at random without replacement. Find the probability that the third spade appears on the sixth draw.

20. Bayes Theorem • Law of total probability – Assume that events C1, ….Ck are mutually exclusive and exhaustive events. Let C be another event, C occurs with one and only one of the events. • Bayes Theorem- To find the conditional probability of Cj given C.

21. Bayes Theorem • The probabilities P(Ci ) are called prior probabilities. • The probabilities P(Ci | C) are called posterior probabilities. • Lets work through example 1.4.5, page 25 from the text book.

22. Independent Events • Let C1 and C2 be two events. We say that C1 and C2 are independent if • Suppose we have three events, then the events are mutually independent if and only if • They are pair wise independent and Lets work through example 1.4.10, page 29.

23. 1.5 RANDOM VARIABLES • Consider a random experiment with a sample space C. A function X which assigns to each element c (c belongs to the sample space) one and only one number X(c) = x, is a random variable. • Notation: • B: any event • D: a new sample space • {di }:simple events which belong to D

24. PMF • Here Px is completely determined by the function • The function px (di) is called the probability mass function (pmf). • Example: Write the pmf for X: sum of the upfaces on a roll of a pair of 6-sideddice. • Compute probability for event B={x: x=7,11}

25. Cumulative Distribution Function • Let X be a random variable. Then its cdf is defined as • Continuing with the previous example, define the cdf of X. • For a continuous random variable, the probability density function can be obtained as

26. Note • If X and Y are two random variables and FX(x)=Fy (y) for all x in R, then X and Y are equal in distribution and it is denoted as • Though X and Y may be equal in distribution, X and Y itself may be quite different. • See example in book.

27. Theorems • Let X be a random variable with cumulative distribution function F(x). Then • Let X be a rv with cdfFx . Then for a < b, P[a<X≤b]=Fx (b)-Fx (a)

28. Theorem • For any random variable • Lets work through the proof.

29. 1.6 DISCRETE RANDOM VARIABLES • We say a random variable is a discrete rv if its space is either finite or countable. • Example: • Consider a sequence of independent flips of a count. Let X be the number of flips required to obtain the first head. What is the pmf for X? • Five fuses are chosen from a lot of 100 fuses. The lot contains 20 defective fuses. Let X be the number of non-defective fuses, what is the pmf for X?

30. Properties of pmf

31. Transformations • Suppose we are interested in some random variable Y, which is some transformation of X, say Y=g(X). Assume X is discrete with space Dx. • Then the space of Y is Dy ={g(x);x belongs to Dx }. • Y can be obtained as

32. Example • Let X have the pmf • Find the pmf of Y where Y = X2

33. 1.7 CONTINUOUS RANDOM VARIABLE • We say a random variable is a continuous random variable if its cumulative distribution function is a continuous function for all x belong to R. • We define the cdf as • The probability density function can be defined as

34. Note

35. Example • Let the random variable be the time in seconds between incoming telephone calls at a busy switchboard. Suppose that a reasonable pdf for X is given below. Find P(X>4).

36. Transformation • Let X be a continuous random variable with pdffX(x) and support S. Let Y =g(X), where g(x) is a one-to-one differentiable function, on the support of X, S. Denote the inverse of g by x=g-1 (y) and let dx/dy =d[g-1 (y)]/dy. Then the pdf of Y is given by

37. Example • Let X have the pdf given below. Find the pdf for Y where Y=-2logX. • f(x)=1 for 0<x<1

38. 1.8 EXPECTATION OF A RANDOM VARIABLE • Let X be a random variable. If X is a continuous random variable with pdf f(x) and

39. Example for a discrete RV • Let the random variable X of the discrete type have the pmf given by the table below. Find E(X).

40. Theorem • Expectation of a constant is a constant itself, that is E(k)=k. • Let X be a random variable and let Y=g(X) for some function g. • Suppose X is continuous with pdf f(x).

41. Theorem • Let g1(X) and g2(X) be the functions of a random variable X. Suppose the expectations of g1(X) and g2(X) exist. Then for any constants k1 and k2, the expectation of k1 g1(X) + k2 g2(X) exists and is given • E[k1 g1(X) + k2 g2(X)]= k1 E(g1(X)) + k2 E(g2(X) )

42. 1.9 SOME SPECIAL EXPECTATIONS • Some expectations have special names. • E(X) is called the mean value of X. • It is denoted by µ. • The mean is the first moment (about 0) of a random variable.

43. Variance • Let X be a random variable with finite mean µ and such that E[(x- µ)2] is finite. Then the variance of X is defined to be E[(x- µ)2]. It is usually denoted by σ2 or by Var(X). • We can write • Here σ is called the standard deviation of X. • Measures the dispersion of the points in the space relative to the mean.

44. Moment Generating Function • Let X be a random variable such that h>0, the expectation of etxexists for –h < t < h. The moment generating function (mgf) of X is defined to be the function M(t)=E(etx) for –h<t<h. • If X and Y have the same distribution, them they have the same mgf in a neighborhood of 0. The converse is also true.

45. Note