Basic Axioms of Probability

1. Basic Axioms of Probability Lecture II

2. Basis of Probability • Using the example from Birenens Chapter 1: Assume we are interested in the game Texas lotto (similar to Florida lotto). • In this game, players choose a set of 6 numbers out of the first 50. Note that the ordering does not count so that 35,20,15,1,5,45 is the same of 35,5,15,20,1,45. • How many different sets of numbers can be drawn? • First, we note that we could draw any one of 50 numbers in the first draw.

3. However for the second draw we can only draw 49 possible numbers (one of the numbers has been eliminated). Thus, there are 50 x 49 different ways to draw two numbers • Again, for the third draw, we only have 48 possible numbers left. Therefore, the total number of possible ways to choose 6 numbers out of 50 is

4. Finally, note that there are 6! ways to draw a set of 6 numbers (you could draw 35 first, or 20 first, …). Thus, the total number of ways to draw an unordered set of 6 numbers out of 50 is • This is a combinatorial. It also is useful for binomial arithmetic:

5. Basic definitions: • Sample Space: The set of all possible outcomes. • In the Texas lotto scenario, the sample space is all possible 15,890,700 sets of 6 numbers which could be drawn. • Event: A subset of the sample space. • A subset of the sample space. In the Texas lotto scenario, possible events include single draws such as {35,20,15,1,5,45} or complex draws such as all possible lotto tickets including {35,20,15}. Note that this could be {35,20,15,1,2,3}, {35,20,15,1,2,4},….

6. Simple Event: An event which cannot be a union of other events • An event which cannot be a union of other events. In the Texas lotto scenario, this is a single draw such as {35,20,15,1,5,45}. • Composite Event: An event which is not a simple event.

7. Axiomatic Foundations • A set of different combinations of outcomes is called an event. These events could be simple events or compound events. In the Texas lotto case, the important aspect is that the event is something you could bet on (for example, you could bet on three numbers in the draw {35,20,15}).

8. A collection of events F is called a family of subsets of sample space Ω. This family consists of all possible subsets of Ω including Ω itself and the null-set Φ. • Following the betting line, you could bet on all possible numbers (covering the board) so that Ω is a valid bet. • Alternatively, you could bet on nothing, or Φ is a valid bet.

9. Next, we will examine a variety of closure conditions. These are conditions that guarantee that if one set is an contained in a family, another related set must also be contained in that family. • First, we note that the family is closed under complementarity

10. Second, we note that the family is closed under union • Definition 1.1 (Bierens): A collection F of subsets of a nonempty set Ω satisfying closure under complementarity and closure under union is called an algebra. • Adding closure under infinite union defined as

11. Definition 1.2 (Bierens): A collection F of subsets of a nonempty set Ω satisfying closure under complementarity and infinite union is called a σ-algebra (sigma-algebra) or a Borel Field.

13. We typically think of this as an odds function (i.e., what are the odds of a winning lotto ticket? 1/15,890,700). • To be mathematically precise, suppose we define a set of events say that we choose n different numbers. The probability of winning the lotto is

14. Our intuition would indicate that , or the probability of winning given that you have covered the board is equal to one (a certainty). • Further, if you don’t bet the probability of winning is zeros or

15. Definition 1.2.2 (Cassella and Berger) : Given a sample space Ω and an associated Borel field B, a probability function is a function P with domain B that satisfies • P (A)0 for all AB. • P (Ω)=1. • If A1,A2,…B are pairwise disjoint, then P(i=1Ai)=i=1P (Ai)

16. Axioms of Probability: • P(A)  0 for any event A. • P(S) = 1 where S is the sample space. • If {Ai}, i=1,2,…, are mutually exclusive (that is, AiAj= for all ij), then P(A1A2…)=P(A1)+P(A2)+…

17. In a little more detail from Casella and Berger: • Definition 1.1.1: The set, S, of all possible outcomes of a particular experiment is called the sample space for the experiment. • Definition 1.1.2: An event is any collection of possible outcomes of an experiment, that is, any subset of S (including S itself).

18. Defining the subset relationship • A  B  x  A  xB • A = B  A  B and B  A • Union: The union of A and B, written AB, is the set of elements that belong to either A or B. • Intersection: The intersection of A and B, written AB, is the set of elements that belong to both A and B.

19. Complementation: The complement of A, written Ac, is the set of all elements that are not in A.

20. Theorem 1.1.1: For any three events A, B, and C defined on a sample space S, • Commutativity: A  B=B  A, A  B=B  A. • Associativity: A  (B  C)=(A  B)  C, A  (B  C)=(A  B)  C • Distributative Laws: A (B C )=(A B )(A  C ),A (B C )=(A  B )(A C ) • DeMorgan’s Laws: (A B )c=Ac  Bc, (A B )c=Ac Bc

21. Counting Techniques • Simple Evens with Equal Probabilities the probability of event A is simply the number possible occurrences of A divided by the number of possible occurrences in the sample.

22. Definition 2.3.1 The number of permutations of taking r elements from n elements is a number of distinct ordered sets consisting of r distinct elements which can be formed out of a set of n distinctive elements and is denoted Pnr.

23. The first point to consider is that of factorials. For example, if you have two objects A and B, how many different ways are there to order the object? Two: {A, B} or {B, A} • If you have three orderings how many ways are there to order the objects? Six: {A, B, C}, {A, C, B}, {B, A, C}, {B, C, A}, {C, A, B}, or {C, B, A}

24. The sequence then becomes two objects can be drawn in two sequences, three objects can be drawn in six sequences (2 x 3). By inductive proof, four objects can be drawn in 24 sequences (6 x 4). • The total possible number of sequences is then for n objects is n! defined as: n!=n (n -1)(n -2)…1

25. Theorem 2.3.1: Pnr=n!/(n-r)!. • Definition 2.3.2: The number of combinations of taking r elements from n elements is the number of distinct sets consisting of r distinct elements which can be formed out of a set of n distinct elements and is denoted Cnr.

26. Conditional Probability and Independence • In order to define the concept of a conditional probability it is necessary to discuss joint probabilities and marginal probabilities. • A joint probability is the probability of two random events. For example, the draw of two cards from a deck of cards. There are 52x51=2652 different combinations of the first two cards from the deck.

27. The marginal probability is overall probability of a single event or the probability of drawing a given card. • The conditional probability of an event is the probability of that event given that some other event has occurred. • In the textbook, what is the probability of the die being a one if you know that the face number is odd? (1/3). • Note if you know that the role of the die is a one, then the probability of the role being odd is 1.

28. Axioms of Conditional Probability: • P (A|B )0 for any event A. • P (A|B )=1 for any event A  B. • If {AiB}, i=1, 2,… are mutually exclusive, then P(A1A2…|B )=P(A1|B )+P(A2|B)+…. • If B H and B G and P (G )0, then

29. Bayes Theorem • Theorem 2.4.1: P (A|B )=P (AB )/P (B) for any pair of events A and B such that P (B)>0. • Theorem 2.4.2 (Bayes Theorem): Let Events A1, A2, …, An be mutually exclusive such that P (A1A2…An)=1 and P (Ai) >0 for each i. Let E be an arbitrary event such that P (E)>0. Then

30. Another manifestation of this theorem is from the joint distribution function: • The bottom equality reduces the marginal probability of event E

31. This yields a friendlier version of Bayes theorem based on the ratio between the joint and marginal distribution function:

32. Statistical Independence • Statistical independence is when the probability of one random variable is independent of the probability of another random variable. • Definition 2.4.1: Events A and B are said to be independent if P (A)=P (A|B ).

33. Definition 2.4.2: Events A, B, and C are said to be mutually independent if the following equalities hold: • P (AB )=P (A )P (B ) • P (AC )=P (A )P (C ) • P (BC )=P (B )P (C ) • P (AB C )=P (A )P (B)P (C )