Statistical Independence and An Invitation to The Art of Conditioning

1. Statistical IndependenceandAn Invitation to The Art of Conditioning Tutorial 2STAT1301 Fall 2010 28SEP2010, MB103@HKUBy Joseph Dong

2. “Look, imagine a remote village where there has been a long drought. One day, local peasants in desperation go to the church, and the priest says a prayer for rain. And the next day the rain arrives! These are independent events.” Legendary answer made by Kolmogorov when he was questioned by a senior Soviet Minister about whether the concept of independent events of his probability theory is consistent with materialistic determinism. What is the intuitive background of Independence? How is the concept of independence related from that of mutual exclusiveness?

3. Motivating Concept Independence • The event of today’s raining is by no means affected by the prayer said at the church yesterday. • How do you represent this wording in symbolic language so that you can analyze it in a more precise manner? • Using the language of event (set): • The realization of the event A={it rains today} is by no means affected by the realization of event B={villagers prayed yesterday}. • If B has already realized, A will have an unaffected probability of realizing. • For convenience we use a notation to denote the probability of A’s realization given B’s realization. Then the foregoing motivation for the concept independence will require us to establish the precise meaning of in a symbolic way. • A and B independent ??

4. Independence & Mutual Exclusiveness • Two events are mutually exclusive if there is no possibility that they can happen together (simultaneously). • Mutual exclusiveness is a concept/phenomenon on the set theory layer. • Independence is a concept/phenomenon on the probability(measure) theory layer. • Two events are independent if … (see next slide)

5. Independence: • A formal, less intuitive, precise, and deep definition: Two events are independent if A = {The villagers prayed yesterday}, , B = {It rains today}, Now we just need to specify one more probability to establish independence! If events (A, B) are independent, so are events (A, Bc), (Ac, B), and (Ac,Bc). Practice: handout problem 3.

6. Now Show algebraically… • If the events , are independent, show that the pairs , , also consist of independent events. • If the events , , and are completely independent (mutually independent), then the pairs , , also consist of independent events. • But not the reverse: • R.E. = Tossing a coin twice. • A = {1st toss shows H}, B = {2nd toss shows H}, C = {both tosses show the same side}

7. More On Mutual Independence • Intrinsic additive relations: • the probabilities of the 4 vertices of a face add to the probability of the face. • the probabilities of the 2 vertices of an edge add to the probability of the edge. • Mutual independence require us to establish all multiplicative relations among vertices, edges, and faces.

8. More On Mutual Independence (cont’d) • Any two of the following three can derive the third. • How to specify the multiplicative relations of the probabilities of a minimal set of vertices, edges, and faces? • Solution 1: 4 vertices • Solution 2: ?

9. Another related source of perspective • Are the first two columns of digits independent? • What about A and Bc? Ac and Bc? • What about second and third? • What about columns A & B together and column C? If you are allowed to add more rows, how could you make them independent? • What about the first and last columns? • What about the first and last columns together and column C?

10. A Concrete Example • R.E. = Tossing a die once. • Event • Event • What’s and ? • If you think A and B are not independent, then what can you say about ? • Actually • Are you surprised? • Can you give 3 more pairs of events that are independent?

11. Conditional Probability (Total: conditional probabilities.) Given an event (and of course also its sample space whereby it’s contained), the probability conditional on event , satisfies Kolmogorov’s Axioms of Probability.

12. Formal Definition: • Conditional Probability is a valid probability: • (“conditional sample space”) • If ’s are disjoint events(subsets of ), then ? • Exercise: Prove the identity • Exercise: Prove the identity • Exercise: Prove • Practice: Handout Problems 1 and 2.

13. Using Conditional Probability On its own • In English, we use “if”, “when”, and the alike to indicate a condition. • In Probabilistic analysis, we use the vertical bar “|” to indicate a condition. • If she says yes, we’ll adopt plan A; if she says no or is indifferent, we’ll go with plan B. • If the coin turns a head, I’ll go basketball tonight; if tail, 50% chance I’ll read a book chapter on Bayes Theory, and 50% chance I’ll go to bed early. • If the airplane crashes in the prairie, then there is a high probability to find it; if in the mountain, then medium probability to find it; if in the sea, then very low probability to find it. • Iftomorrow the announcement of last season’s financial result is above public expectation, then the stock price will go up with probability 0.89; if equals the expectation, the stock price will go up with probability 0.5; if below expectation, the stock price will go up with probability 0.08.

14. Partition (of Sample space)and Condition (on the Partition) • Formal Def. of a Partition of a Set. • Any disjoint and exhaustive collectionof subsets of a given set forms a partition of that set. • A trivial partition of : For each subset , is a partition of . • Throwing a die 10 times and we are interested in the sequence of the ten outcomes. Then If , then Each is a subset of The six s are disjoint The six s together are exhaustive. The six s together is a partition of .

15. Exercise • We work in the sample space . Let the collection of events be a partition of . Also we have another event . Prove the following identity: • Further, prove the following additional identity:

16. Partition and ConditionMonty Hall revisited – Layman edition • You have made your first choice at box A, hoping that it contains the key to unlock a brand new BMW. Then the host opens box B and it’s empty. Now it is your decision to make on whether you will switch to choose C or stay with A. • What particular partition to form, and hence what conditions to make? • In a probability veteran’s language, it is said that we would “condition on A” • Meaning we first identify the event A as . • Then form the trivial partition of the sample space. • Then you proceed the analysis by making a condition “if event X happens, then” for each event X in the partition. • Here… • if event happens (ie, …), and with the knowledge is empty, then must also be empty. • if event happens (ie, …), and with the knowledge is empty, then must contain the key. • Now it is your turn to reach the end of this thread …

17. Question to the layman edition • What subtlety will be involved if, unfortunately, you “conditioned on C”, instead of A?!

18. Partition and ConditionMonty Hallrevisited – Expert edition • Want: • Want = Now it’s your turn: mimic an expert solution for the “Three Prisoners Problem”. • Brief setup: Let , where • Recall an observation made in tutorial 1: What are the two events involved when the host, seeing you have chosen box A, shows you the emptiness of box B? • Let • Known:

19. Exercise Handout Problem 4 Handout Problem 5 • Require a clever conditioning (although the hint has effectively eliminated this requirement) and some mathematical maturity • What events are involved in the statement: • “The initial search was in the mountains, and the plane was not found.”