Bayes Theorem

1. B5 B4 B2 B3 B1 B6 Bayes Theorem • Mutually exclusive events A collection of events (B1, B2, …, Bk) is said to be mutually exclusive if no two of them overlap. If mutually exclusive events together add up to the entire sample space, i.e. cover sample space, they are called collectively exhaustive events. They partition the sample space, like walls partition a house. Example. B1, …, B6 Events B1 – B6 partition sample space.

2. Total Probability Formula Take any event A. A is the sum of its intersections with Bs. P(A) = P(A and B1) + P(A and B2) + P(A and B3) + …+ P(A and Bk) Write P(A and Bi) using conditional probabilities: Total Probability Formula P(A) = P(A|B1)P(B1) + P(A|B2)P(B2) + … + P(A|Bk)P(Bk). B5 B4 B2 A B3 B6 B1

3. Example A chip manufacturing plant has 3 machines producing chips. Machine 1 produces 30% of the output and of these, 2% are defective; machine 2 produces 45% of the output and of these, 1% are defective; machine 3 produces the remaining 25% chips and of these 3% are defective. Find the probability that a randomly selected chip produced by this plant is defective. Solution. Define events. A= randomly selected chip is defective; B1=chip was produced by machine 1, P(B1)=0.3; B2=chip was produced by machine 2, P(B2)=0.45; B3=chip was produced by machine 3, P(B3)=0.25. P(A|B1)=0.02, P(A|B2)=0.01, P(A|B3)=0.03. By the Total Probability Formula: P(A)= P(A|B1)xP(B1) + P(A|B2)xP(B2) + P(A|B3)xP(B3)= = 0.02 x 0.3 + 0.01 x 0.45 + 0.03 x 0.25= 0.018.

4. Bayes Formula Bayes formula goes “the other way” from the Total Probability Formula. We look for conditional probabilities of the sets from the partition. Bayes Formula P(Bi | A) = P(A and Bi)/P(A)= P(A|Bi)xP(Bi) = ----------------------------------------------------------------------- . P(A|B1)P(B1) + P(A|B2)P(B2) + … + P(A|Bk)P(Bk) Bayes formula provides conditional probability of Bi given A in terms of the other conditional probabilities, i.e. P(A given Bi).

5. EXAMPLE Chips manufacturing example continued. New question. If you bought a defective chip produced by that factory, what is the probability that it was produced by machine 3? Solution. Need P(B3|A). By Bayes formula, P(A|B3)xP(B3) P(B3|A) = ----------------------------------------------------------------- = P(A|B1)P(B1) + P(A|B2)P(B2) + P(A|B3)P(B3) 0.03 x 0.25 = ----------------------------------------------------------------- = 0.42 0.02 x 0.3 + 0.01 x 0.45 + 0.03 x 0.25