1 / 27

Bayesian Models

Bayesian Models. Agenda. Project WebCT Late HW Math Independence Conditional Probability Bayes Formula & Theorem Steyvers, et al 2003. Independence. Two events A and B are independent if the occurrence of A has no influence on the probability of the occurrence of B.

janina
Download Presentation

Bayesian Models

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Bayesian Models

  2. Agenda • Project • WebCT • Late HW • Math • Independence • Conditional Probability • Bayes Formula & Theorem • Steyvers, et al 2003

  3. Independence • Two events A and B are independent if the occurrence of A has no influence on the probability of the occurrence of B. • Independent: “It doesn’t matter who is elected president, the world will still be a mess.” • Not independent: “If candidate B is elected president, the probability that the world will be a mess is 99%. If candidate A is elected, the probability that the world will be a mess will be lowered to 98%.”

  4. Independence • A and B are independent if P(AB) = P(A) x P(B). • Independent: • Pick a card from a deck. • A = “The card is an ace”, P(A) = 1/13. • B = “The card is a spade”, P(B) = 1/4 • P(AB) = 1/13 x 1/4 = 1/52. • Not independent: • Draw two cards from a deck without replacement. • A = “The first card is a space”, P(A) = 1/4 • B = “The second card is a spade”, P(B) = 1/4 • P(AB) = (13 x 12) / (52 x 51) < 1/4 x 1/4.

  5. Conditional Probability • Example: • What is the probability that a husband will vote Democrat given that his wife does? • P(HusbandDemocrat|WifeDemocrat) • This is different from: • What is the probability that a husband will vote democrat? • What is the probability that a hustband and wife will vote democrat?

  6. Conditional Probability • P(B|A) is the conditional probability that B will occur given that A has occurred: P(B|A) = P(BA) / P(A). All possible events P(A) = A occurs P(BA) = A and B occur P(B) = B occurs

  7. Conditional Probability • Suppose we roll two dice • A = “The sum is 8” • A = {(2,6), (3,5), (4,4), (5,3), (6,2)} • P(A) = 5/36 • B = “The first die is 3” • B = {(3,1), (3,2), (3,3), (3,4), (3,5), (3,6)} • P(B) = 6/36 • AB = {(3,5)} • P(B|A) = (1/36)/(5/36) = 1/5.

  8. Conditional Probability 36 6 5 1 P(A) = 5/36 P(B) = 6/36 P(BA) = 1/36 P(B|A) = (1/36)/(5/36) = 1/5 P(B|A) = P(BA)/P(A)

  9. Bayes Formula Suppose families have 1, 2, or 3 children with 1/3 probability each. Bobby has no brothers. What is the probability he is an only child?

  10. Bayes Formula Suppose families have 1, 2, or 3 children with 1/3 probability each. Bobby has no brothers. What is the probability he is an only child? Let child1, child2, and child3 be the events that A family has 1, 2, or 3 children, respectively. Let boy1 be the event that a family has only 1 boy. Want to compute P(child1|boy1) = P(child1boy1)/P(boy1)

  11. Bayes Formula Suppose families have 1, 2, or 3 children with 1/3 probability each. Bobby has no brothers. What is the probability he is an only child? Want to compute P(child1|boy1) = P(child1boy1)/P(boy1) We need to compute P(child1boy1) and P(boy1)

  12. Bayes Formula We need to compute P(child1boy1) and P(boy1) Because P(B|C) = P(CB) / P(C), we can write: P(CB) = P(C) P(B|C). So, P(child1boy1) = P(child1)P(boy1| child1) = 1/3 x 1/2 = 1/6.

  13. Bayes Formula We need to compute P(child1boy1) and P(boy1) P(boy1) = P(child1boy1) + P(child2boy1) + P(child3boy1) We know P(child1boy1) = 1/6. Likewise, P(child2boy1) = 1/6 P(child3boy1) = 1/8 P(boy1) = 1/6 + 1/6 + 1/8

  14. Bayes Formula Suppose families have 1, 2, or 3 children with 1/3 probability each. Bobby has no brothers. What is the probability he is an only child? P(child1|boy1) = P(child1boy1)/P(boy1) = (1/6) / (1/6 + 1/6 + 1/8) = 4/11

  15. Bayes Formula Suppose families have 1, 2, or 3 children with 1/3 probability each. Bobby has no brothers. What is the probability he is an only child? 1 Child 120 2 Children 120 3 Children 120 1 boy 60 1 boy 60 1 boy 45

  16. Bayes Formula 1 Child 120 2 Children 120 3 Children 120 1 boy 60 1 boy 60 1 boy 45 P(child1|boy1) = P(child1boy1)/P(boy1) = (60/360) / ((60+60+45)/360) = 60/165 = 4/11

  17. Bayes Formula Event1 Event2 Eventn … Sub- event Sub- event Sub- event Known: P(Eventi), P(Eventi) = 1, and P(Subevent|Eventi) Compute: P(Event1|Subevent)

  18. Bayes Formula Event1 Event2 Eventn … Sub- event Sub- event Sub- event P(Event1|Subevent) = P(Event1Subevent) / P(Subevent) P(EventiSubevent) = P(Eventi)P(Subevent|Eventi) P(Subevent) =  P(EventiSubevent) =  P(Eventi)P(Subevent|Eventi)

  19. Bayes Formula Event1 Event2 Eventn … Sub- event Sub- event Sub- event P(Event1)P(Subevent|Event1) P(Event1|Subevent) = -----------------------------------------  P(Eventi)P(Subevent|Eventi)

  20. Bayes Formula 1% of the population has a disease. 99% of the people who have the disease have the symptoms. 10% who don’t have the disease have the symptoms. Let D = “A person has the disease”. Let S = “A person has the symptoms”. P(D) = .01 and so P(~D) = .99 P(S|D) = .99 and P(S|~D) = .10 What is P(D|S)?

  21. Bayes Formula P(D) P(S|D) P(D|S) = -------------------------------------------- P(D) P(S|D) + P(~D) P(S|~D) = (.01 x .99) / (.01 x .99 + .99 x .10) = .091

  22. Bayes Formula P(Event1)P(Subevent|Event1) P(Event1|Subevent) = -----------------------------------------  P(Eventi)P(Subevent|Eventi) If there are a very large portion of events, the denominator may be very hard to calculate. If, however, you are only interested in relative probabilities…

  23. Bayes Formula P(Event1)P(Subevent|Event1) P(Event1|Subevent) = -----------------------------------------  P(Eventi)P(Subevent|Eventi)  P(Event1|Subevent) P(Event1)P(Subevent|Event1) --------------------------- = ---------------------------------------- P(Event2|Subevent) P(Event2)P(Subevent|Event2) This is called the “odds”.

  24. Bayes Formula • Let’s say you have 2 hypotheses (or models), H1 and H2, under consideration. • The log odds of these two hypotheses given experimental data, D, is:

  25. Bayes Formula Posterior odds = Relative belief in 2 hypotheses given the data. Quantity of interest.

  26. Bayes Formula Prior odds = Relative belief in 2 hypotheses Before observing any data. Often assumed to be 1 (0 in log odds).

  27. Bayes Formula Likelihoods = Relative probability of the data, given the two hypotheses. Usually computed from your models…

More Related