1 / 61

Independence

Independence. X, Y are independent if changing X does not affect the probability of Y, and vice versa. Pr(X, Y) = Pr(X)∙Pr(Y) Pr(X | Y) = Pr(X) Pr(Y | X) = Pr(Y). Independent or Dependent?. Doritos for dinner, getting the flu Miles per gallon, driving habits

yardley-carver
Download Presentation

Independence

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Independence • X, Y are independent if changing X does not affect the probability of Y, and vice versa. • Pr(X, Y) = Pr(X)∙Pr(Y) • Pr(X | Y) = Pr(X) • Pr(Y | X) = Pr(Y)

  2. Independent or Dependent? • Doritos for dinner, getting the flu • Miles per gallon, driving habits • Alice and Bob tossing two separate coins • Alice and Bob tossing the same coin

  3. Independent or Dependent? • Doritos for dinner, getting the flu • Independent • Miles per gallon, driving habits • Dependent • Alice and Bob tossing two separate coins • Independent • Alice and Bob tossing the same coin • Dependent?

  4. Alice and Bob toss the same coin • Coin may be biased • If Alice’s toss is Heads, more likely that Bob’s toss will be Heads.

  5. Alice and Bob toss the same coin • Coin may be biased • If Alice’s toss is Heads, more likely that Bob’s toss will be Heads. • The coin tosses are dependent on what separate event?

  6. Alice and Bob toss the same coin • Coin may be biased • If Alice’s toss is Heads, more likely that Bob’s toss will be Heads. • The coin tosses are dependent on a separate event: Coin is biased. • Given that the coin is biased, Alice’s toss is independent from Bob’s toss.

  7. Alice and Bob toss the same coin • A = outcome of Alice’s toss • B = outcome of Bob’s toss • C = event that the coin is biased towards Heads

  8. Alice and Bob toss the same coin • A = outcome of Alice’s toss • B = outcome of Bob’s toss • C = event that the coin is biased towards Heads • A and B areconditionally independent given C if: • Pr(A| C Λ B)=Pr(A | C) • Pr(B| C ΛA)=Pr(B | C)

  9. Bayes Networks • Somewhat simplified • Few assumptions • A bit mathy

  10. Joint Probability Distributions

  11. Joint Probability Distributions For n variables, how many probabilities?

  12. Joint Probability Distributions For n variables, how many probabilities? 2n

  13. Joint Probability Distributions For n variables, how many probabilities? 2n Joint Probability Distributions require a lot of storage.

  14. Bayesian Network • Data Structure that represents dependencies among random variables of a joint probability distribution.

  15. Bayesian Network • Data Structure that represents dependencies among random variables of a joint probability distribution. • Directed Acyclic Graph with nodes and edges: • Node for every variable X

  16. Bayesian Network • Data Structure that represents dependencies among random variables of a joint probability distribution. • Directed Acyclic Graph with nodes and edges: • Node for every variable X • Edge (Y, X) from Y to X if X depends on Y  “Y is a parent of X” Y X

  17. Bayesian Network for Flu example (on Board)

  18. Bayesian Network for Flu example (on Board) Bayes Network contains 5 probabilities instead of 8

  19. Find Probability of an Event (B) Y • Simplest Case (Case 1): • B has one parent, Y • Y has no parents • Y has possible values , , …, B

  20. Find Probability of an Event (B) Y • Simplest Case (Case 1): • B has one parent, Y • Y has no parents • Y has possible values , , …, • Take conditional probability of B over all possible values of Y B

  21. Find Probability of an Event (B) Y • Simplest Case (Case 1): • B has one parent, Y • Y has no parents • Y has possible values , , …, • Take conditional probability of B over all possible values of Y • Notation: Pr(B) = P(B, Y=) + P(B, Y=) + …+ P(B, Y=) B

  22. Find Probability of an Event (B) Y • Simplest Case (Case 1): • B has one parent, Y • Y has no parents • Y has possible values , , …, • Take conditional probability of B over all possible values of Y • Notation: Pr(B) = P(B, Y=) + P(B, Y=) + …+ P(B, Y=) = Pr(B | Y=)Pr(Y=)+ Pr(B | Y=)Pr(Y=)+ …+ Pr(B | Y=)Pr(Y=) B

  23. Find Probability of an Event (B) Y • Simplest Case (Case 1): • B has one parent, Y • Y has no parents • Y has possible values , , …, • Take conditional probability of B over all possible values of Y • Notation: Pr(B) = P(B, Y=) + P(B, Y=) + …+ P(B, Y=) = Pr(B | Y=)Pr(Y=)+ Pr(B | Y=)Pr(Y=)+ …+ Pr(B | Y=)Pr(Y=) = B

  24. Example-Flu Network Probability of Sore Throat?

  25. Example-Flu Network Probability of Sore Throat? Flu is parent of Sore Throat. Compute Probability of Sore Throat over all possible values of Flu.

  26. Example-Flu Network Probability of Sore Throat? Flu is parent of Sore Throat. Compute Probability of Sore Throat over all possible values of Flu: Pr(S) = P(S | F=)Pr(F=) + P(S | F=) Pr(F=) + … +P(S | F=) Pr(F=)

  27. Example-Flu Network Probability of Sore Throat? Flu is parent of Sore Throat. Compute Probability of Sore Throat over all possible values of Flu: Pr(S) = P(S | F=)Pr(F=) + P(S | F=) Pr(F=) + … +P(S | F=) Pr(F=) Possible values of F (Flu)?

  28. Example-Flu Network Probability of Sore Throat? Flu is parent of Sore Throat. Compute Probability of Sore Throat over all possible values of Flu: Pr(S) = P(S | F=)Pr(F=) + P(S | F=) Pr(F=) + … +P(S | F=) Pr(F=) Possible values of F (Flu)? Just True/False Pr(S) = Pr(S | F)Pr(F) + Pr(S | ⌐ F)Pr(⌐ F) = (.12)(.2) + (.08)(.8)

  29. Case 2: B has multiple parents Y X Z What if B has multiple parents? B

  30. Case 2: B has multiple parents Y X Z What if B has multiple parents? X, Y, Z are parents of B with possible values , , …, , , …, , , …, Assume X, Y, Z are independent of each other B

  31. Case 2: B has multiple parents Y X Z What if B has multiple parents? X, Y, Z are parents of B with possible values , , …, , , …, , , …, Assume X, Y, Z are independent of each other Consider all possible combinations of X, Y, Z. B

  32. Probability of an Event (B) Pr(B) = P(B, X=Y=) + P(B, X=Y=) + P(B, X=Y=) + …+ P(B, X=Y=)

  33. Probability of an Event (B) Pr(B) = P(B, X=Y=) + P(B, X=Y=) + P(B, X=Y=) + …+ P(B, X=Y=) = Pr(B|X=Y=)Pr(X=Y=)+ Pr(B|X=Y=)Pr(X=Y=)+ Pr(B|X=Y=)Pr(X=Y=)+…+ Pr(B|X=Y=)Pr(X=Y=)

  34. More Complex Example • You install new burglar alarm • Reliable at detecting burglary but also rings occasionally during a minor earthquake • Two neighbors Jack and Mary who promise to call you if they hear the alarm • Jack almost always calls when he hears alarm but occasionally confuses phone with alarm and sometimes calls when he hears the phone • Mary is hard of hearing: occasionally does not hear the alarm

  35. But First…How does this relate to AI? Want to create intelligent agents that can make rational decisions given uncertainty. Jack and Mary clearly lack uncertainty.

  36. More Complex Example • You install new burglar alarm • Reliable at detecting burglary but also rings occasionally during a minor earthquake • Two neighbors Jack and Mary who promise to call you if they hear the alarm • Jack almost always calls when he hears alarm but occasionally confuses phone with alarm and sometimes calls when he hears the phone • Mary is hard of hearing: occasionally does not hear the alarm

  37. Burglary Alarm Example • (Bayes Network on the board) • Probability that alarm goes off?

  38. Pr(A) = P(A, B, Q) How many possible combinations of B, Q?

  39. Pr(A) = P(A, B, Q) How many possible combinations of B, Q? 4

  40. Pr(A) = P(A, B, Q) How many possible combinations of B, Q? 4 = Pr(A|B Λ Q)Pr(B Λ Q) + Pr(A|B Λ ⌐Q)Pr(B Λ ⌐Q) + Pr(A| ⌐ B Λ Q)Pr(⌐ B Λ Q) + Pr(A| ⌐ B Λ ⌐ Q)Pr(⌐ B Λ ⌐ Q)

  41. Pr(A) = P(A, B, Q) How many possible combinations of B, Q? 4 = Pr(A|B Λ Q)Pr(B Λ Q) + Pr(A|B Λ ⌐Q)Pr(B Λ ⌐Q) + Pr(A| ⌐ B Λ Q)Pr(⌐ B Λ Q) + Pr(A| ⌐ B Λ ⌐ Q)Pr(⌐ B Λ ⌐ Q) =(.95)

  42. Pr(A) = P(A, B, Q) How many possible combinations of B, Q? 4 = Pr(A|B Λ Q)Pr(B Λ Q) + Pr(A|B Λ ⌐Q)Pr(B Λ ⌐Q) + Pr(A| ⌐ B Λ Q)Pr(⌐ B Λ Q) + Pr(A| ⌐ B Λ ⌐ Q)Pr(⌐ B Λ ⌐ Q) =(.95)(.001)(.002)

  43. Pr(A) = P(A, B, Q) How many possible combinations of B, Q? 4 = Pr(A|B Λ Q)Pr(B Λ Q) + Pr(A|B Λ ⌐Q)Pr(B Λ ⌐Q) + Pr(A| ⌐ B Λ Q)Pr(⌐ B Λ Q) + Pr(A| ⌐ B Λ ⌐ Q)Pr(⌐ B Λ ⌐ Q) =(.95)(.001)(.002) + (.94)(.001)(.998) +

  44. Pr(A) = P(A, B, Q) How many possible combinations of B, Q? 4 = Pr(A|B Λ Q)Pr(B Λ Q) + Pr(A|B Λ ⌐Q)Pr(B Λ ⌐Q) + Pr(A| ⌐ B Λ Q)Pr(⌐ B Λ Q) + Pr(A| ⌐ B Λ ⌐ Q)Pr(⌐ B Λ ⌐ Q) =(.95)(.001)(.002) + (.94)(.001)(.998) + (.29)(.999)(.002) + (.001)(.999)(.998)

  45. How to find probability of 2 events A, B? Assume A and B are independent.

  46. How to find probability of 2 events A, B? Assume A and B are independent. Suppose X, Y, Z are parents of A with possible values: X: Y: Z: P, Q, R are parents of B with possible values: P: Q: R: Y Q X Z P R A B

  47. How to find probability of 2 events A, B? Assume A and B are independent. Suppose X, Y, Z are parents of A with possible values: X: Y: Z: P, Q, R are parents of B with possible values: P: Q: R: Consider all possible combinations of X, Y, Z, P, Q, R Y Q X Z P R A B

  48. X, Y, Z, P, Q, R can each take on 3 values. How many possible combinations?

  49. X, Y, Z, P, Q, R can each take on 3 values. How many possible combinations? 36 E1 E2 … E36 Pr(A, B) = P(A, B, E1) + P(A, B, E1) + … P(A, B, E36)

  50. Case 3: “Chain” of Parents Y What if B has a parent X, and X has a parent Y? X B

More Related