ITEC4310 Applied Artificial Intelligence

1. ITEC4310Applied Artificial Intelligence Lecture 3 Probability and Bayes Theorem

2. Outline • Simple probability • Introduction to Bayes theorem • Applications of Bayes theorem

3. Introductory Probability I • What is the probability of a heads or a tails when you flip a coin? • What is the probability of a given value (1-6) when you roll a single six-sided die?

4. Introductory Probability II • What is the probability of a given value (2-12) when you roll two six-sided dice? • What is the probability that you will win a lottery prize? • Encore • Lotto Max

5. Introductory Probability III • The probability of all possible events is 1 or 100% • The probability of any individual event is 0-1 or 0-100%

6. Introductory Probability IV • What is the probability of a heads or a tails when you flip a coin? • P(heads) = 0.5 • P(tails) = 0.5 • P(edge) = 0 • P(heads) + P(tails) + P(edge) = 1

7. Introductory Probability V • What is the probability of a given value (1-6) when you roll a single six-sided die? • P(1) = P(2) = P(3) = P(4) = P(5) = P(6) = 1/6 • “fair” coin/die

8. Introductory Probability VI • What is the probability of a given value (2-12) when you roll two six-sided dice?

9. Introductory Probability VII • Six outcomes for die 1 • Six outcomes for die 2 • Assuming dice are independent • 6 * 6 = 36 total possibilities • die 1 = 1, die 2 = 1, 2, 3, 4, 5, 6

10. Introductory Probability VIII • What is the probability that you will win a prize in Encore? • Last digit  1 in 10 • Listed as worse odds since you could win more! • Last 2 digits  1 in 100 • Jackpot  1 in 10,000,000

11. Introductory Probability IX • What is the probability that you will win a prize in Lotto Max? • “Choose” based probabilities • Beyond the scope of this introduction… • List all possible winning tickets over all possible tickets

12. Independent Events I • Probabilities for result of two dice assumes each die roll is independent • If the first die roll is a 1, what is the probability that the second die roll is a 1? • Do the winning numbers of today’s lottery draw affect the winning numbers of next week’s draw?

13. Independent Events II • Are real world events modelled as independent really independent? • If home owner 1 defaults, does this increase the risk that home owner 2 defaults? • CDOs and the 2007 US housing crisis

14. Related Events • Certain cancer rates are related to other health indicators • E.g. smoking, drinking, BMI, cholesterol • The result of rolling two dice can change the posterior probability of what the two dice each rolled • If I know the result was 5, what is the probability that die 1 rolled a 1?

15. Bayesian Probabilities I • Bayes' Theorem is about conditional probability • If something happens, what is the probability of something else happening?

16. Bayesian Probabilities II

17. Bayesian Probabilities III • For independent events • P(A) = P(A|B) • The probability of A does not change if B happens or not • For related events • P(A) != P(A|B) • The information of B changes our information on A. We can calculate a new probability

18. Bayesian Probabilities IV

19. Bayesian Probabilities V

20. Using Bayes I • What is the probability that the first die rolled was 1? • With no other information, P(1) = 1/6

21. Using Bayes II • What is the probability that the first die rolled was 1, if the total is 2? • Intuitively, we know this must be 100% because both dice must be 1 • P(A|B) = P(B|A) P(A) / P(B) • P(B|A) = 1/6  2 happens 1/6 times the first die is 1 • 1 = (1/6)(1/6) / (1/36)

22. Using Bayes III • What is the probability that the first die rolled was 1, if the total is 3? • Intuitively, we know this must be 50% because one must be 1 (and the other 2) • P(A|B) = P(B|A) P(A) / P(B) • P(B|A) = 1/6  3 happens 1/6 times the first die is 1 • 1/2 = (1/6)(1/6) / (2/36)

23. Using Bayes IV • What is the probability that the first die rolled was 1, if the total is 8? • Intuitively, we know this must be 0 the max total is 7 if the first die is 1 • P(A|B) = P(B|A) P(A) / P(B) • P(B|A) = 0  8 never happens if the first die is 1 • 0 = (0)(1/6) / (5/36)

24. Using Bayes V • Cancer screening tests • Rare disease • Imperfect screening test • Need to have lots of information from previous tests to determine occurrence of disease in general population and accuracy rates of test

25. Using Bayes VI • P(A|B) = P(B|A) P(A) / P(B|A) P(A) + P(B|A’) P(A’) • P(A|B) is the probability of having cancer (A) given a positive test result (B) • P(A) is probability of having cancer • P(B|A) is probability of true +ve test • P(B|A’) is probability of false +ve test

26. Using Bayes VII • P(A) = 0.0001 • 0.01% of general population has rare disease • P(B|A) = 0.99 • +ve test result occurs for 99% of people with the disease • P(B|A’) = 0.01 • +ve test result occurs for 1% of people without the disease

27. Using Bayes VIII • What’s the probability that you have the disease if you have a +ve test result? • P(B|A) P(A) = 0.99 * 0.0001 = 0.0099 • P(B|A’) P(A’) = 0.01 * 0.9999 = 0.009999 • P(A|B) = P(B|A) P(A) / P(B|A) P(A) + P(B|A’) P(A’) = 0.0099 / (0.0099 + 0.009999) ≈ 0.5

28. Using Bayes IX • Even with a very accurate test, you only have a 50% chance of the disease when it’s rare • Don’t screen general population – only higher risk groups

30. Resources • Bayes’ Theorem Calculator • https://betterexplained.com/articles/an-intuitive-and-short-explanation-of-bayes-theorem/