A patient comes into a doctor’s office with a fever and a bad cough. Hypothesis space H :

1. Bayesian NetworksTextbook: Probabilistic Reasoning, Sections 1-2, pp.168-175; Section 4.1, pp. 180-181; Section 5, pp. 188-196 S. Wooldridge, Bayesian Belief Networks(linked from course webpage)

3. A patient comes into a doctor’s office with a fever and a bad cough. Hypothesis space H: h1: patient has flu h2: patient does not have flu Data D: coughing= true, fever = true, smokes = true

4. Naive Bayes Cause cough fever flu Effects

5. What if attributes are not independent? cough fever flu

6. What if more than one possible cause? smokes cough fever flu

7. Full joint probability distribution smokes Sum of all boxes is 1. In principle, the full joint distribution can be used to answer any question about probabilities of these combined parameters. However, size of full joint distribution scales exponentially with number of parameters so is expensive to store and to compute with.  smokes

8. Full joint probability distribution smokes For example, what if we had another attribute, “allergies”? How many probabilities would we need to specify?  smokes

9. Allergy Allergy smokes smokes Allergy Allergy smokes smokes

10. Allergy Allergy smokes smokes Allergy Allergy smokes smokes But can reduce this if we know which variables are conditionally independent

11. Bayesian networks • Idea is to represent dependencies (or causal relations) for all the variables so that space and computation-time requirements are minimized. Allergies smokes cough fever flu “GraphicalModels”

13. Bayesian Networks = Bayesian Belief Networks = Bayes Nets Bayesian Network: Alternative representation for complete joint probability distribution “Useful for making probabilistic inference about models domains characterized by inherent complexity and uncertainty” Uncertainty can come from: • incomplete knowledge of domain • inherent randomness in behavior in domain

14. Example: cough Conditional probability tables for each node flu smoke smoke flu flu smoke cough fever fever flu

15. Inference in Bayesian networks • If network is correct, can calculate full joint probability distribution from network. where parents(Xi) denotes specific values of parents of Xi.

16. Example • Calculate

17. In general... • If network is correct, can calculate full joint probability distribution from network. where parents(Xi) denotes specific values of parents of Xi. But need efficient algorithms to do this (e.g., “belief propagation”, “Markov Chain Monte Carlo”).

18. Example from the reading:

19. What is the probability that Student A is late? What is the probability that Student B is late?

20. What is the probability that Student A is late? What is the probability that Student B is late? Unconditional (“marginal”) probability. We don’t know if there is a train strike.

21. Now, suppose we know that there is a train strike. How does this revise the probability that the students are late?

22. Now, suppose we know that there is a train strike. How does this revise the probability that the students are late? Evidence: There is a train strike.

23. Now, suppose we know that Student A is late. How does this revise the probability that there is a train strike? How does this revise the probability that Student B is late? Notion of “belief propagation”. Evidence: Student A is late.

24. Now, suppose we know that Student A is late. How does this revise the probability that there is a train strike? How does this revise the probability that Student B is late? Notion of “belief propagation”. Evidence: Student A is late.

25. Another example from the reading: pneumonia smoking temperature cough

26. What is P(cough)?

27. Three types of inference • Diagnostic: Use evidence of an effect to infer probability of a cause. • E.g., Evidence: cough=true. What is P(pneumonia| cough)? • Causal inference: Use evidence of a cause to infer probability of an effect • E.g., Evidence: pneumonia=true. What is P(cough| pneumonia)? • Inter-causal inference: “Explain away” potentially competing causes of a shared effect. • E.g., Evidence: smoking=true. What is P(pneumonia| cough and smoking)?

28. Diagnostic: Evidence: cough=true. What is P(pneumonia| cough)? pneumonia smoking temperature cough

29. Diagnostic: Evidence: cough=true. What is P(pneumonia| cough)? pneumonia smoking temperature cough

30. Causal: Evidence: pneumonia=true. What is P(cough| pneumonia)? pneumonia smoking temperature cough

31. Causal: Evidence: pneumonia=true. What is P(cough| pneumonia)? pneumonia smoking temperature cough

32. Inter-causal: Evidence: smoking=true. What is P(pneumonia| cough and smoking)? pneumonia smoking temperature cough

33. “Explaining away”

34. Math we used: • Definition of conditional probability: • Bayes Theorem • Unconditional (marginal) probability • Probability inference in Bayesian networks:

35. In-Class Exercise

36. Complexity of Bayesian Networks For n random Boolean variables: • Full joint probability distribution: 2n entries • Bayesian network with at most k parents per node: • Each conditional probability table: at most 2kentries • Entire network: n 2k entries

37. What are the advantages of Bayesian networks? • Intuitive, concise representation of joint probability distribution (i.e., conditional dependencies) of a set of random variables. • Represents “beliefs and knowledge” about a particular class of situations. • Efficient (approximate) inference algorithms • Efficient, effective learning algorithms

38. Issues in Bayesian Networks • Building / learning network topology • Assigning / learning conditional probability tables • Approximate inference via sampling

39. Real-World Example: The Lumière Project at Microsoft Research • Bayesian network approach to answering user queries about Microsoft Office. • “At the time we initiated our project in Bayesian information retrieval, managers in the Office division were finding that users were having difficulty finding assistance efficiently.” • “As an example, users working with the Excel spreadsheet might have required assistance with formatting “a graph”. Unfortunately, Excel has no knowledge about the common term, “graph,” and only considered in its keyword indexing the term “chart”.

41. Networks were developed by experts from user modeling studies.

43. Offspring of project was Office Assistant in Office 97, otherwise known as “clippie”. http://www.youtube.com/watch?v=bt-JXQS0zYc

44. The famous “sprinkler” example(J. Pearl, Probabilistic Reasoning in Intelligent Systems, 1988)

45. Recall rule for inference in Bayesian networks: Example: What is

46. In-Class Exercise

47. More Exercises What is P(Cloudy| Sprinkler)? What is P(Cloudy| Rain)? What is P(Cloudy| Wet Grass)?

48. More Exercises What is P(Cloudy| Sprinkler)?

49. More Exercises What is P(Cloudy| Rain)?

50. More Exercises What is P(Cloudy| Wet Grass)?