1 / 20

Probabilistic Reasoning

Probabilistic Reasoning. Comp3710 Artificial Intelligence Computing Science Thompson Rivers University. Course Outline. Part I – Introduction to Artificial Intelligence Part II – Classical Artificial Intelligence Part III – Machine Learning Introduction to Machine Learning Neural Networks

jdomingue
Download Presentation

Probabilistic Reasoning

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Probabilistic Reasoning Comp3710 Artificial Intelligence Computing Science Thompson Rivers University

  2. Course Outline • Part I – Introduction to Artificial Intelligence • Part II – Classical Artificial Intelligence • Part III – Machine Learning • Introduction to Machine Learning • Neural Networks • Probabilistic Reasoning and Bayesian Belief Networks • Artificial Life: Learning through Emergent Behavior • Part IV – Advanced Topics Bayesian Networks

  3. Learning Objectives Bayesian Networks

  4. Unit Outline • Introduction • Bayesian networks • Summary Bayesian Networks

  5. Topics 1. Introduction • A systematic way, called Bayesian networks, to represent the world probabilistically – independence and conditional independence relationships • Application areas: • consumer help desks, • nuclear reactor diagnosis, • tissue pathology, • pattern recognition, • credit assessment, • computer network diagnosis, • ... • Representing knowledge in an uncertain domain • Semantics of Bayesian networks Example: Bayesian Networks

  6. 2. Bayesian networks • Definitions • Example – Weather, Cavity, Catch, and Toothache • Example – Burglary net • How to answer queries in the Burglary net • How to construct Bayesian networks Bayesian Networks

  7. 2. Definitions • Bayesian network, belief network, probabilistic network, causal network, or also called knowledge map: A simple, graphical notation for conditional independence assertions and hence for compact specification of full joint distributions • Syntax: • a set of nodes, one per random variable • a directed, acyclic graph (link means“a parent node influences its child nodes.”) • a conditional probability distribution for each node given its parents: P(Xi | Parents (Xi)) • In the simplest case, conditional probability distribution represented as a conditional probability table (CPT) giving the distribution over Xi for each combination of parent values Bayesian Networks

  8. 2. Example – Weather, Toothache, C, C • Considering Weather, Toothache, Cavity, and Catch, Weather is independent of the other variables. • [Q] What else does not get any influence from others? • Cavity directly influencesToothache and Catch. • Toothache and Catch are conditionally independent given Cavity. • Topology of network encodes conditional independence assertions: Bayesian Networks

  9. Example - Burglary • [Q] I'm at work, neighbor John calls to say my alarm is ringing, but neighbor Mary doesn't call. Sometimes it's set off by minor earthquakes. Is there a burglar? • [Q] Random variables? • Burglary, Earthquake, Alarm, JohnCalls, MaryCalls • Network topology reflects direct "causal" or “influence” knowledge. • [Q] In what way? Can you draw the network then? • A burglar can set the alarm off. • An earthquake can set the alarm off. • The alarm can cause Mary to call. • The alarm can cause John to call. Bayesian Networks

  10. No need to include P(e) [Q] What is P(Alarm=T)? P(a) = P(abe) + P(abe) + P(abe) +P(abe) = P(a|b,e) P(be) + P(a|b,e) P(be) + P(a|b,e) P(be) + P(a|b,e)P(be) = ??? P(a | b,e) P(a | b,e) = ??? John calls to say my alarm is ringing. [Q] What does it mean? [Q] What is P(JohnCalls=T)? Bayesian Networks

  11. [Q]What is P(Alarm=T)? P(a) = P(abe) + P(abe) + P(abe) +P(abe) = P(a|b,e) P(be) + P(a|b,e) P(be) + P(a|b,e) P(be) + P(a|b,e)P(be) = P(a|b,e) P(b)P(e) + … = ??? P(a | b,e) P(a | b,e) = ??? John calls to say my alarm is ringing. [Q]What is P(JohnCalls=T)? P(j) = P(j  a) + ??? = ??? + ??? = ??? Bayesian Networks

  12. [Q] What is P(Alarm=T)? P(a) = P(abe) + P(abe) + P(abe) +P(abe) = P(a|b,e) P(be) + P(a|b,e) P(be) + P(a|b,e) P(be) + P(a|b,e)P(be) = P(a|b,e) P(b)P(e) + … = ??? P(a | b,e) John calls to say my alarm is ringing. [Q] What is P(JohnCalls=T)? P(j) = P(j  a) + P(j  a) = P(j | a) P(a) + P(j | a) P(a) = ??? Bayesian Networks

  13. 2. Compactness • A CPT for Boolean Xi with k Boolean parents has 2k rows for the combinations of parent values. • Each row requires one number p for Xi = true(the number for Xi = false is just 1 - p) • If each variable has no more than k parents, the complete network requires O(n · 2k) numbers • I.e., grows linearly with n, vs. O(2n)for the full joint distribution • But, for burglary net, 1 + 1 + 4 + 2 + 2 = 10 numbers (vs. 25-1 = 31) Bayesian Networks

  14. How to answer queries in Burglary net The full joint distribution is defined as the product of the local conditional distributions: P(X1, …,Xn) = Πi = 1P(Xi | Parents(Xi)) [Q] P(j  m  a b e) = ??? = P(j | a) P(m | a) P(a | b, e) P(b) P(e) = .9 * .7 * .001 * .999 * .998 ≈ .00063 n Bayesian Networks

  15. [Q] I'm at work, neighbor John calls to say my alarm is ringing, but neighbor Mary doesn't call. Sometimes it's set off by minor earthquakes. What is the chance that the alarm was ringing and there is a burglar? P(j m  ab) = ??? [Q] How to deal with Earthquake? = P(j m  a  b  e) + P(j m  a  b  e) = P(j | a)P(m | a)P(a | b,e) P(b) P(e) + P(j | a)P(m | a) (P(a | b,e) P(b) P(e) = P(j | a)(1 - P(m | a))P(a | b,e) P(b) P(e) + P(j | a)P(m | a) (P(a | b,e) P(b) P(e) = ??? [Q] Mary calls to say alarm is ringing. what is the chance that there was an earthquake? P(X1, …,Xn) = P(Xi | Parents(Xi)) Bayesian Networks

  16. [Q] P(a) = ??? = P(a  b  e) + P(a  b  e) + P(a b  e) + P(a b  e) = P(a | b,e) P(b) P(e) + P(a | b,e) P(b) P(e) + P(a | b,e) P(b) P(e) + P(a | b,e) P(b) P(e)) = … [Q] P(m) = ??? P(X1, …,Xn) = P(Xi | Parents(Xi)) Bayesian Networks

  17. 2. • [Q] How to implement for queries? Bayesian Networks

  18. How to construct Bayesian networks • 1. Choose an ordering of random variables X1, …,Xn • 2. For i = 1 to n • add Xi to the network • select parents from X1, …,Xi-1 such that P (Xi | Parents(Xi)) = P (Xi | X1, ..., Xi-1) i.e., Parents(Xi) are conditionally independent of the others. This choice of parents guarantees: P (X1, …,Xn) = Πi =1P(Xi | X1, …, Xi-1) (by chain rule) = Πi =1P (Xi | Parents(Xi)) (by construction) [Q] Which random variables first? n n Bayesian Networks

  19. 2. How to choose a correct order? Topics • “Root causes” first, • [Q] What can be a root? • then the variables they directly influence, • and so on, • until the leaves • Can we use causal relations? • For each node, find child nodes (i.e., the child nodes caused by the node); • Example of Weather, Toothache, Catch, and Cavity Bayesian Networks

  20. Topics Summary • Bayesian networks provide a natural representation for (causally induced) conditional independence. • Topology + CPTs = compact representation of joint distribution • Generally easy for domain experts to construct Bayesian Networks

More Related