1 / 18

Bayes Nets

Bayes Nets. Rong Jin. O 1. O 3. O 0. O 2. O 4. q 0. q 1. q 2. q 3. q 4. Hidden Markov Model. Inferring from observations ( o i ) to hidden variables ( q i ) This is a general framework for representing and reasoning about uncertainty

chogan
Download Presentation

Bayes Nets

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Bayes Nets Rong Jin

  2. O1 O3 O0 O2 O4 q0 q1 q2 q3 q4 Hidden Markov Model • Inferring from observations (oi) to hidden variables (qi) • This is a general framework for representing and reasoning about uncertainty • Representing uncertain information with random variables (nodes) • Representing the relationship between information with conditional probability distribution (directed arcs) • Infer from observation (shadowed nodes) to the hidden variables (circled nodes)

  3. An Example of Bayes Network • S: It is sunny • L: Ali arrives slightly late • O: Slides are put on web late

  4. S O L Bayes Network Example Absence of an arrow: Random S and O are independent. Knowing S will not help predicate O Two arrows into L: L depends on S and O. Knowing S and O will help predicate L.

  5. S O L Inference in Bayes Network • S = 1, O = 0, P(L) = ? • S = 1, P(O) = ?, P(L) = ? • L = 1, P(S) = ?, P(O) = ? • L = 1, S = 1, P(O) = ?

  6. Conditional Independence • Formal definition: A and B are conditional independent given C iff • Different from independence • Example: • A: shoe size • B: glove size • C: heigh • Shoe size is not independent from glove size C B A

  7. S O L Distinguish Two Cases • A: shoe size • B: glove size • C: heigh C B A Given C: A and B are independent Without C: A and B can be dependent • S: It is sunny • L: Ali arrives slightly late • O: Slides are put on web late Without L: S and O are independent Given L: S and O can be dependent

  8. Cloudy Sprinkle Rain WetGrass Another Example for Bayes Nets Inference questions • W=1, P(R) =? • W= 1, P(C) = ? • W= 1, C = 1, P(S) = ?, P(C) = ?, P(S,R) = ?

  9. Bayes Nets Formalized A Bayes net (also called a belief network) is an augmented directed acyclic graph, represented by the pair V , E where: • V is a set of vertices. • E is a set of directed edges joining vertices. No loops of any length are allowed. Each vertex in V contains the following information: • The name of a random variable • A probability distribution table indicating how the probability of this variable’s values depends on all possible combinations of parental values.

  10. Building a Bayes Net • Choose a set of relevant variables. • Choose an ordering for them • Assume they’re called X1 .. Xm (where X1 is the first in the ordering, X1 is the second, etc) • For i = 1 to m: • Add the Xi node to the network • Set Parents(Xi ) to be a minimal subset of {X1…Xi-1} such that we have conditional independence of Xi and all other members of {X1…Xi-1} given Parents(Xi ) • Define the probability table of P(Xi=k Assignments of Parents(Xi ) ).

  11. Example of Building Bayes Nets Suppose we’re building a nuclear power station. There are the following random variables: GRL : Gauge Reads Low. CTL : Core temperature is low. FG : Gauge is faulty. FA : Alarm is faulty AS : Alarm sounds • If alarm working properly, the alarm is meant to sound if the gauge stops reading a low temp. • If gauge working properly, the gauge is meant to read the temp of the core.

  12. CTL GRL AS FA FG Bayes Net for Power Station GRL : Gauge Reads Low. CTL : Core temperature is low. FG : Gauge is faulty. FA : Alarm is faulty AS : Alarm sounds

  13. Inference with Bayes Nets • Key issue: computing joint probability P(X1=x1^ X2=x2^ ….Xn-1=xn-1^ Xn=xn) • Using the conditional independence relations to simplify the computation

  14. Cloudy Sprinkle Rain WetGrass Example for Inference Inference questions • W=1, P(R) =? • W= 1, P(C) = ? • W= 1, C = 1, P(S) = ?, P(C) = ?, P(S,R) = ?

  15. Problem with Inference using Bayes Nets • Inference • Infer from observations EO to unknown variables Eu Suppose you have m binary-valued variables in your Bayes Net and expression Eo mentions k variables. How much work is the above computation?

  16. Problem with Inference using Bayes Nets • General querying of Bayes nets is NP-complete. • Some solutions: • Belief propagation • Take advantage of the structure of Bayes nets • Stochastic simulation • Similar to the sampling approaches for Bayesian average

  17. More Interesting Questions • Learning Bayes nets • Given the topological structure of a Bayes net, learn all the conditional probability tables from examples • Example: Hierarchical mixture model • Learning the topological structure of Bayes net • Very very hard question • Unfortunately, the lecturer does not have enough knowledge to teach you if he wants to !

  18. Cloudy Sprinkle Rain WetGrass Learning Cond. Probabilities in Bayes Nets • Three types of training examples • C, S, R, W • C, R, W • S, C, W • Maximum likelihood approach for estimating the conditional probabilities • EM algorithm for optimization

More Related