1 / 30

S3 SEMINAR ON DATA MINING -BAYESIAN NETWORKS- A. BASICS

S3 SEMINAR ON DATA MINING -BAYESIAN NETWORKS- A. BASICS . Concha Bielza , Pedro Larrañaga Computational Intelligence Group Departamento de Inteligencia Artificial Universidad Politécnica de Madrid. Master Universitario en Inteligencia Artificial. Reasoning under uncertainty.

sveta
Download Presentation

S3 SEMINAR ON DATA MINING -BAYESIAN NETWORKS- A. BASICS

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. S3 SEMINAR ON DATA MINING -BAYESIAN NETWORKS- A. BASICS Concha Bielza, Pedro Larrañaga ComputationalIntelligenceGroup Departamento de Inteligencia Artificial Universidad Politécnica de Madrid Master Universitario en Inteligencia Artificial

  2. Reasoning under uncertainty Conditional independence D-separation Bayesian networks: formal definition Building BNs Conceptos básicos Basics of Bayesian networks C.Bielza, P.Larrañaga -UPM-

  3. Reasoning Cond.Indep. D-separ Definition Building Reasoning under uncertainty Advantages of BNs • Explicitrepresentation of the uncertain knowledge • Graphical, intuitive, closer to a world repres. • Deal with uncertainty for reasoninganddecision-making • Founded on probability theory, provide a clear semantics and • a sound theoretical foundation • Manage many variables • Both data and experts can be used to construct the model • Current and huge development • Support the expert; do not try to replace him C.Bielza, P.Larrañaga -UPM-

  4. Reasoning Cond.Indep. D-separ Definition Building Reasoning under uncertainty The joint probability distribution (global model) is specified via marginal and conditional distributions (local models) taking into account conditionalindependence relationships among variables • This modularity: • Provides an easy maintenance • Reduces the number of parameters needed for the global model Estimation/elicitation is easier Reduction of the storing needs Efficient reasoning (inference) Modularity C.Bielza, P.Larrañaga -UPM-

  5. Reasoning Cond.Indep. D-separ Definition Building Conditional independence The joint probability distribution • Dealing with a joint probability distribution • n diseases D1,…,Dn • m symptoms S1,…,Sm • Represent P(D1,…,Dn,S1,…,Sm), with 2n+m-1 parameters • E.g.: m=30, n=10, need of 240-1≈1012 That’s complete dependence: intractable in practice C.Bielza, P.Larrañaga -UPM-

  6. Reasoning Cond.Indep. D-separ Definition Building Conditional independence Independence With mutual independence, only specify P(X1),…,P(Xn) n parameters (lineal) instead of 2n-1 (exponential) Unfortunately, it rarely holds in most domains Fortunately, there are some conditional independences. Exploit them (representation and inference) C.Bielza, P.Larrañaga -UPM-

  7. Reasoning Cond.Indep. D-separ Definition Building Conditional independence Conditional independence Independence (marginal) sets of vars Conditional independence of X and Y given Z 3 disjoint sets of variables for all possible values x,y,z Intuitively, whenever Z=z, the information Y=y does not influence on the probability of x Notation: C.Bielza, P.Larrañaga -UPM-

  8. ReasoningCond.Indep. D-separDefinitionBuilding Further factorizing the JPD Chain rule and factorization via c.i. Joint distribution factorized C.Bielza, P.Larrañaga -UPM-

  9. ReasoningCond.Indep. D-separDefinitionBuilding BNs Informal definition: 2 components in a BN • Qualitative part: a directed acyclic graph (DAG) • Nodes = variables • Arcs = • direct dependence relations (otherwise it indicates absence of direct dependence; there may be indirect dependences and independences) YES Not necessarily causality Quantitative part: a set of conditional probabilities that determine a unique JPD C.Bielza, P.Larrañaga -UPM-

  10. ReasoningCond.Indep. D-separDefinitionBuilding BNs: nodes Target node Parents Ancestors Children Descendants Rest Family C.Bielza, P.Larrañaga -UPM-

  11. ReasoningCond.Indep. D-separ Definition Building BNs: arcs (types of independence) Independences in a BN A BN represents a set of independences Distinguish: Basic independences: we should take care of verifying them when constructing the net Derived independences: from the previous independences, by using the properties of the independence relations Check them by means of the d-separation criterion C.Bielza, P.Larrañaga -UPM-

  12. ReasoningCond.Indep. D-separ Definition Building Basic independences Basic independence: Markov condition Xi c.i. of its non-descendants, given its parents Pa(Xi) C.Bielza, P.Larrañaga -UPM-

  13. ReasoningCond.Indep. D-separ Definition Building Basic independences Example Fever is conditionally independent of Jaundice given Malaria and Flu C.Bielza, P.Larrañaga -UPM-

  14. ReasoningCond.Indep. D-separ Definition Building Quantitative part Factorizing the JPD …Now with the quantitative part of the net, the JPD: Specify it intelligently. Use the chain rule and the Markov condition Let X1,…,Xn be an ancestral ordering (parents appear before their children in the sequence). It always exists (DAG) Using that ordering in the chain rule, in {Xi-1,…,X1} there are non-descendants of Xi, and we have C.Bielza, P.Larrañaga -UPM-

  15. ReasoningCond.Indep. D-separ Definition Building Quantitative part MODEL CONSTRUCTION EASIER: Only store local distributions at each node Fewer parameters to assign and more naturally Inference easier Factorizing the JPD Therefore, we can recover the JPD by using the following factorization: C.Bielza, P.Larrañaga -UPM-

  16. ReasoningCond.Indep. D-separ Definition Building Quantitative part B E A N 1 1 W 4 2 2 Withallbinary variables: 32=25-1 probabilities for the JPD 10 withthefactorization in the BN: C.Bielza, P.Larrañaga -UPM-

  17. ReasoningCond.Indep. D-separ Definition Building Quantitative part BN Alarm for monitoring ICU patients 254 probabilities for the JPD vs. 509 in BN C.Bielza, P.Larrañaga -UPM-

  18. ReasoningCond.Indep. D-separ Definition Building Independencesderivedfrom d-separation u-separation Equivalentcriterionto d-separation and sometimeswithfewerchecks Obtain the minimum graph containing X,Y,Z and their ancestors (ancestral graph) The subgraph obtained is moralized (add a link between parents with children in common) and remove direction of arcs Zu-separatesX and Y whenever Z is in all paths between X and Y C.Bielza, P.Larrañaga -UPM-

  19. ReasoningCond.Indep. D-separ Definition Building Joining the two parts disjoint d-separation defined by G c.i. defined by P Graph G representsalldependences of P Some independences of P may be not identified by d-separation in G d-separation Theorem [Verma and Pearl’90, Neapolitan’90] Let P be a prob. distribution of the variables in V and G=(V,E) a DAG. (G,P) holds the Markov condition iff C.Bielza, P.Larrañaga -UPM-

  20. ReasoningCond.Indep. D-separ Definition Building Definition of BN (taking an ancestral ordering) Formal definition • Let P be a JPD over V={X1,…,Xn}. • A BNis a tuple (G,P), where G=(V,E) is a DAG suchthat: • Eachnode of G represents a variable of V • TheMarkovconditionisheld • Eachnode has associated a localprob. • distrib. suchthat • d-separated variables in the graph are independent (G is a minimal I-map of P) quantitative part C.Bielza, P.Larrañaga -UPM-

  21. ReasoningCond.Indep. D-separ Definition Building Definition of BN A property Set of nodes that makes X c.i. of the rest of the network: A node is c.i. of all other nodes in the BN, given its parents, childrenand children’s parents -itsMarkov blanket- C.Bielza, P.Larrañaga -UPM-

  22. ReasoningCond.Indep. D-separ Definition Building Definition of BN Malaria is conditionally independent of Aches given ExoticTrip, Jaundice, Fever and Flu C.Bielza, P.Larrañaga -UPM-

  23. ReasoningCond.Indep. D-separ Definition Building Building a BN Learning from a database Database algorithm Bayesian net A combination(experts → structure; database → probabilities) Expert /from data /both Manual with the aid of an expert in the domain modelisation probabilities Causal mechanisms Causal graph Bayesian net Build it in the causal direction: BNs simpler and efficient C.Bielza, P.Larrañaga -UPM-

  24. ReasoningCond.Indep. D-separ Definition Building Building a BN Summary C.Bielza, P.Larrañaga -UPM-

  25. ReasoningCond.Indep. D-separ Definition Building Building a BN Summary C.Bielza, P.Larrañaga -UPM-

  26. ReasoningCond.Indep. D-separ Definition Building Building a BN Summary C.Bielza, P.Larrañaga -UPM-

  27. ReasoningCond.Indep. D-separ Definition Building Building a BN Example: Asia BN [Lauritzen & Spiegelhalter’88] C.Bielza, P.Larrañaga -UPM-

  28. ReasoningCond.Indep. D-separ Definition Building Building a BN C.Bielza, P.Larrañaga -UPM-

  29. Texts and readings C.Bielza, P.Larrañaga -UPM-

  30. S3 SEMINAR ON DATA MINING -BAYESIAN NETWORKS- A. BASICS Concha Bielza, Pedro Larrañaga ComputationalIntelligenceGroup Departamento de Inteligencia Artificial Universidad Politécnica de Madrid Master Universitario en Inteligencia Artificial

More Related