1 / 76

- Relational - Graphical Models

MINVOLSET. KINKEDTUBE. PULMEMBOLUS. INTUBATION. VENTMACH. DISCONNECT. PAP. SHUNT. VENTLUNG. VENITUBE. PRESS. MINOVL. FIO2. VENTALV. PVSAT. ANAPHYLAXIS. ARTCO2. EXPCO2. SAO2. TPR. INSUFFANESTH. HYPOVOLEMIA. LVFAILURE. CATECHOL. LVEDVOLUME. STROEVOLUME. ERRCAUTER. HR.

nona
Download Presentation

- Relational - Graphical Models

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. MINVOLSET KINKEDTUBE PULMEMBOLUS INTUBATION VENTMACH DISCONNECT PAP SHUNT VENTLUNG VENITUBE PRESS MINOVL FIO2 VENTALV PVSAT ANAPHYLAXIS ARTCO2 EXPCO2 SAO2 TPR INSUFFANESTH HYPOVOLEMIA LVFAILURE CATECHOL LVEDVOLUME STROEVOLUME ERRCAUTER HR ERRBLOWOUTPUT HISTORY CO CVP PCWP HREKG HRSAT HRBP BP Advanced IWS 06/07 Based on Cussens and Kersting‘s ICML 2004 tutorial, De Raedt and Kersting‘s ECML/PKDD 2005 tutorial, and Friedman and Koller‘s NIPS 1999 tutorial - Relational - Graphical Models Wolfram Burgard, Luc De Raedt, Kristian Kersting, Bernhard Nebel Albert-Ludwigs University Freiburg, Germany

  2. Outline • Introduction • Reminder: Probability theory • Basics of Bayesian Networks • Modeling Bayesian networks • Inference (VE, Junction tree) • [Excourse: Markov Networks] • Learning Bayesian networks • Relational Models

  3. E B A J M Bayesian Networks • Finite, acyclic graph • Nodes: (discrete) random variables • Edges: direct influences • Associated with each node: a table representing a conditionalprobability distribution (CPD), quantifying the effect the parents have on the node - Relational

  4. MINVOLSET KINKEDTUBE PULMEMBOLUS INTUBATION VENTMACH DISCONNECT PAP SHUNT VENTLUNG VENITUBE PRESS MINOVL FIO2 VENTALV PVSAT ANAPHYLAXIS ARTCO2 EXPCO2 SAO2 TPR INSUFFANESTH HYPOVOLEMIA LVFAILURE CATECHOL LVEDVOLUME STROEVOLUME ERRCAUTER HR ERRBLOWOUTPUT HISTORY CO CVP PCWP HREKG HRSAT HRBP BP Bayesian Networks The “ICU alarm” network • 37 binary random variables • 509 parameters instead of - Relational

  5. Bayesian Networks • Effective (and to some extend efficient) inference algorithms • Variable elimination • Junction Trees • MPE • Effective (and to some extend efficient) learning approaches • Expectation Maximization • Gradient Ascent Dealing with noisy data, missing data and hidden variables Probability - Relational Knowledge Acquisition Bottleneck, Data cheap Learning

  6. Bayesian Networks: Problem • Bayesian nets use propositional representation • Real world has objects, related to each other Intelligence Difficulty - Relational Grade [slide due to Friedman and Koller]

  7. Intell_J.Doe Diffic_CS101 Intell_FGump Diffic_CS101 Grade_JDoe_CS101 Grade_FGump_CS101 Intell_FGump Diffic_Geo101 Grade_FGump_Geo101 Bayesian Networks: Problem • Bayesian nets use propositional representation • Real world has objects, related to each other These “instances” are not independent! - Relational A C [slide due to Friedman and Koller]

  8. How to Craft and Publish Papers • Are there similar papers? • Which papers are relevant? • Keywords Extraction • Does anybody know L. D. Raedt? - Relational Real World

  9. A2 A1 J1 JLP How to Craft and Publish Papers L. D. Raedt? P3 - Relational author-of published-in follow-up author publication medium

  10. P2 CIVR C1 C3 P4 ILP A2 P7 P1 A4 A1 P2 J1 P6 JLP P8 T-RO C2 ICRA J2 A3 P5 How to Craft and Publish Papers L. D. Raedt? P3 - Relational author-of published-in follow-up author publication medium

  11. aa AA aa aa aa Aa AA Aa Aa AA Aa Aa Aa aa aa Aa Aa aa AA Aa Blood Type / Genetics/ Breeding • 2 Alleles: A and a • Probability of Genotypes AA, Aa, aa ? Father Mother Offsprings Prior for founders - Relational

  12. aa AA aa aa aa Aa AA Aa Aa AA Aa Aa Aa aa aa Aa Aa aa AA Aa Blood Type / Genetics/ Breeding • 2 Alleles: A and a • Probability of Genotypes AA, Aa, aa ? Father Mother Offsprings Prior for founders - Relational CEPH Genotype DB,http://www.cephb.fr/

  13. Bongard´s Problems - Relational Noise? Some objects are opaque? (e.g. in relation is not always observed)

  14. Bongard´s Problems - Relational Noise? Some objects are opaque? (e.g. in relation is not always observed) Clustering?

  15. e e d c d c d c a b a b e a b c e d a b ... Other Application Areas Social Networks Planning Activity Recognition BioInformatics Scene interpretation/ segmentation Natural Language Processing - Relational Robotics Games Data Cleaning ?

  16. Why do we need relational models? • Rich Probabilistic Models • Comprehensibility • Generalization (similar situations/individuals) • Knowledge sharing • Parameter Reduction / Compression • Learning • Reuse of experience (training one RV might improve prediction at other RV) • More robust • Speed-up - Relational

  17. When to apply relational models ? • When it is impossible to elegantly represent your problem in attribute value form • variable number of ‘objects’ in examples • relations among objects are important - Relational attribute value form

  18. Probability Learning Logic Statistical Relational Learning … deals with machinelearning and data mining in relational domains where observations may be missing, partially observed, and/or noisy … and is one of the key open questions in AI. - Relational

  19. E B A J M BNs = Probabilistic Propositional Logic E. B. A :- E, B. J :- A. M :- A. + CPDs +

  20. Logic Programming father(rex,fred). mother(ann,fred). father(brian,doro). mother(utta, doro). father(fred,henry). mother(doro,henry). pc(rex,a). mc(rex,a). pc(ann,a). mc(ann,b). ... The maternal information mc/2 depends on the maternal and paternal pc/2information of the mother mother/2: mchrom(fred,a). mchrom(fred,b),... - Relational or better mc(P,a) :- mother(M,P), pc(M,a), mc(M,a). mc(P,a) :- mother(M,P), pc(M,a), mc(M,b). mc(P,b) :- mother(M,P), pc(M,a), mc(M,b). ... Placeholder Could be rex, fred, doro, …

  21. P2 CIVR C1 sameAuthor(A1,A2) :- nth-author-of(A1,P1), sameTitle(P1,P2), nth-author-of(A2,P2). C3 P4 Use general rules with placeholders ILP A2 P7 P1 A4 A1 P2 J1 P6 JLP P8 T-RO C2 ICRA J2 A3 P5 How to Craft and Publish Papers publication(p1). publication(p2). … author(a1). author(a2). … medium(c2). medium(m2). … proceedings(m1). … journal(m1). … author-of(a1,p3). author-of(a1,p3). … author-of(a1,p1). author-of(a2,p2). … published-in(p1,m1). published-in(p3,m2). … P3 - Relational author-of published-in follow-up author publication medium

  22. Outline Relational Models • Relational Models • Probabilistic Relational Models • Baysian Logic Programs • Relational Markov networks • Markov Logic - Relational

  23. Database theory Entity-Relationship Models Attributes = RVs P(A | B,E) E B Earthquake Burglary e b 0.9 0.1 b 0.2 0.8 e Alarm 0.9 0.1 e b 0.99 0.01 e b MaryCalls JohnCalls Probabilistic Relational Models (PRMs) [Koller,Pfeffer,Getoor] Database alarm system - Relational Earthquake Burglary Table Alarm MaryCalls JohnCalls Attribute

  24. Binary Relation Table Probabilistic Relational Models (PRMs) [Koller,Pfeffer,Getoor] (Father) (Mother) Bloodtype Bloodtype M-chromosome M-chromosome P-chromosome P-chromosome Person Person M-chromosome P-chromosome Bloodtype Person - Relational

  25. Probabilistic Relational Models (PRMs) [Koller,Pfeffer,Getoor] father(Father,Person). (Father) (Mother) mother(Mother,Person). Bloodtype Bloodtype M-chromosome M-chromosome P-chromosome P-chromosome Person Person bt(Person,BT). M-chromosome P-chromosome pc(Person,PC). mc(Person,MC). Bloodtype Person - Relational Dependencies (CPDs associated with): bt(Person,BT) :- pc(Person,PC), mc(Person,MC). pc(Person,PC) :- pc_father(Father,PCf), mc_father(Father,MCf). View : pc_father(Person,PCf) | father(Father,Person),pc(Father,PC). ...

  26. Probabilistic Relational Models (PRMs) [Koller,Pfeffer,Getoor] father(rex,fred). mother(ann,fred). father(brian,doro). mother(utta, doro). father(fred,henry). mother(doro,henry). pc_father(Person,PCf) | father(Father,Person),pc(Father,PC). ... mc(Person,MC) | pc_mother(Person,PCm), pc_mother(Person,MCm). pc(Person,PC) | pc_father(Person,PCf), mc_father(Person,MCf). bt(Person,BT) | pc(Person,PC), mc(Person,MC). State RV mc(ann) pc(ann) mc(rex) pc(rex) mc(utta) mc(brian) pc(brian) pc(utta) - Relational pc(fred) pc(doro) mc(fred) mc(doro) bt(brian) bt(utta) bt(rex) bt(ann) mc(henry) pc(henry) bt(fred) bt(doro) bt(henry)

  27. PRM Application:Collaborative Filterting [Getoor, Sahami] • User preference relationships for products / information. • Traditionally: single dyactic relationship between the objects. ... buys11 buys12 buysNM - Relational ... ... classProd2 classProdM classPersN classProd1 classPers1 classPers2

  28. Relational Naive Bayes PRM Application:Collaborative Filtering [Getoor, Sahami; simplified representation] buys/2 topicPage/1 reputationCompany/1 classProd/1 visits/2 classPers/1 manufactures - Relational subscribes/2 topicPeriodical/1 colorProd/1 costProd/1 incomePers/1

  29. Probabilistic Relational Models (PRMs) [Koller,Pfeffer,Getoor] • Database View • Unique Probability Distribution over finite Herbrand interpretations • No self-dependency • Discrete and continuous RV • BN used to do inference • Graphical Representation - Relational

  30. Outline Relational Models • Relational Models • Probabilistic Relational Models • Baysian Logic Programs • Relational Markov networks • Markov Logic - Relational

  31. P(A | B,E) E B Earthquake Burglary e b 0.9 0.1 b 0.2 0.8 e Alarm 0.9 0.1 e b 0.99 0.01 e b MaryCalls JohnCalls Bayesian Logic Programs (BLPs) [Kersting, De Raedt] - Relational

  32. P(A | B,E) E B Earthquake Burglary e b 0.9 0.1 b 0.2 0.8 e Alarm 0.9 0.1 e b 0.99 0.01 e b Rule Graph MaryCalls JohnCalls earthquake/0 burglary/0 alarm/0 maryCalls/0 johnCalls/0 Bayesian Logic Programs (BLPs) [Kersting, De Raedt] - Relational

  33. P(A | B,E) E B Earthquake Burglary e b 0.9 0.1 b 0.2 0.8 e Alarm 0.9 0.1 e b 0.99 0.01 e b Rule Graph MaryCalls JohnCalls earthquake/0 burglary/0 local BN fragment earthquake P(A | B,E) burglary E B e b 0.9 0.1 alarm/0 b 0.2 0.8 e 0.9 0.1 e b 0.99 alarm 0.01 e b maryCalls/0 johnCalls/0 alarm :- earthquake, burglary. Bayesian Logic Programs (BLPs) [Kersting, De Raedt] - Relational

  34. Mother pc mc mother mc Person argument mc(Person) pc(Mother) mc(Mother) (.9,.05,.05) a a (.495,.495,.01) a b Person atom ... ... ... pc mc bt(Person) pc(Person) mc(Person) (.9,.03,.03,.03) a a (.03,.03,.9,.03) a b bt ... ... ... bt(Person) :- pc(Person),mc(Person). predicate Bayesian Logic Programs (BLPs) [Kersting, De Raedt] Rule Graph pc/1 mc/1 bt/1 variable - Relational

  35. pc(Person) pc(Father) mc(Father) (.9,.05,.05) a a (.495,.495,.01) a b ... ... ... Bayesian Logic Programs (BLPs) [Kersting, De Raedt] Father pc/1 mc/1 pc mc father pc bt/1 Person - Relational mc(Person) | mother(Mother,Person), pc(Mother),mc(Mother). pc(Person) | father(Father,Person), pc(Father),mc(Father). bt(Person) | pc(Person),mc(Person).

  36. Bayesian Logic Programs (BLPs) [Kersting, De Raedt] father(rex,fred). mother(ann,fred). father(brian,doro). mother(utta, doro). father(fred,henry). mother(doro,henry). mc(Person) | mother(Mother,Person), pc(Mother),mc(Mother). pc(Person) | father(Father,Person), pc(Father),mc(Father). bt(Person) | pc(Person),mc(Person). Bayesian Network induced over least Herbrand model mc(ann) pc(ann) mc(rex) pc(rex) mc(utta) mc(brian) pc(brian) pc(utta) - Relational pc(fred) pc(doro) mc(fred) mc(doro) bt(brian) bt(utta) bt(rex) bt(ann) mc(henry) pc(henry) bt(fred) bt(doro) bt(henry)

  37. Answering Queries P(bt(ann)) ? Bayesian Network induced over least Herbrand model mc(ann) pc(ann) mc(rex) pc(rex) mc(utta) mc(brian) pc(brian) pc(utta) - Relational pc(fred) pc(doro) mc(fred) mc(doro) bt(brian) bt(utta) bt(rex) bt(ann) mc(henry) pc(henry) bt(fred) bt(doro) bt(henry)

  38. P(bt(ann),bt(fred)) Bayes‘ rule P(bt(fred)) P(bt(ann)| bt(fred)) = Answering Queries P(bt(ann), bt(fred)) ? Bayesian Network induced over least Herbrand model mc(ann) pc(ann) mc(rex) pc(rex) mc(utta) mc(brian) pc(brian) pc(utta) - Relational pc(fred) pc(doro) mc(fred) mc(doro) bt(brian) bt(utta) bt(rex) bt(ann) mc(henry) pc(henry) bt(fred) bt(doro) bt(henry)

  39. Combining Partial Knowledge ... Topic discusses Book discusses/2 read/1 prepared read Student prepared(Student,Topic) | read(Student,Book), discusses(Book,Topic). prepared/2 logic prepared - Relational bn passes passes/1 prepared Student passes(Student) | prepared(Student,bn), prepared(Student,logic).

  40. Combining Partial Knowledge discusses(b2,bn) Topic • variable # of parents for prepared/2 due to read/2 • whether a student prepared a topic depends on the books she read • CPD only for one book-topic pair discusses discusses(b1,bn) Book prepared read Student prepared(s2,bn) prepared(s1,bn) - Relational

  41. Combining Rules Topic P(A|B) and P(A|C) discusses Book prepared read Student CR P(A|B,C) • Any algorithm which • has an empty output if and only if the input is empty • combines a set of CPDs into a single (combined) CPD • E.g. noisy-or, regression, ... - Relational

  42. Aggregates Map multisets of values to summary values (e.g., sum, average, max, cardinality) ... registration_grade/2 registered/2 - Relational student_ranking/1

  43. registered/2 Functional Dependency (average) Course Student registration_grade grade_avg Probabilistic Dependency (CPD) grade_avg student_ranking Student Aggregates Map multisets of values to summary values (e.g., sum, average, max, cardinality) ... registration_grade/2 registered/2 grade_avg/1 - Relational Deterministic student_ranking/1

  44. Experiments KDD Cup 2001 localization task predict the localization based on local features and interactions 862 training genes 381 test genes >1000 interactions 16 classes WebKB predict the type of web pages 877 web pages from 4 CS department 1516 links 6 classes - Relational

  45. KDD Cup: Protein Localization RFK (72.89%) better then Hayashi et al.’s KDD Cup 2001 winning nearest- neighbour approach (72.18%) - Relational

  46. WebKB: Web Page Classification • Collective NB ~ PRMs [Getoor et al. 02] • RFK outperforms PRMs • PRM with structural uncertainty over the links , best acc. (68%) • on Washington - Relational Leave-one-university-out cross-validation

  47. Bayesian Logic Programs (BLPs) • Unique probability distribution over Herbrand interpretations • Finite branching factor, finite proofs, no self-dependency • Highlight • Separation of qualitative and quantitative parts • Functors • Graphical Representation • Discrete and continuous RV - Relational

  48. Learning Tasks • Parameter Estimation • Numerical Optimization Problem • Model Selection • Combinatorical Search Learning Algorithm Database Model - Relational

  49. What is the data about? RVs + States = (partial) Herbrand interpretation Probabilistic learning from interpretations Background m(ann,dorothy), f(brian,dorothy), m(cecily,fred), f(henry,fred), f(fred,bob), m(kim,bob), ... Family(2) bt(cecily)=ab, pc(henry)=a, mc(fred)=?, bt(kim)=a, pc(bob)=b Family(1) pc(brian)=b, bt(ann)=a, bt(brian)=?, bt(dorothy)=a - Relational Family(3) pc(rex)=b, bt(doro)=a, bt(brian)=?

  50. Parameter Estimation + - Relational

More Related