Kristian Kersting University of Freiburg Germany

1. „Application of Probabilistic ILP II“, FP6-508861 www.aprill.org Probabilistic Logic Learning al and Relational Probability Logic Learning James Cussens University of York UK Kristian Kersting University of Freiburg Germany

2. Special thanks ... ... for discussions, materials, and collaborations to Alexandru Cocura,Luc De Raedt, Uwe Dick, Pedro Domingos, Thomas Gaertner, Lise Getoor, Martin Guetlein, Bernd Gutmann, Manfred Jaeger, Stephen Muggleton,Tapani Raiko, Reimund Renner, Richard Schmidt, Ingo Thon

3. Tutorial´s Aims • Introductory survey • Identification of important probabilistic, relational/logical and learning concepts

4. Objectives One of the key open questions of AI concerns Probabilistic Logic Learning: The integration of probabilistic reasoningwith Probabilitiy first order / relational logic representationsand Logic Learning machine learning.

5. Text Classification Computer troubleshooting Economic Why do we need PLL? Robotics Medicine Diagnosis Prediction Classification Decision-making Description Web Mining Computational Biology Let‘s look at an example PLMs

6. Web Mining / Linked Bibliographic Data / Recommendation Systems / … [illustration inspired by Lise Getoor] books B2 authors publishers B1 B3 A2 A1 P2 B4 P1 Real World

7. Web Mining / Linked Bibliographic Data / Recommendation Systems / … books B2 authors publishers B1 B3 series A2 author-of publisher-of A1 P2 B4 P1 Fantasy Science Fiction Real World

8. Not flat but rich representations: Multi-relational, heterogeneous and semi-structured Structured Domains Dealing with noisy data, missing data and hidden variables Uncertainty Knowledge Acquisition Bottleneck, Data cheap Machine Learning Why do we need PLL? Real World Applications Let‘s look at some more examples

9. AA AA AA aa aa aa AA Aa aa AA Aa Aa AA Aa Aa aa Aa aa Aa Aa aa AA Aa Blood Type / Genetics/ Breeding Prior CEPH Genotype DB,http://www.cephb.fr/

10. Bongard´s Problems Noise? Opaque? (partially observable)

11. Bongard´s Problems Noise? Clustering? Opaque? (partially observable)

12. Others Protein Secondary Structure Social Networks Data Cleaning ? Scene interpretation Metabolic Pathways Phylogenetic Trees

13. SRL Why do we need PLL ? Statistical Learning (SL) Probabilistic Logics Uncertainty • attribute-value representations: some learning problems cannot (elegantly) be described using attribute value representations • no learning: to expensive to handcraft models + soft reasoning, expressivity + soft reasoning, learning PLL Real World Applications Structured Domains Machine Learning Inductive Logic Programming (ILP) Multi-Relational Data Mining (MRDM) - crisp reasoning: some learning problems cannot (elegantly) be described wihtout explicit handling of uncertainty + expressivity, learning

14. Why do we need PLL? • Rich Probabilistic Models • Comprehensibility • Generalization (similar situations/individuals) • Knowledge sharing • Parameter Reduction / Compression • Learning • Reuse of experience (training one RV might improve prediction at other RV) • More robust • Speed-up

15. Why Learning ? • Knowledge acquisition bottleneck / data is cheap • General purpose systems • Combining domain expert knowledge with data • Logical structure provides insight into domain • Handling missing data bt(luc)=? Learning Algorithm Database Model

16. When to apply PLL ? • When it is impossible to elegantly represent your problem in attribute value form • variable number of ‘objects’ in examples • relations among objects are important • Background knowledge can be defined intensionally : • define ‘benzene rings’ as view predicates

17. ´97 ´93 ´00 ´90 ´95 ´94 ´02 ´96 Logical Bayesian Networks: Blockeel,Bruynooghe, Fierens,Ramon, LOHMMs: De Raedt, Kersting, Raiko 1BC(2): Flach, Lachiche First KBMC approaches: Bresse, Bacchus, Charniak, Glesner, Goldman, Koller, Poole, Wellmann Prob. Horn Abduction: Poole RMMs: Anderson,Domingos, Weld BLPs: Kersting, De Raedt PLP: Haddawy, Ngo LAPD: Bruynooghe Vennekens,Verbaeten PRMs: Friedman,Getoor,Koller, Pfeffer,Segal,Taskar PRISM: Kameya, Sato Markov Logic: Domingos, Richardson SLPs: Cussens,Muggleton CLP(BN): Cussens,Page, Qazi,Santos Costa Prob. CLP: Eisele, Riezler (Incomplete) Historical Sketch [names in alphabetical order] ´99 ´03 Future Present 2003 many more ...

18. Overview • Introduction to PLL • Foundations of PLL • Logic Programming, Bayesian Networks, Hidden Markov Models, Stochastic Grammars • Frameworks of PLL • Independent Choice Logic,Stochastic Logic Programs, PRISM,Probabilistic Logic • Programs,Probabilistic Relational Models, Bayesian Logic Programs • Relational Hidden Markov Models • Applications