APRIL

1. APRIL ApplicationsofProbabilisticInductive LogicProgramming Albert-Ludwigs University Freiburg, Germany Imperial College of Science, Technology and Medicine, London, Great Britain

2. Abstract The project adresses „probabilistic logic learning“ i.e. the integration of probabilistic reasoning with first order logic representations and machine learning. The objective is to critically assess the promise of this approach using application of functional genomics.

3. Context Real-world applications Uncertainty Complex, structured domains Logic: objects, relations, functors Probability theory Learning Hidden Markov Models, Stochastic Context-Free Grammars, Bayesian networks, .... Logic Programs (Prolog) „Probabilistic Logic“ ? „Probabilistic Logic Learning“

4. Objectives (1) One of the key open questions of artificial intelligence concerns "probabilisitic logic learning", i.e. the integration of probabilistic reasoning with first order logic representations and machine learning. Probabilitstic Logic Learning

5. What is „Probabilistic Logic Learning? - Probabilistic • Representations and reasoning mechanisms grounded in probability theory, e.g. HMMs, Bayesian networs, stochastics grammars ... • Successfully used in a wide range of applications such as computational biology, speech recognition, ... • Robust models Probabilistic Logic Learning

6. What is „Probabilistic Logic Learning? - Logic • First-order logic • Elegant representation of complex situations involving a variety of objects as well as reltions among these objects: Probabilistic Logic Learning bloodtype(X,a) <- mother(M,X),bloodtype(M,a), father(F,X),bloodtype(F,a).

7. What is „Probabilistic Logic Learning? - Learning • Computing various aspects of a „probabilistic logic“ on the basis of data Probabilistic Logic Learning • Often it is easier to obtain data • and to learn a model than using • traditional knowledge engineering techniques • Parameter estimation • Learning the „logical“ structure • Fully vs. unobservable random variables

8. Research Context:„Probabilistic Logics“ • Studies of probabilistic logics: Nielson, Halpern, Poole, ... • Recently: Koller, Sato, Jaeger ,... Probabilistic Logic Learning objects/relations Probabilistic Logics Logics HMMs, Bayesian networks, ... propositional deterministic probabilistic

9. Research Context: „Probabilistic Learning“ • (ML) Parameter estimation • Gradient-based algorithms • Family of EM algorithm • Different scores • LogLikelihood, BIC, MDL, MML, ... • Structural learning • Score-based hill-climbing • Independency tests • Monte Carlo Markov Chain (MCMC) techniques Probabilistic Logic Learning

10. Research Context: „Logic Learning“ • Inductive Logic Learning • Machine Learning and Data Mining within first-order representations • Broadened the application domain of Data Mining, especially in bio- and chemoinformatics • Some of the best-known examples of Scientific Discovery by AI systems Probabilistic Logic Learning

11. Objectives (2) Overall goal: To critically investigate "probabilistic logic learning" methods by answering the following questions: • Are there significant applications for which "first order probabilistic logic" is better than state-of-the-art representations? • Can models be learned within such a "first order probabilistic logic"?

12. Description of Work (1) • In order to answer the questions, we plan • to investigate and evaluate various alternative first order probabilistic representations and reasoning mechanisms such as: • Stochastic Logic Programs [Muggleton 95, Cussens 99], • Bayesian Logic Programs" [Kersting, De Raedt 00].

13. Description of Work (2) • to employ the most promising such representations to model a functional genomics application; • to develop and to employ simple learning techniques to enable the learning of parts of the probabilistic logic representation; • to identify the main goals, directions and questions to be addressed in probabilistic logic learning.

14. Application: Functional Genomics (1) Hieter P. and Boguski M. (1997) Functional Genomics: It's all how you read it. Science 278, 601-02: Functional Genomics „Development and application of global (genome-wide or system-wide) experimental approaches to assess gene function by making use of the information and reagents provided by structural genomics.“

15. Application: Functional Genomics (2) Hieter P. and Boguski M.: "[Functional genomics] is characterized by high throughput or large-scale experimental methodologies combine with statistical and computational analysis of the results. The fundamental strategy in a functional genomics approach is to expand the scope of biological investigation from studying single genes or proteins to studying all genes or proteins at once in a systematic fashion." Probabilistic Logic Learning

16. Application: Functional Genomics (3) • Inductive Logic Learning supports the use (as background knowledge) of generally known metabolic pathways and encoded enzyme chemistry. • Probabilistic approaches make probabilistic descriptions of outcomes of experiments, i.e. they account for uncertainty due to e.g. noise, hidden variables, etc. Probabilistic Inductive Logic Learning should enable probabilistic descriptions of experiments, as well as supporting the general, i.e. first-order description of probabilistic behaviour in e.g. the enzyme chemistry. deterministic propositional

17. Stochastic Logic Programs /Bayesian Logic Programs Bayesian Logic Programs [Kersting, De Raedt 00] • Ground atoms = random variables • Conditional probability distribution associated to Horn clause • Dependency graph resticted to least Herbrand universum= (possibly infinite) Bayesian network • Stochastic Logic Programs [Muggleton 95, Cussens 99] • Probabilities defined over proofs • Single probability values associated to Horn clause • Log-linear models

18. Related Projects DARPA project: Evidence Extraction and Link Discovery (EELD) http://www.darpa.mil/iso/EELD/