1 / 39

Inductive Logic Programming

Inductive Logic Programming. Content. Introduction to ILP Basic ILP techniques An overview of the different ILP systems The application field of ILP Summary. Introduction to ILP. Inductive logic programming (ILP) = Inductive concept learning (I) Logic Programming (LP) Goal:

jennifer-stone
Download Presentation

Inductive Logic Programming

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Inductive Logic Programming

  2. Content • Introduction to ILP • Basic ILP techniques • An overview of the different ILP systems • The application field of ILP • Summary

  3. Introduction to ILP • Inductive logic programming (ILP) =Inductive concept learning (I) Logic Programming (LP) • Goal: • Develop a theoretical framework for induction • Build practical algorithms for inductive learning of relational concepts described in the form of logic programs • Background: • ILP theory based on the theory of LP • ILP algorithms based on experimental and application oriented ML research • Motivation: • Use of an expressive representational formalism as proportional logic • Use background knowledge in learning (in AI the use of domain knowledge is essential for achieving intelligent behaviour)

  4. Introduction to ILP 2 • Inductive learning with background knowledge:Given a set of training examples E and background knowledge B find a hypothesis H, expressed in some concept description language L, such that H is complete and consistent with respect to the background knowledge B and the examples E • A hypothesis H is complete with regard to the background knowledge B and examples E if it covers all the positive examples i.e., if • A hypothesis H is consistent with respect to the background knowledge B and examples E if it covers none of the negative examples i.e.,

  5. Introduction to ILP 3

  6. Introduction to ILP 4 • Example: The task is to define the target relation daughter(X,Y) • Background knowledge consists of ground facts about the predicates female(X) and parent(Y,X):parent(ann, mary) female(ann)parent(ann, tom) female(marry)parent(tom, eve) female(eve)parent(tom, ian) • Training examples:+: daughter(marry, ann) daughter(eve, tom)-: daughter(tom, ann) daughter(eve, ann) • Possible target relation: • Here the target relation is:

  7. Introduction to ILP 5 • Dimension of ILP • Learning either a single concept or multiple concepts • Requires all the training examples to be given before the learning process (batch learners) or accepts examples one by one (incremental learners) • The learner may rely on an oracle to verify the validity of generalisation and/or classify examples generated by the learner (interactive; non interactive) • The learner may try to learn a concept from scratch or can accept an initial hypothesis (theory) which is then revised in the learning process. The latter system is called theory revision. • Existing ILP systems • Empirical ILP system: Batch non-interactive system that learns single predicates from scratch • Interactive ILP system: Interactive and incremental theory revision system that learns multiple predicates

  8. Content • Introduction to ILP • Basic ILP techniques • Generalisation techniques • Specialisation techniques • An overview of the different ILP systems • The application field of ILP • Summary

  9. -subsumption • STRUCTURING THE HYPOTHESIS SPACE: Introducing partial ordering into a set of clauses based on the -subsumption • Def: A substitution is a function from variables to terms. The application of a substitution to a W is obtained by replacing all occurences of each variable in W by the same term • Def:Let c and c' be two program clauses. Clause c -subsumes c' if there exits a substitution , such that • Def: Two clauses c and d are -subsumption equivalent if c -subsumes d and d -subsumes c. • Def: A clause is reduced if it is not -subsumption equivalent to any proper subset of itself.

  10. -subsumption (2) • Example1: Let c be the clause: c = daughter(X,Y) parent(Y,X).A substitution applied to clause c is obtained by applying to each of its literal: c = daughter(mary, ann) parent(ann, mary). • Example2: Clause c -subsumes the clause c' = daughter(X,Y) female(X), parent(Y,X)under the empty substitution • Example3: Clause c -subsumes the clause c' = daughter(mary,ann) female(mary),parent(ann,mary),parent(ann,tom)under the substitution

  11. -subsumption (3) • -subsumption introduces the syntactic notation of generality: • Clause c is at least as general as clause c' ( ), if c -subsumes c' • Clause c is more general than clause c' ( ), if holds and does not hold • c' is a refinement (specialisation) of c • c is a generalisationof c'

  12. -subsumption (4) • -subsumption is important for learning: • It provides a generality order for hypotheses, thus structuring the hypothesis space • It can be used to prune large parts of the search space: • If generalising c to c' all the examples covered by cwill be also covered by c'This property is used to prune the search of more general clauses when e is a negative example: if c is inconsistent then all its generalisations will also be inconsistent. Hence, the generalisation of c do not need to be considered. • When specialising c to c' an example not covered by c will not be covered by any of its specialisations either.This property is used to prune the search of more specific clauses:if c does not cover a positive example none of its specialisation will do. Hence, the specialisations of c do not need to be considered.

  13. -subsumption (5) • .Techniques based on the -subsumption: • Bottom-up: creating the least general generalisation from the training examples, relative to the background knowledge • Top-down searching of refinement graphs

  14. Least General Generalisation • Properties of -subsumption: • If c -subsumes c' then c logically entails c', the reverse is not always true • The relation defines a lattice over the set of reduced clauses.This means that any two clauses have a least upper bound (lub) and a greatest lower bound (glb). • Def: The least general generalisation(lgg) of two reduced clauses c and c', denoted by lgg(c,c'), is the least upper bound of c and c' in the -subsumption lattice. • Example: • Let c and c' be two clauses:c= daughter(mary,ann) female(mary), parent(ann,mary).c'= daughter(eve,tom) female(eve), parent(tom,eve). • lgg of c and c': daughter(X,Y) female(X), parent(Y,X).

  15. Least General Generalisation 2 • Computation of lgg with -subsumption: • lgg of terms • and V is a variable which represents • and at least one of s and t is a variable in this case, V is a variable which represents • Example: where V stands for lgg(a,b)

  16. Least General Generalisation 3 • lgg of atoms • If atoms have the same predicate symbol p • lgg of literals • If and are atoms, then is computed as defined above • If both and are negative literals and then • If is a positive and is a negative literal, or vice versa,is undefined • Example:

  17. Least General Generalisation 4 • lgg of clauseLet and . Then • Examples:If and thenwhere X stands for lgg(mary,eve) and Y stands for lgg(ann,tom)

  18. Least General Generalisation 5 • Search lgg( , ) Input: and are atoms or literalsOutput: lgg( , ) = function lgg( , ): beginif head( ) head( ) then := ; return( )else := ; := whiledo find_position( , , , ); generate_variable ; substitute( , , , , )endwhile := ; return( )endifend

  19. Least General Generalisation 6 Input: and are terms or literals Output: and terms procedure find_position( , , , ) begin ifthenreturn( , ) endif if ( is atomic or is atomic) then return( , ) endif ifthenreturn( , ) endif i:=1 while do find_position( , , , ) if thenreturn( , ) endif i:= i+1 endwhile end

  20. Least General Generalisation 7 Input: and are terms or literals, and terms, X variable Output: and with substitution X for and procedure substitute( , , , , X) begin ifandthenreturn(X,X) endif ifthenreturn( , ) endif if ( is atomic or is atomic) thenreturn( , ) endif ; ; if thenreturn( , ) endif i:=1 whiledo substitute( , , , , X); i:= i+1 endwhile end

  21. Generalisation techniques • Start from the most specific clause that covers a given positive example and then generalise the clause until it cannot be further generalised without covering negative examples. • Generalisation operator: Let L be a language bias, a generalisation operator maps a clause c to a set of clauses which are generalisations of c: • Generalisation operators perform two basic syntactic operations on a clause: • Apply an inverse substitution to the clause • Remove a literal from the body of the clause • Basic generalisation techniques: • Relative least generalisation (rlgg) • Inverse resolution

  22. Relative Least Generalisation 1 • Relative least generalisation: The relative least generalisationrelativeto background knowledgeB K is the conjunction of ground facts and are positive example • Example: • Positive example: and and B as before where K is a conjunctionresult:

  23. Relative Least Generalisation 2 Search for rlgg( , ) Input: and are two clauses in the form = { } and = { } Output: rlgg( , ) = function rlgg( , ): begin k := 1; l := 1; while k < n do while l < m do lgg( , , L); ; l := l+1 endwhile k := k+1 endwhile ; return( ) end

  24. Inverse resolution • Basic idea: invert the resolution rule of deductive inference (Robinson) e.g., invert the SLD-Resolution proof procedure for definite programs • Example: Given the theory suppose we want to derive uproposition w resolves with to give v which is then resolved with derive u.

  25. Inverse resolution 2 • Example: first order derivation tree for family example and andLet Suppose we want to derive the fact from

  26. Inverse resolution 3 • Inverse resolution inverts the resolution process using the generalisation operator based on the inverting substitution • Given a W, an inverse substitution of a substitution is a function that maps terms in to variable such that • Example: • Take and the substitution : • Applying the inverse substitution the original clause is obtained • Example: Inverse substitution with places • Let and • Applying to W: • The specifies that the first occurrence of the subterm ann in the term is replaced by variable Xand the second occurrence is replaced by Y. • The use of places ensures that

  27. Inverse resolution 4 • Example: Inverse Resolution • B consists of two clauses and • Let • The learner encounters the positive example: • The inverse resolution processes as follows: • It attempts to find which will together with entail and can be added to the current hypothesis Choosing the inverse resolution step generates becomes the current hypothesis H such that • It takes andBy computing using it generalise with respect to B, yieldingIn the H can be replaced by which together b entailsThe induced hypothesis is

  28. Inverse resolution 4

  29. Specialisation techniques • They search the hypothesis space in top-down manner, from general to specific hypotheses using -subsumption based on a specialisation operator (refinement operator) • Refinement operator: Given a language bias L, a refinement operator maps a clause c to a set of clause which are specialisations (refinements) of c • This operator typically computes only the set of minimum (most general) specialisations of a clause under -subsumptionIt employs two basic syntactic operations on a clause: • Apply a substitution to the clause • Add a literal to the body of the clause

  30. Specialisation techniques (2) • Basic specialisation technique is top-down search of the refinement graph • Top-down learners start from the most general clauses and repeatedly refine them until they no longer cover negative examples

  31. Specialisation techniques 2 • For a selected L and a given B the hypothesis space of program clauses is a lattice structured by the -subsumption generality ordering • In this lattice a refinement graph can be defined and used to direct the search from general to specific hypotheses • The refinement graph is a directed, acyclic graph in which nodes are program clauses and edges correspond to the basic refinement operators: substituting a variable with a term, adding a literal to the body of the clause. • First used Model Inference System (MIS, Shapiro 1983)

  32. Specialisation techniques 3

  33. Specialisation techniques 4 • Generic Top-Down-Algorithm:Input:E the set of positive examples, B the background knowledgeL the description languageOutput: Hypothesis Hprocedure top_down_ILP( E, B, H) begin ; repeat choose repeat find the best refinement ; untilC is acceptable ; until hypothesis H satisfies stopping criterionreturn( H)end

  34. Content • Introduction to ILP • Basic ILP techniques • An overview of the different ILP systems • The application field of ILP • Summary

  35. An overview of the different ILP systems • Prehistory(1970) • Plotkin lgg, rlgg 1970-1971 • Early enthousiasm (1975-1983) • Vere 1975,1977, Summers 1977, Kodratoff and Jouanaud 1979 • Shapiro 1981, 1983 • Dark ages (1983-87)

  36. An overview of the different ILP systems 2 • Renaissance (1987- ...) • MARVIN - Sammut & Banereji 1986, DUCE Muggleton 1987 • Helft 1987 • QuMAS - Mozetic 1987 Linus Lavrac et. al. 1991 • CIGOL - Muggleton &Buntine 1988 • ML-SMART Bergadano Giordana & Saitta 1988 • CLINT - De Raedt & Bruynooghe 1988 • BLIP, MOBAL Morik, Wrobel et. al. 1988 1989 • GOLEM - Muggleton &Feng 1990 • FOIL - quinland 1990 mFOIL - Dzerowski 1991 FOCL - Brunk & Pazzani FFOIL - Quinland 1996 • CLAUDIEN De Raedt & Bruynooghe 1993 • PROGOL - Muggleton et al 1995 • FORS- Karalic & Bratko 1996 • TILDE - Blockeel & De Raedt 1997 • .... • Studies Comparing the inductive learning system Foil with its versions FOCL, mFOIL, FFOIL, FOIL-I, FOSSIL, FOIDL and IFOIL

  37. Content • Introduction to ILP • Basic ILP techniques • An overview of the different ILP systems • The application field of ILP • Summary

  38. The application field of ILP • Application areas: • Knowledge acquisition for expert systems • Knowledge discovery in databases • Scientific knowledge discovery • Logic programs synthesis and verification • Inductive data engineering • Successful application • Finite element mesh design • Structure-activity prediction for drug design • Protein secondary-structure prediction • Predicting mutagenicity of chemical compounds

  39. Summary • Inductive logic programming (ILP) =Inductive concept learning (I) Logic Programming (LP) • Empirical vs. Interactive ILP systems • Generalisation vs. specialisation techniques

More Related