Johanna GOLD

1. Rough Sets Theory Logical Analysis of Data. Monday, November 26, 2007 Johanna GOLD

2. Introduction • Comparison of two theories for rules induction. • Different methodologies • Same results?

3. Generalities • Set of objects described by attributes. • Each object belongs to a class. • We want decision rules.

4. Approaches • There are two approaches: • Rough Sets Theory (RST) • Logical Analysis of Data (LAD) • Goal : compare them

5. Contents Rough Sets Theory Logical Analysis Of data Comparison Inconsistencies

6. Inconsistencies • Two examples having the exact same values in all attributes, but belonging to two different classes. • Example: two sick people have the same symptomas but different disease.

7. Covered by RST • RST doesn’t correct or aggregate inconsistencies. • For each class : determination of lower and upper approximations.

8. Approximations • Lower : objects we are sure they belong to the class. • Upper : objects than can belong to the class.

9. Impact on rules • Lower approximation → certain rules • Upper approximation → possible rules

10. Pretreatment • Rules induction on numerical data → poor rules → too many rules. • Need of pretreatment.

11. Discretization • Goal : convert numerical data into discrete data. • Principle : determination of cut points in order to divide domains into successive intervals.

12. Algorithms • First algorithm: LEM2 • Improved algorithms: • Include the pretreatment • MLEM2, MODLEM, …

13. LEM2 • Induction of certain rules from the lower approximation. • Induction of possible rules from the upper approximation. • Same procedure

14. Definitions (1) • For an attribute x and its value v, a block [(x,v)] of attribute-value pair (x,v) is all the cases where the attribute x has the value v. • Ex : [(Age,21)]=[Martha] [(Age,22)]=[David ; Audrey]

15. Definitions (2) • Let B be a non-empty lower or upper approximation of a concept represented by a decision-value pair (d,w). • Ex : (level,middle)→B=[obj1 ; obj5 ; obj7]

16. Definitions (3) • Let T be a set of pairs attribute-value (a,v). • Set B depends on set T if and only if:

17. Definitions (4) • A set T is minimal complex of B if and only if B depends on T and there is no subset T’ of T such as B depends on T’.

18. Definitions (5) • Let T be a non-empty collection of non-empty set of attribute-value pairs. • T is a set of T. • T is a set of (a,v).

19. Definitions (6) • T is a local cover of B if and only if: • Each member T of T is a minimal complex of B. • T is minimal

20. Algorithm principle • LEM2’s output is a local cover for each approximation of the decision table concept. • It then convert them into decision rules.

21. Algorithm

22. Heuristics details Among the possible blocks, we choose the one: • With the highest priority • With the highest intersection • With the smallest cardinal

23. Heuristics details • As long as it is not a minimal complex, pairs are added. • As long as there is not a local cover, minimal complexes are added.

24. Illustration • Illustration through an example. • We consider that the pretreatment has already been done.

25. Data set

26. Cut points • For the attribute Height, we have the values 160, 170 and 180. • The pretreatment gives us two cut points: 165 and 175.

27. Blocks [(a,v)] • [(Height, 160..165)]={1,3,5} • [(Height, 165..180)]={2,4} • [(Height, 160..175)]={1,2,3,5} • [(Height, 175..180)]={4} • [(Hair, Blond)]={1,2} • [(Hair, Red)]={3} • [(Hair, Black)]={4,5,6}

28. First concept • G = B = [(Attraction,-)] = {1,4,5,6} • Here there is no inconsistencies. If there were some, it’s at this point that we have to chose between the lower and the upper approximation.

29. Eligible pairs • Pair (a,v) such as [(a,v)]∩[(Attraction,-)]≠Ø • (Height,160..165) • (Height,165..180) • (Height,160..175) • (Height,175..180) • (Hair,Blond) • (Hair,Black)

30. Choice of a pair • We chose the most appropriate, which is to say (a,v) for which | [(a,v)] ∩ [(Attraction,-)] | is the highest. • Here : (Hair, Black)

31. Minimal complex • The pair (Hair, Black) is a minimal complex because:

32. New concept • B = [(Attraction,-)] – [(Hair,Black)] = {1,4,5,6} - {4,5,6} = {1}

33. Choice of a pair (1) • Through the pairs (Height,160..165), (Height,160..175) and (Hair, Blond). • Intersections having the same cardinality, we chose the pair having the smallest cardinal: (Hair, Blond)

34. Choice of a pair (2) • Problem : • (Hair, Blond) is non a minimal complex. • We chose the following pair: (Height,160..165).

35. Minimal Complex • {(Hair, Blond),(Height,160..165)} is a second minimal complex.

36. End of the concept • {{(Hair, Black)}, {(Hair, Blond), (Height, 160..165)}} is a local cover of [(Attraction,-)].

37. Rules • (Hair, Red) → (Attraction,+) • (Hair, Blond) & (Height,165..180 ) → (Attraction,+) • (Hair, Black) → (Attraction,-) • (Hair, Blond) & (Height,160..165 ) → (Attraction,-)

38. Contents Rough Sets Theory Logical Analysis Of data Comparison Inconsistencies

39. Principle • Work on binary data. • Extension of boolean approach on non-binary case.

40. Definitions (1) • Let S be the set of all observations. • Each observation is described by n attributes. • Each observation belongs to a class.

41. Definitions (2) • The classification can be considered as a partition into two sets • An archive is represented by a boolean function Φ :

42. Definitions (3) • A literal is a boolean variable or its negation: • A term is a conjunction of literals : • The degree of a term is the number of literals.

43. Definitions (4) • A term Tcovers a point if T(p)=1. • A characteristic term of a point p is the unique term of degree n covering p. • Ex :

44. Definitions (5) • A term T is an implicant of a boolean function f if T(p) ≤ f(p) for all • An implicant is called prime if it is minimal (its degree).

45. Definitions (6) • A positive prime patternis a term covering at least one positive example and no negative example. • A negative prime patternis a term covering at least one negative example and no positive example.

46. Example

47. Example • is a positive pattern : • There is no negative example such as • There is one positive example : the 3rd line. • It's a positive prime pattern : • covers one negative example : 4th line. • covers one negative example : 5th line.

48. Pattern generation • symmetry between positive and negative patterns. • Two approaches : • Top-down • Bottom-up

49. Top-down • we associate each positive example to its characteristic term→ it’s a pattern. • we take out the literals one by one until having a prime pattern.

50. Bottom-up • we begin with terms of degree one: • if it does not cover a negative example, it is a pattern • If not, we add literals until having a pattern.