Rule Generation using DM in Training on Expert system Development using Agri-Daksh

1. Rule Generation using DMin Training on Expert system Development using Agri-Daksh Rajni Jain rajni@ncap.res.in

2. Expert System: Major Constraint • Expert systems are automated systems that provide knowledge like experts and are useful when it is difficult to get expert advice • Development of expert system is challenging because it is to hard to develop the complete rule base which can work like an expert: • Non availability of experts • Scarcity of time • Cost of their time

3. DM: An aid for development of Expert System • DM: Extraction of potentially useful, meaningful and novel patterns from the data using suitable algorithms • It is a great idea to use data mining tools and techniques to extract rules. • reduce the time required for interactions with the domain experts • Rules should never be used without validation from experts

4. Introduction • Importance of Rules • Directly from Decision Tables • DM • DT • RS • Fuzzy • ANN

5. Decision TREE Colour White Red N Y • Used to represent a sequence of decisions to arrive at a particular result

6. Decision-Tree (ID3) • A decision-tree is a set of nodes and leaves where each node tests the value of an attribute and branches on all possible values. • ID3 is the first most popular decision-tree building algorithm. • A statistical property called information gain is used to decide which attribute is best for the node under consideration. • Greedy Algorithm, all attributes till the examples reach leaf node are retained for computation. Parent node attribute is eliminated.

7. CLS (Concept Learning System) • Step 1: If all instances in C are positive, then create YES node and halt. • If all instances in C are negative, create a NO node and halt. • Otherwise select a feature, F with values v1, ..., vn and create a decision node. • Step 2: Partition the training instances in C into subsets C1, C2, ..., Cn according to the values of V. • Step 3: apply the algorithm recursively to each of the sets Ci.

8. CLS contd.. • Note, the trainer (the expert) decides which feature to select.

9. ID3 Algorithm • ID3 improves on CLS by adding a feature selection heuristic. • ID3 searches through the attributes of the training instances and extracts the attribute that best separates the given examples. If the attribute perfectly classifies the training sets then ID3 stops; otherwise it recursively operates on the n (where n = number of possible values of an attribute) partitioned subsets to get their "best" attribute. • The algorithm uses a greedy search, that is, it picks the best attribute and never looks back to reconsider earlier choices.

10. Data Requirements for ID3 • Attribute value Description: The same attributes must describe each example and have a fixed number of values • Predefined classes: An example’s attributes must already be defined as they are not learned by ID3 • Discrete Classes • Sufficient Examples

11. Attribute Selection in ID3 • How does ID3 decides which attribute is the best? • Information Gain, a statistical property is used • Information gain measures how well a given attribute separates training examples into targeted classes • The attribute with the highest information gain is selected • Entropy measures the amount of the information

12. Entropy Given a collection S of c outcomes Entropy(S) = S -p(I) log2 p(I) where p(I) is the proportion of S belonging to class I. S is over c. Log2 is log base 2. Note that S is not an attribute but the entire sample set.

13. Example 1: Entropy If S is a collection of 14 examples with 9 YES and 5 NO examples then Entropy(S) = - (9/14) Log2 (9/14) - (5/14) Log2 (5/14) = 0.940 Notice entropy is 0 if all members of S belong to the same class (the data is perfectly classified). The range of entropy is 0 ("perfectly classified") to 1 ("totally random").

14. Information Gain • Gain(S, A) is information gain of example set S on attribute A is defined as • Gain(S, A) = Entropy(S) - S ((|Sv| / |S|) * Entropy(Sv)) • Where: • is each value v of all possible values of attribute A • Sv = subset of S for which attribute A has value v • |Sv| = number of elements in Sv • |S| = number of elements in S

15. Example 2: Information Gain Suppose S is a set of 14 examples in which one of the attributes is wind speed. The values of Wind can be Weak or Strong. The classification of these 14 examples are 9 YES and 5 NO. For attribute Wind, suppose there are 8 occurrences of Wind = Weak and 6 occurrences of Wind = Strong. For Wind = Weak, 6 of the examples are YES and 2 are NO. For Wind = Strong, 3 are YES and 3 are NO. Therefore Gain(S,Wind)=Entropy(S)-(8/14)*Entropy(Sweak)-(6/14)*Entropy(Sstrong) = 0.940 - (8/14)*0.811 - (6/14)*1.00 = 0.048 Entropy(Sweak) = - (6/8)*log2(6/8) - (2/8)*log2(2/8) = 0.811 Entropy(Sstrong) = - (3/6)*log2(3/6) - (3/6)*log2(3/6) = 1.00 For each attribute, the gain is calculated and the highest gain is used in the decision node.

16. Example 3: ID3 • Suppose we want ID3 to decide whether the weather is amenable to playing baseball. Over the course of 2 weeks, data is collected to help ID3 build a decision tree (see table 1). • The target classification is "should we play baseball?" which can be yes or no. • The weather attributes are outlook, temperature, humidity, and wind speed. They can have the following values: • outlook = { sunny, overcast, rain } • temperature = {hot, mild, cool } • humidity = { high, normal } • wind = {weak, strong }

18. Real Example:Summary of dataset • NSSO carried out a nationwide survey on cultivation practices as a part of its 54th round from Jan-June 1998 • The enquiry was carried out in the rural areas of India through house hold survey to ascertain the problems and utilization of facilities and the level of adoption in farms of different sizes in the country • Focus on the spread of improved agricultural technology • Data is extracted for the Haryana State

19. Dataset Summary • 39 independent variables from Haryana • 1 dependent variable (ifpesticide) value 1,2 • 1:adoption of PIF • 2:non-adoption of PIF • 36-nominal, 4 real-valued attributes • 1832 cases • 48% adopting PIF • 52% not adopting

20. Why Haryana? Distribution of Class Variable • Reasonable dataset size (1832 cases) • Independent variable (if pesticide) is some what uniformly distributed in Haryana

21. Variables in Haryana-Farmers Dataset (NSSO, 54th round)

22. Hypothetical Farmer Dataset

23. Seedtype 1 2 3 Crop no Crop rice wheat fodder rice wheat fodder yes no ? no yes yes Decision Tree (DT) using RDT Architecture for Hypothetical Farmer Data • The Tree is simple to comprehend • The tree can be easily traversed to generate rules

24. Description of the codes of the attributes in the Approximate Core based RDT model

25. If weedicide 1 2 1 1 CropCode 99 1 2 3 4 5 6 7 9 2 1 2 2 2 2 2 2 2 2 2 2 11 2 2 2 2 8 Iffertilizer Iffertilizer 1 1 3 2 3 2 1 1 2 1 1 1 2 Approximate Core based RDT model for Farmers Adopting Plant Protection Chemicals Ifweedicide: yes -1, no-2 1 Iffertilizer: entirely -1, partly-2, none-3 Cropecode: paddy -1, wheat -2, other cereals -3, pulses- 4, oil seeds -5, mixed crop – 6, sugarcane -7, vegetables -8, fodder -9, fruits & nuts -10, other cash crops -11, others 99 Ifpesticide:yes-1, no-2

26. Decision Rules for Adopters • If Weedicide= yes then pesticide=yes • If weedicide=no and crop=paddy then pesticide=yes • If weedicide=no and crop=vegetables and fertilizer=entirely then pesticide=yes • If weedicide=no and crop=vegetables and fertilizer=none then pesticide=yes • If weedicide=no and crop=cash and fertilizer=entirely then pesticide=yes • If weedicide=no and crop=cash and fertilizer=partly then pesticide=yes

27. Decision Rules for Non-adopters • If weedicide=no and crop=(wheat, other than rice cereals, pulses, oil seeds, mixed crop, sugarcane, fodder, other crops) then pesticide=no • If weedicide=no and crop = vegetables and fertilizer=partly then pesticide=no • If weedicide=no and crop = other cash crops and fertilizer=none then pesticide=no

28. A case study of forwarning Powdery Mildew of Mango disease

29. Dataset: PWM status for 11 years and corresponding temperature and humidity conditions

30. Techniques for Rule based learning Or The hybrids like RDT NATIONAL CENTRE FOR AGRICULTURAL ECONOMICS AND POLICY RESEARCH

31. Selection of Algorithms • Accuracy: Fraction of test instances which are predicted accurately • Interpretability: Ability to understand • Complexity: No of conditions in the model • No of rules • No f variables Selection of an algorithm involves trade-off with the parameters

32. Train and Test Data

33. The Prediction model for PWM Epidemic as obtained using CJP Algorithm on 1987-97 data as the training dataset >87 <=87 <=30.7 >30.7 NATIONAL CENTRE FOR AGRICULTURAL ECONOMICS AND POLICY RESEARCH

34. Rules • If (H811>87) then Status = 1 • If (H811<=87) and (T814<=30.71) then Status = 0 • If (H811<=87) and (T814>30.71) then Status = 1 NATIONAL CENTRE FOR AGRICULTURAL ECONOMICS AND POLICY RESEARCH

35. Conclusions • Rule induction can be very useful for development of expert systems • Strong linkage and collaborations are required among expert system developer, domain experts and data mining experts

36. THANK YOU

37. Thank you National Centre for Agricultural Economics and Policy Research (NCAP) DPS Marg, Pusa New Delhi – 110 012 Tel: +91 11 25847628; Fax: +91 11 2584 2684 Website: www.ncap.res.in