Ch5 Mining Frequent Patterns, Associations, and Correlations

1. Ch5 Mining Frequent Patterns, Associations, and Correlations Dr. Bernard Chen Ph.D. University of Central Arkansas

2. Outline • Association Rules • Association Rules with FP tree • Misleading Rules • Multi-level Association Rules

3. What Is Frequent Pattern Analysis? • Frequent pattern: a pattern (a set of items, subsequences, substructures, etc.) that occurs frequently in a data set • First proposed by Agrawal, Imielinski, and Swami [AIS93] in the context of frequent itemsets and association rule mining

4. What Is Frequent Pattern Analysis? • Motivation: Finding inherent regularities in data • What products were often purchased together? bread and milk? • What are the subsequent purchases after buying a PC? • What kinds of DNA are sensitive to this new drug? • Can we automatically classify web documents? • Applications • Basket data analysis, cross-marketing, catalog design, sale campaign analysis, Web log (click stream) analysis, and DNA sequence analysis.

5. Association Rules

6. Association Rules • support, s, probability that a transaction contains X  Y • confidence, c,conditional probability that a transaction having X also contains Y

7. Association Rules • Let’s have an example • T100 1,2,5 • T200 2,4 • T300 2,3 • T400 1,2,4 • T500 1,3 • T600 2,3 • T700 1,3 • T800 1,2,3,5 • T900 1,2,3

8. Association Rules with AprioriMinimum support=2/9Minimum confidence=60%

9. The Apriori Algorithm • Pseudo-code: Ck: Candidate itemset of size k Lk : frequent itemset of size k L1 = {frequent items}; for(k = 1; Lk !=; k++) do begin Ck+1 = candidates generated from Lk; for each transaction t in database do increment the count of all candidates in Ck+1 that are contained in t Lk+1 = candidates in Ck+1 with min_support end returnkLk;

10. Strong Association Rule • Strong association rules means the frequent rules that also pass the minimum confidence. • For example frequent rules: {I1, I2} • Confidence(I1->I2)= 4/6 (strong association rule!) • Confidence(I2->I1)= 4/7

11. Exercise • A dataset has five transactions, let min-support=60% and min_support=80% • Find all frequent itemsets using Apriori and all strong association rules

12. Association Rules with Apriori K:5KE:4 KE E:4 KM:3 KM M:3KO:3 KO O:3 => KY:3 => KY => KEO Y:3 EM:2 EO EO:3 EY:2 MO:1 MY:2 OY:2

13. Outline • Association Rules • Association Rules with FP tree • Misleading Rules • Multi-level Association Rules

14. Mining Frequent Itemsets without Candidate Generation • In many cases, the Apriori candidate generate-and-test method significantly reduces the size of candidate sets, leading to good performance gain. • However, it suffer from two nontrivial costs: • It may generate a huge number of candidates (for example, if we have 10^4 1-itemset, it may generate more than 10^7 candidata 2-itemset) • It may need to scan database many times

15. Association Rules with AprioriMinimum support=2/9Minimum confidence=70%

16. Bottleneck of Frequent-pattern Mining • Multiple database scans are costly • Mining long patterns needs many passes of scanning and generates lots of candidates • To find frequent itemset i1i2…i100 • # of scans: 100 • # of Candidates: (1001) + (1002) + … + (110000) = 2100-1 = 1.27*1030 ! • Bottleneck: candidate-generation-and-test • Can we avoid candidate generation?

17. Mining Frequent Patterns WithoutCandidate Generation • Grow long patterns from short ones using local frequent items • “abc” is a frequent pattern • Get all transactions having “abc”: DB|abc • “d” is a local frequent item in DB|abc  abcd is a frequent pattern

18. Process of FP growth • Scan DB once, find frequent 1-itemset (single item pattern) • Sort frequent items in frequency descending order • Scan DB again, construct FP-tree

19. Association Rules • Let’s have an example • T100 1,2,5 • T200 2,4 • T300 2,3 • T400 1,2,4 • T500 1,3 • T600 2,3 • T700 1,3 • T800 1,2,3,5 • T900 1,2,3

20. FP Tree

21. Mining the FP tree

22. Benefits of the FP-tree Structure • Completeness • Preserve complete information for frequent pattern mining • Never break a long pattern of any transaction • Compactness • Reduce irrelevant info—infrequent items are gone • Items in frequency descending order: the more frequently occurring, the more likely to be shared • Never be larger than the original database (not count node-links and the count field) • For Connect-4 DB, compression ratio could be over 100

23. Exercise • A dataset has five transactions, let min-support=60% and min_confidence=80% • Find all frequent itemsets using FP Tree

24. Association Rules with FP Tree K:5 E:4 M:3 O:3 Y:3

25. Association Rules with FP Tree Y: KEMO:1 KEO:1 KY:1 K:3 KY O: KEM:1 KE:2 KE:3 KO EO KEO M: KE:2 K:1 K:3 KM E: K:4 KE

26. FP-Growth vs. Apriori: Scalability With the Support Threshold Data set T25I20D10K

27. Why Is FP-Growth the Winner? • Divide-and-conquer: • decompose both the mining task and DB according to the frequent patterns obtained so far • leads to focused search of smaller databases • Other factors • no candidate generation, no candidate test • compressed database: FP-tree structure • no repeated scan of entire database • basic ops—counting local freq items and building sub FP-tree, no pattern search and matching

28. Outline • Association Rules • Association Rules with FP tree • Misleading Rules • Multi-level Association Rules

29. Example 5.8 Misleading “Strong” Association Rule • Of the 10,000 transactions analyzed, the data show that • 6,000 of the customer included computer games, • while 7,500 include videos, • And 4,000 included both computer games and videos

30. Misleading “Strong” Association Rule • For this example: • Support (Game & Video) = 4,000 / 10,000 =40% • Confidence (Game => Video) = 4,000 / 6,000 = 66% • Suppose it pass our minimum support and confidence (30% , 60%, respectively)

31. Misleading “Strong” Association Rule • However, the truth is : “computer games and videos are negatively associated” • Which means the purchase of one of these items actually decreases the likelihood of purchasing the other. • (How to get this conclusion??)

32. Misleading “Strong” Association Rule • Under the normal situation, • 60% of customers buy the game • 75% of customers buy the video • Therefore, it should have 60% * 75% = 45% of people buy both • That equals to 4,500 which is more than 4,000 (the actual value)

33. From Association Analysis to Correlation Analysis • Lift is a simple correlation measure that is given as follows • The occurrence of itemset A is independent of the occurrence of itemset B if P(AUB) = P(A)P(B) • Otherwise, itemset A and B are dependent and correlated as events • Lift(A,B) = P(AUB) / P(A)P(B) • If the value is less than 1, the occurrence of A is negatively correlated with the occurrence of B • If the value is greater than 1, then A and B are positively correlated

34. Outline • Association Rules • Association Rules with FP tree • Misleading Rules • Multi-level Association Rules

35. Mining Multiple-Level Association Rules • Items often form hierarchies

36. Mining Multiple-Level Association Rules • Items often form hierarchies

37. uniform support reduced support Level 1 min_sup = 5% Milk [support = 10%] Level 1 min_sup = 5% Level 2 min_sup = 5% 2% Milk [support = 6%] Skim Milk [support = 4%] Level 2 min_sup = 3% Mining Multiple-Level Association Rules • Flexible support settings • Items at the lower level are expected to have lower support

38. Multi-level Association: Redundancy Filtering • Some rules may be redundant due to “ancestor” relationships between items. • Example • milk  wheat bread [support = 8%, confidence = 70%] • 2% milk  wheat bread [support = 2%, confidence = 72%] • We say the first rule is an ancestor of the second rule.