EECS 800 Research Seminar Mining Biological Data

1. EECS 800 Research SeminarMining Biological Data Instructor: Luke Huan Fall, 2006

2. Administrative • No class next Monday (Labor day holiday)

3. Outline for today • Mining with constraints • Summary

4. Maximal & Closed Patterns • Reduce the number of patterns • Maximal patterns: the boundary of the frequent vs. infrequent patterns • Close patterns: an information- lossless compression of frequent patterns

5.  a b c d ab ac ad bc bd cd abc abd acd bcd abcd Borders of Frequent Itemsets • Connected • X and Y are frequent and X is an ancestor of Y implies that all patterns between X and Y are frequent

6. Closed and Maximal Patterns • Solution: Mine closed patterns andmax-patterns • An itemset Xis closedif X is frequent and there exists no super-pattern YX, with the same support as X (proposed by Pasquier, et al. @ ICDT’99) • Closed pattern is a lossless compression of freq. patterns • Reducing the # of patterns and rules • An itemset Xis maximalif X is frequent and there exists no super-pattern YX such that Y is frequent

7. Closed Patterns and Max-Patterns • Exercise. DB = {<a1, a2>, < a1, a2, a3>} • Min_sup = 1. • What is the set of all frequent patterns? • < a1>: 2, < a2>: 2, < a3>: 1, • < a1, a2 >: 2, < a1, a3 >: 1, < a2, a3 >: 1, • < a1, a2, a3 >: 1, • What is the set of max-pattern? • < a1, a2, a3 >: 1 • What is the set of closed itemset? • < a1, a2, a3 >: 1 • < a1, a2>: 2

8. Example of Vertical Data Format

9. Frequent, Closed and Maximal Itemsets Database

10. Mining Quantitative Associations • Techniques can be categorized by how numerical attributes, such as ageorsalary are treated • Static discretization based on predefined concept hierarchies (data cube methods) • Dynamic discretization based on data distribution (quantitative rules, e.g., Agrawal & Srikant@SIGMOD96) • Clustering: Distance-based association (e.g., Yang & Miller@SIGMOD97) • one dimensional clustering then association • Deviation: (such as Aumann and Lindell@KDD99) Sex = female => Wage: mean=$7/hr (overall mean = $9)

11. Which Measures Should Be Used? • lift and 2are not good measures for correlations in large transactional DBs • all-conf or coherence could be good measures (Omiecinski@TKDE’03) • Both all-conf and coherence have the downward closure property • Efficient algorithms can be derived for mining (Lee et al. @ICDM’03sub)

12. Knowledge Discovery (KDD) Process • Preprocessing: Identify the nature of the data • Learning the application domain: relevant prior knowledge and goals of application • Creating a target data set: sampling • Data cleaning and preprocessing: (may take 60% of effort!) • Data reduction and transformation • Find useful features, dimensionality/variable reduction, invariant representation • Data mining: identify the structure of data • Computational task: pattern discovery, classification, clustering, etc. • Subdivide computational tasks into computational component: choosing the mining algorithm(s) • Result evaluation and presentation • Visualization • Hypothesis generation • Prediction

13. KDD Process: Several Key Steps Knowledge • Data mining—core of knowledge discovery process Pattern Evaluation Data Mining Task-relevant Data Selection Data Warehouse Data Cleaning Data Integration Databases

14. Constraint-based (Query-Directed) Mining • Findingallthe patterns in a database autonomously? — unrealistic! • The patterns could be too many but not focused! • Data mining should be an interactiveprocess • User directs what to be mined using a data mining query language (or a graphical user interface) • Constraint-based mining • User flexibility: provides constraints on what to be mined • System optimization: explores such constraints for efficient mining—constraint-based mining

15. Constraints in Data Mining • Knowledge type constraint: • classification, association, etc. • Data constraint — using SQL-like queries • find product pairs sold together in stores in Chicago in Dec.’02 • Dimension/level constraint • in relevance to region, price, brand, customer category • Rule (or pattern) constraint • small sales (price < $10) triggers big sales (sum > $200) • Interestingness constraint • strong rules: min_support  3%, min_confidence  60%

16. Constrained Mining vs. Constraint-Based Search • Constrained mining vs. constraint-based search/reasoning • Both are aimed at reducing search space • Finding all patterns satisfying constraints vs. finding some (or one) answer in constraint-based search in AI • Constraint-pushing vs. heuristic search • It is an interesting research problem on how to integrate them • Constrained mining vs. query processing in DBMS • Database query processing requires to find all • Constrained pattern mining shares a similar philosophy as pushing selections deeply in query processing

17. How to Formalize Constraints • Anti-monotonicity • Monotonicity • Succinctness

18. Anti-Monotonicity in Constraint Pushing • Anti-monotonicity • When an intemset S violates the constraint, so does any of its superset • sum(S.Price)  v is anti-monotone • sum(S.Price)  v is not anti-monotone • Example. C: range(S.profit)  15 is anti-monotone • Itemset ab violates C • So does every superset of ab TDB (min_sup=2)

19. Monotonicity for Constraint Pushing TDB (min_sup=2) • Monotonicity • When an intemset S satisfies the constraint, so does any of its superset • sum(S.Price)  v is monotone • min(S.Price)  v is monotone • Example. C: range(S.profit)  15 • Itemset ab satisfies C • So does every superset of ab

20. Succinctness • Succinctness: • Given A1, the set of items satisfying a succinctness constraint C, then any set S satisfying C is based on A1 , i.e., S contains a subset belonging to A1 • Idea: Without looking at the transaction database, whether an itemset S satisfies constraint C can be determined based on the selection of items • min(S.Price) v is succinct • sum(S.Price)  v is not succinct

21. The Apriori Algorithm — Example Database D L1 C1 Scan D C2 C2 L2 Scan D L3 C3 Scan D

22. Naïve Algorithm: Apriori + Constraint Database D L1 C1 Scan D C2 C2 L2 Scan D L3 C3 Constraint: Sum{S.price} < 5 Scan D

23. The Constrained Apriori Algorithm: Push an Anti-monotone Constraint Deep Database D L1 C1 Scan D C2 C2 L2 Scan D L3 C3 Constraint: Sum{S.price} < 5 Scan D

24. The Constrained Apriori Algorithm: Push a Succinct Constraint Deep Database D L1 C1 Scan D C2 C2 L2 Scan D not immediately to be used L3 C3 Scan D Constraint: min{S.price } <= 1

25. Converting “Tough” Constraints • Convert tough constraints into anti-monotone or monotone by properly ordering items • Examine C: avg(S.profit)  25 • Order items in value-descending order • <a, f, g, d, b, h, c, e> • If an itemset afb violates C • So does afbh, afb* • It becomes anti-monotone! TDB (min_sup=2)

26. Strongly Convertible Constraints • avg(X)  25 is convertible anti-monotone w.r.t. item value descending order R: <a, f, g,d, b, h, c, e> • If an itemset af violates a constraint C, so does every itemset with af as prefix, such as afd • avg(X)  25 is convertible monotone w.r.t. item value ascending order R-1: <e, c, h, b, d, g, f, a> • If an itemset d satisfies a constraint C, so does itemsets df and dfa, which having d as a prefix • Thus, avg(X)  25 is strongly convertible TDB (min_sup=2)

27. Can Apriori Handle Convertible Constraint? • A convertible, not monotone nor anti-monotone nor succinct constraint cannot be pushed deep into the an Apriori mining algorithm • Within the level wise framework, no direct pruning based on the constraint can be made • Itemset df violates constraint C: avg(X)>=25 • Since adf satisfies C, Apriori needs df to assemble adf, df cannot be pruned • But it can be pushed into frequent-pattern growth framework!

28. Handling Multiple Constraints • Different constraints may require different or even conflicting item-ordering • If there exists an order R s.t. both C1 and C2 are convertible w.r.t. R, then thereis no conflict between the two convertible constraints • If there exists conflict on order of items • Try to satisfy one constraint first • Then using the order for the other constraint to mine frequent itemsets in the corresponding projected database

29. What Constraints Are Convertible?

30. Constraint-Based Mining—A General Picture

31. Monotone Antimonotone Strongly convertible Succinct Convertible anti-monotone Convertible monotone Inconvertible A Classification of Constraints

32. Frequent-Pattern Mining: Research Problems • Mining fault-tolerant frequent • Patterns allows limited faults (insertion, deletion, mutation) • Mining truly interesting patterns • Surprising, novel, concise, … • Theoretic foundation of patterns • For compressing data? • For classification analysis? • Application exploration • Pattern discovery in molecule structures • Pattern discovery in bionetworks

33. Mining Biological Data Increasing potential to support biological discoveries Increasing potential to support business decisions End User End User Hypothesis testing DecisionMaking Domain expert Analyst Hypothesis generation Business Analyst Data Presentation Visualization Techniques Visualization Techniques Data Mining Data Analyst Data Mining Data Analyst Information Discovery Information Discovery Data Exploration Statistical Summary, Querying, and Reporting Data Exploration Statistical Summary, Querying, and Reporting Data Preprocessing/Integration, Data Warehouses DBA Data Preprocessing/Integration, Data Warehouses Data Sources DBA Paper, Microarray, Bio-molecule structures, Mass Spectrometry data, … Data Sources Paper, Files, Web documents, Scientific experiments, Database Systems

34. Bio-Data Mining: Classification Schemes • Different views lead to different classifications • Knowledge view: Kinds of knowledge to be discovered • Data view: Kinds of data to be mined • Method view: Kinds of techniques utilized • Application view: Kinds of applications adapted

35. Summary • Constrained item set mining and association rules • The data mining process