Graph Substructure Search

1. Graph Substructure Search Xuemin Lin School of Computer Science and Engineering University of New South Wales Sydney, Australia

2. Applications of Graphs • Chem-informatics • Chemical Compounds (Small Size) • Bio-informatics • Protein Interaction Networks (Medium Size) • Other Applications • Social Networks (Large Size) • … …

3. Fundamental Problems in Graph Database • Given a graph database D = {g1, ..., gn} of n data graphs and a query graph q, • Substructure Search Retrieve all data graphs which contain q. • Application : Chemical Compounds’ substructure Identification et al . • Supstructure Search Retrieve all data graphs which are contained by q. • Application : Molecule Function Prediction et al.

4. Substructure Search

5. Similarity Search? Input Mistake Exploration Queries ......

6. Outline • Substructure Search (VLDB08) • Substructure Similarity Search (SIGMOD10) • Superstructure Search (SSDBM10) • Superstructure Similarity Search (ICDE2010) • Conclusions and Remarks

7. Substructure Search: gIndex (SIGMOD04) • Index a set F of features from D. • , sup(f): set of graph ids in D contain f • Filtering: • Verification: verify each data graph in Cq.

8. q g1 g2 g3 gIndex (SIGMOD04) Feature A: ID-List: {g1,g2} Filtering: Pass (Feature A) Pass (Feature A) Pruned Verification: False Positive Answer

9. Tree+Delta (VLDB07) Objective: reduce the costs for • building index Observation: • lower costs for generating tree-based features. • most (95%) frequent subgraphs (in sparse graphs) are trees. Tree+Delta: • select frequent tree features, and then • add a small number of effective subgraphs.

10. FG Index (SIGMOD07) Objective: Index only query processing. • q is a frequent subgraph: • if q is indexed, then return sup(q). • if a supergraph q’ of q is indexed, no verification for sup (q’). • q is not a frequent subgraph: • # of verifications is bounded by

11. QuickSI (VLDB08): our work Objective: develop efficient verification algorithm • speed up both verification and filtering. QuickSI: An efficient verification algorithm. • Encode query graphs: terminate earlier. • Enforce connectivity • Three novel pruning techniques Up to orders of magnitude speed up.

12. q g QuickSI Depth-First Traversal 3 4 1 2 5 7 6 1 2 Forwarding 3 4 5 Backtracking 7 6

13. q g QuickSI Depth-First Traversal 3 4 1 2 5 7 6 1 2 Forwarding 3 4 7 Backtracking 5 6

14. q g QuickSI Depth-First Traversal 3 4 1 2 5 7 6 Forwarding 3 4 1 2 5 7 6

15. q g QuickSI Depth-First Traversal Access infrequent labels as early as possible

16. q g QuickSI Depth-First Traversal Access infrequent labels as early as possible 1 1 1 1 1 1 1 1 1 1 1

17. q g QuickSI Synchronized Depth-First Traversal 2 Access infrequent labels as early as possible 1 Sparse Graph! 2x5=10 possible matching pairs Retain connectivity 1 2 2 1 2 2 2

18. q g QuickSI Depth-First Traversal Access infrequent labels as early as possible 1 2 Sparse Graph! ONLY 2 possible matching pairs Retain connectivity 1 2 1 2

19. q g QuickSI Access infrequent labels as early as possible Depth-First Traversal Deg=3 3 4 1 2 2 5 Retain connectivity 7 6 Stop here Deg=2 Effectively use degree information 1 2 2 3 4 5 7 6

20. q g QuickSI Access infrequent labels as early as possible Depth First Traversal Deg=3 3 4 Retain connectivity 1 2 5 7 6 Effectively use degree information Stop here Deg=2 1 2 3 4 1 2 5 7 6 Continue Deg=3 Determine the access order for q.

21. Experimental Results Settings AIDS Antiviral dataset, a popular benchmark, 43k chemical bonds, “C” “N” “O” are the most frequent labels. The data sets and query sets are same as in gIndex and FG Index

22. Experiments – Response Time Real dataset Large real dataset

23. Substructure Similarity Search – Grafil (SIGMOD07) Maximum Common Subgraph MCS: Given g1 and g2, the common graph of g1 and g2 with the maximal number of edges, mcs(g1, g2). Grafil: find all g in D s.t. (|q| - |mcs(q, g)|) ≤ σ (|q| - |mcs(q, g)|): number of missing edges Some variants...

24. Substructure Similarity Search Subgraph Similarity ?

25. Connected Substructure Similarity Search: our work (SIGMOD10) Maximum Connected Common Subgraph MCCS: Given g1 and g2, the connected common graph of g1 and g2 with the maximal number of edges, mccs(g1, g2). dis (q, g) := |q| - |mccs(q, g)| Goal: find all g in D s.t. dis(q,g) ≤σ NP-Complete.

26. Filtering: triangular inequality? dis(Q,D)+dis(D,F) ≥ dis(Q,F)  dis(Q, D) ≥ dis(Q,F) – dis(D,F) dis(Q,D) dis(Q,F) dis(D,.F) Query (Q) Feature(F) Data (D)

27. Filtering: triangular inequality? dis(Q,D)+dist(D,F) ≥ dist(Q,F) ? 1 1 dis(Q,D) Query (Q) Data (D)

28. Similarity Search (triangular inequality) dist(Q,D)+dist(D,F) ≥ dist(Q,F) ? 1 2 2 dist(F,D) Feature(F) Data (D)

29. Filtering: triangular inequality? dis(Q,D)+dis(D,F) ≥ dis(Q,F) ---- HOLD! 1 2 2 2 dis(Q,F) Query (Q) Feature(F)

30. Triangular inequality: not always hold dis(Q,D)+dis(D,F) ≥ dis(Q,F) X 0 1 3 dist(Q,D) dist(Q,F) dist(D,F) Query(Q) Feature(F) Data (D)

31. Connectivity Dominance Connectivity Dominance: The connectivity of mccs(g1, g2) dominates the connectivity of g2 if there is a subgraph isomorphic mapping F from mccs(g1, g2) to g2 such that if removing a set S of edges in mccs(g1, g2) causes mccs(g1, g2) disconnected, then removing F(S) always causes g2 disconnected. Theorem. Given three graphs g1, g2, and g3, if the connectivity of mccs(g1, g2) dominates g2 or the connectivity of mccs(g2, g3) dominates g2, then dist(g1, g3) ≤ dist(g1, g2) + dist(g2, g3). Remark: Linear Algorithm given embeeding.

32. dist(Q,F)+dist(F,D) ≥ dist(Q,D) Validation Rule 1 – index only dist(Q,F)+dist(F,D) ≤ => dist(Q,D) ≤ (if mccs(Q, F) dominates F or mccs(F, D) dominates F) dist(Q,D)+dist(D,F) ≥ dist(Q,F) Pruning Rule 1: dist(Q,F)-dist(D,F)> => dist(Q,D)> (if mccs(D, F) dominates D) dist(F,Q)+dist(Q,D) ≥ dist(F,D) Pruning Rule 2: dist(F, D)-dist(F, Q)> => dist(Q,D)> (if mccs(F, Q) dominates Q)

33. Verification Algorithm Basic idea: 1. enumerate sub-spanning trees of query graph such that the # of missing edges ≤ ; try to terminate the algorithm as early as possible. 2. sharing the enumeration costs by two ways: a. not enumerate every thing from scratch. b. once enumerated, keep enumerated spanning trees. – organized in a binary tree to reduce storage space. 3. extend QucikSI [VLDB08].

34. Experiments

35. Experiments

36. fa fb q (gb) fc (gc) cIndex [VLDB’07]: super structure search • Filtering-Verification Framework • Filter false results by a feature-based index: exclusion based. • Verify each candidate against the query graph. Database Index Query (ga) Filtered! Filtered! Verification Filtering Answer! Candidate!

37. GPTree [EDBT’09] • Enhanced Filtering-Verification Framework • Share test cost in filtering and verification, respectively. Sharing between two phases? sharing across groups? b a (ga) b c c c a a fa c b b b a a c a (gb) c (fa) (fb) c a b a a a sharing across suffixes? a a b c b c a (gc) (ga) (gb) (gc) fb c a c

38. PrefIndex [SSDBM10] • Sharing-based Filtering-Verification Framework • Share test cost in filtering and verification, respectively. sharing between two phases sharing across groups b a (ga) b c c c a a fa c b b b a a c a (gb) c (fa) (fb) c a b a a a No sharing across suffixes a a b c b c a (gc) (ga) (gb) (gc) fb c a c

39. Computation Sharing Cost Model • Cost Gain (Computation Sharing Benefits) • Given k master groups of data graphs, assume that 1) all data graphs in each group Gi contain a master feature fi ( 1 ≤ i ≤ k ); 2) the subgraph isomorphism test from fi to a query graph q is costfi; • The total cost gain (computation sharing benefits) from each master group Gican be represented as follows: • Maximized Gain • Cluster the database into a disjoint set of master groups such that the total gain is maximized (NP-hard).

40. Efficiency Test • Database and Query Sets • Database: AIDS10K; • Query Sets: Q20, Q40, Q60 Q80, Q80+;

41. Superstructure Similarity Search: our work (ICDE10) Given a q and a g, dis(q, g) = |g| − |mccs(q, g)|. Superstructure Similarity Search: find all g from D such that dis(q, g) ≤ σ. Note: dis(q, g) = |q| − |mccs(q, g)| in substructure similarity search. Observations: • filtering framework in SIGMOD10 is immediately applicable. • techniques in SIGMOD10 may not be effective for a nearly “super-containment” relationship. • Sharing is possible just like PrefIndex.

42. SG-Enum Index (ICDE2010) Key Ideas: • For a g, enumerate all subgraphs with at most σ edges removed, σ-missing subgraphs. • dis(q, g) ≤σ if and only if q contains a σ-missing subgraph. • Key issues: • Automorphic subgraphs? • Prefix-sharing? • in one g • among different • data graphs • Query processing?

43. SG-Enum Index (ICDE2010) Top-down Construction: • Enumerate all σ-missing subgraphs. • Iteratively, choose an edge as follows: • Always select an edge contained by most σ-missing subgraphs. • Split the group into 2: one contain the edge and another does not contain the edge. Bottom-up Construction: • Generate a sequence for each σ-missing subgraph. • Merge the prefixes by chance. Bottom-up among data graphs. Query algorithm: extends QuickSI.

44. Experiments – Query Response Time

45. Conclusion and Remarks • Substructure search and its similarity search • Superstructure search and its similarity search. (VLDB08, ICDE10, SIGMOD10, SSDBM10) Issues: • Similarity measures? • Large data graphs?