1 / 62

Frequent Structure Mining

Frequent Structure Mining. Robert Howe University of Vermont Spring 2014. Original Authors. This presentation is based on the paper Zaki MJ (2002). Efficiently mining frequent trees in a forest. Proceedings of the 8th ACM SIGKDD International Conference .

jude
Download Presentation

Frequent Structure Mining

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Frequent Structure Mining Robert Howe University of Vermont Spring 2014

  2. Original Authors • This presentation is based on the paper Zaki MJ (2002). Efficiently mining frequent trees in a forest. Proceedings of the 8th ACM SIGKDD International Conference. • The author’s original presentation was used to make this one. • I further adapted this from Ahmed R. Nabhan’s modifications.

  3. Outline • Graph Mining Overview • Mining Complex Structures - Introduction • Motivation and Contributions of author • Problem Definition and Case Examples • Main Ingredients for Efficient Pattern Extraction • Experimental Results • Conclusions

  4. Outline • Graph Mining Overview • Mining Complex Structures - Introduction • Motivation and Contributions of author • Problem Definition and Case Examples • Main Ingredients for Efficient Pattern Extraction • Experimental Results • Conclusions

  5. Why Graph Mining? • Graphs are convenient structures that can represent many complex relationships. • We are drowning in graph data: • Social Networks • Biological Networks • World Wide Web

  6. UVM • High School • BU • Facebook Data • (Source: Wolfram|Alpha Facebook Report)

  7. Facebook Data • (Source: Wolfram|Alpha Facebook Report)

  8. Biological Data • (Source: KEGG Pathways Database)

  9. Some Graph Mining Problems • Pattern Discovery • Graph Clustering • Graph Classification and Label Propagation • Structure and Dynamics of Evolving Graphs

  10. Graph Mining Framework • Mining graph patterns is a fundamental problem in data mining. • Exponential Pattern Space • Relevant Patterns • Mine • Select • Graph Data • Structure Indices • Exploratory Task • Clustering • Classification

  11. A • A • B • B Basic Concepts • C • C • D • Graph – A graph G is a 3-tuple G = (V, E, L) where • V is the finite set of nodes. • E ⊆ V × V is the set of edges • L is a labeling function for edges and nodes. • Subgraph – A graph G1 = (V1, E1, L1) is a subgraph of G2 = (V2, E2, L2) iff: • V1 ⊆ V2 • E1 ⊆ E2 • L1(v) = L2(v) for all v ∈ V1. • L1(e) = L2(e) for all e ∈ E1.

  12. 3 • A • 5 • B Basic Concepts • 4 • C • D • E • 1 • 2 • Graph Isomorphism – “A bijection between the vertex sets of G1 and G2 such that any two vertices u and v which are adjacent in G1 are also adjacent in G2.” (Wikipedia) • Subgraph Isomorphism is even harder (NP-Complete!)

  13. Basic Concepts • Graph Isomorphism – Let G1 = (V1, E1, L1) and G2 = (V2, E2, L2). A graph isomorphism is a bijective function f: V1 → V2 satisfying • L1(u) = L1( f (u)) for all u ∈ V1. • For each edge e1 = (u,v) ∈ E1, there exists e2 = ( f(u), f(v)) ∈ E2 such that L1(e1) = L2(e2). • For each edge e2 = (u,v) ∈ E2, there exists e1 = ( f –1(u), f –1(v)) ∈ E1 such that L1(e1) = L2(e2).

  14. Discovering Subgraphs • TreeMiner and gSpan both employ subgraph or substructure pattern mining. • Graph or subgraph isomorphism can be used as an equivalence relation between two structures. • There is an exponential number of subgraph patterns inside a larger graph (as there are 2n node subsets in each graph and then there are edges.) • Finding frequent subgraphs (or subtrees) tends to be useful in data mining.

  15. Outline • Graph Mining Overview • MiningComplex Structures - Introduction • Motivation and Contributions of author • Problem Definition and Case Examples • Main Ingredients for Efficient Pattern Extraction • Experimental Results • Conclusions

  16. Mining Complex Structures • Frequent structure mining tasks • Item sets – Transactional, unordered data. • Sequences – Temporal/positional, text, biological sequences. • Tree Patterns – Semi-structured data, web mining, bioinformatics, etc. • Graph Patterns – Bioinformatics, Web Data • “Frequent” is a broad term • Maximal or closed patterns in dense data • Correlation and other statistical metrics • Interesting, rare, non-redundant patterns.

  17. Anti-Monotonicity • The black line is always decreasing • A monotonic function is a consistently increasing or decreasing function*. • The author refers to a monotonically decreasing function as anti-monotonic. • The frequency of a super-graph cannot be greater than the frequency of a subgraph (similar to Apriori). • * Very Informal Definition • (Source: SIGMOD ’08)

  18. Outline • Graph Mining Overview • Mining Complex Structures - Introduction • Motivation andContributions of author • Problem Definition and Case Examples • Main Ingredients for Efficient Pattern Extraction • Experimental Results • Conclusions

  19. Tree Mining – Motivation • Capture intricate (subspace) patterns • Can be used (as features) to build global models (classification, clustering, etc.) • Ideally suited for categorical, high-dimensional, complex, and massive data. • Interesting Applications • Semi-structured Data – Mine structure and content • Web usage mining – Log mining (user sessions as trees) • Bioinformatics – RNA secondary structures, Phylogenetic trees • (Source: University of Washington)

  20. Classification Example • Subgraph patterns can be used as features for classification. • “Hexagons are a commonly occurring subgraph in organic compounds.” • Off-the-shelf classifiers (like neural networks or genetic algorithms) can be trained using these vectors. • Feature selection is very useful too.

  21. Contributions • Systematic subtree enumeration. • Extensions for mining unlabeled or unordered subtrees or sub-forests. • Optimizations • Representing trees as strings. • Scope-lists for subtree occurrences.

  22. Outline • Graph Mining Overview • Mining Complex Structures - Introduction • Motivation and Contributions of author • ProblemDefinition and Case Examples • Main Ingredients for Efficient Pattern Extraction • Experimental Results • Conclusions

  23. How does searching for patterns work? • Start with graphs with small sizes. • Extend k-size graphs by one node to generate k + 1 candidate patterns. • Use a scoring function to evaluate each candidate. • A popular scoring function is one that defines the minimum support. Only graphs with frequency greater than minisup are kept.

  24. How does searching for patterns work? • “The generation of size k + 1 subgraph candidates from size k frequent subgraphs is more complicated and more costly than that of itemsets” – Yan and Han (2002), on gSpan • Where do we add a new edge? • It is possible to add a new edge to a pattern and then find that doesn’t exist in the database. • The main story of this presentation is on good candidate generation strategies.

  25. TreeMiner • TreeMiner uses a technique for numbering tree nodes based on DFS. • This numbering is used to encode trees as vectors. • Subtrees sharing a common prefix (e.g. the first k numbers in vectors) form an equivalence class. • Generate new candidate (k + 1)-subtrees from equivalence classes of k-subtrees (e.g. Apriori)

  26. TreeMiner • This is important because candidate subtrees are generated only once! • (Remember the subgraph isomorphism problem that makes it likely to generate the same pattern over and over)

  27. Definitions • Tree – An undirected graph where there is exactly one path between any two vertices. • Rooted Tree – Tree with a special node called root. • This tree has no root node. • It is an unrooted tree. • This tree has a root node. • It is a rooted tree.

  28. Definitions • Ordered Tree – The ordering of a node’s children matters. • Example: XML Documents • Exercise – Prove that ordered trees must be rooted. • ≠ • v2 • v1 • v3 • v1 • v2 • v3

  29. Definitions • Labeled Tree – Nodes have labels. • Rooted trees also have some special terminology. • Parent – The node one closer to the root. • Ancestor – The node n edges closer to the root, for any n. • Siblings – Two nodes with the same parent. • ancestor • embedded sibling • parent • embedded sibling • sibling • ancestor(X,Y) :- • parent(X,Y). • ancestor(X,Y) :- • parent(Z,Y), • ancestor(X,Z). • sibling(X,Y) :- • parent(Z,X), • parent(Z,Y).

  30. Definitions • Embedded Siblings – Two nodes sharing a common ancestor. • Numbering – The node’s position in a traversal (normally DFS) of the tree. • A node has a number ni and a label L(ni). • Scope – The scope of a node nl is [l, r], where nris the rightmost leaf under nl (again, DFS numbering).

  31. Definitions • v0 • Embedded Subtrees – S = (Ns, Bs) is an embedded subtree of T = (N, B)if and only if the following conditions are met: • Ns ⊆ N (the nodes have to be a subset). • b = (nx, ny) ∊ Bs iff nx is an ancestor of ny. • For each subset of nodes Ns there is one embedded subtree or subforest. • v1 • v6 • v7 • v8 • v2 • v3 • v4 • v5 • subtree • v1 • v4 • v5 • (Colors are only on this graph to show corresponding nodes)

  32. Definitions • v0 • Match Label – The node numbers (DFS numbers) in T of the nodes in S with matching labels. • A match label uniquely identifies a subtree. • This is useful because a labeling function must be surjective but will not necessarily be bijective. {v1, v4, v5} or {1, 4, 5} • v1 • v6 • v7 • v8 • v2 • v3 • v4 • v5 • subtree • v1 • v4 • v5 • (Colors are only on this graph to show corresponding nodes)

  33. Definitions • v0 • Subforest – A disconnected pattern generated in the same way as an embedded subtree. • v1 • v6 • v7 • v8 • v2 • v3 • v4 • v5 • subforest • v1 • v7 • v4 • v8 • (Colors are only on this graph to show corresponding nodes)

  34. Problem Definition • Given a database (forest) D of trees, find all frequent embedded subtrees. • Frequent – Occurring a minimum number of times (use user-defined minisup). • Support(S) – The number of trees in D that contain at least one occurrence of S. • Weighted-Support(S) – The number of occurrences of S across all trees in D.

  35. v1 • v0 Exercise • v1 • v6 • v7 • v2 • v5 • v8 • v2 • v3 Generate an embedded subtree or subforest for the set of nodes Ns = {v1, v2, v5}. Is this an embedded subtree or subforest, and why? Assume a labeling function L(x) = x. • v4 • v5 • This is an embedded subtree because all of the nodes are connected. • (*Cough* Exam Question *Cough*)

  36. Outline • Graph Mining Overview • Mining Complex Structures - Introduction • Motivation and Contributions of author • Problem Definition and Case Examples • MainIngredients for Efficient Pattern Extraction • Experimental Results • Conclusions

  37. Main Ingredients • Pattern Representation • Trees as strings • Candidate Generation • No duplicates. • Pattern Counting • Scope-based List (TreeMiner) • Pattern-based Matching (PatternMatcher)

  38. String Representation • With N nodes, M branches, and a max fanout of F: • An adjacency matrix takes (N)(F + 1) space. • An adjacency list requires 4N – 2 space. • A tree of (node, child, sibling) requires 3N space. • String representation requires 2N – 1 space.

  39. 0 String Representation • 2 • 1 • 3 • 2 • String representation is labels with a backtrack operator, –1. • 1 • 2

  40. Candidate Generation • Equivalence Classes – Two subtrees are in the same equivalence class iff they share a common prefix string P up to the (k – 1)-th node. • This gives us simple equivalence testing of a fixed-size array. • Fast and parallel – Can be run on a GPU. • Caveat – The order of the tree matters.

  41. Candidate Generation • Generate new candidate (k + 1)-subtrees from equivalence classes of k-subtrees. • Consider each pair of elements in a class, including self-extensions. • Up to two new candidates for each pair of joined elements. • All possible candidate subtrees are enumerated. • Each subtree is generated only once!

  42. Candidate Generation • Each class is represented in memory by a prefix string and a set of ordered pairs indicating nodes that exist in that class. • A class is extended by applying a join operator ⊗ on all ordered pairs in the class.

  43. Candidate Generation • Equivalence Class • Prefix String 12 • 1 • 1 • 2 • 4 • 2 • 3 • This generates two elements: (3, v1) and (4, v0) • The element notation can be confusing because the first item is a label and the second item is a DFS node number.

  44. Candidate Generation • Theorem 1. Define a join operator ⊗ on two elements as (x, i) ⊗ (y, j). Then apply one of the following cases: • If i = j and P is not empty, add (y, j) and (y, j + 1) to class [Px]. If P is empty, only add (y, j + 1) to [Px]. • If i > j, add (y, j) to [Px]. • If i < j, no new candidate is possible.

  45. Candidate Generation • Consider the prefix class from the previous example: P = (1, 2) which contains two elements, (3, v1) and (4, v0). • Join (3, v1) ⊗ (3, v1) – Case (1) applies, producing (3, v1) and (3, v2) for the new class P3 = (1, 2, 3). • Join (3, v1) ⊗ (4, v0) – Case (2) applies. (Don’t worry, there’s an illustration on the next slide.)

  46. 1 • 1 Candidate Generation • 2 • 2 • 1 • 1 • 3 • 3 • = • ⊗ • 2 • 2 • 3 • 3 • 3 • A class with prefix {1,2} contains a node with label 3. This is written as (3, v1), meaning a node labeled ‘3’ is added at position 1 in DFS order of nodes. • 3 • Prefix = (1, 2, 3) • New nodes = (3, v2), (3, v1)

  47. 1 • 1 • 1 • 1 Candidate Generation • 2 • 2 • 4 • 2 • 2 • 3 • 3 • 3 • 1 • 3 • 3 • 4 • 2 • = • ⊗ • 3 • Prefix = (1, 2, 3) • New nodes = (3, v2), (3, v1),(4, v0)

  48. The Algorithm TreeMiner( D, minisup ): F1 = { frequent 1-subtrees} F2 = { classes [P]1 of frequent 2-subtrees } for all [P]1 ∈ E do Enumerate-Frequent-Subtrees( [P]1 ) Enumerate-Frequent-Subtrees( [P] ): for each element (x, i) ∈ [P] do [Px] = ∅ for each element (y, j) ∈ [P] do R = { (x, i) ⊗ (y, j) } L(R) = { L(x) ∩⊗ L(y) } if for any R ∈ R, R is frequent, then [Px] = [Px] ∪ {R} Enumerate-Frequent-Subtrees( [Px] )

  49. ScopeList Join • Recall that the scope is the interval between the lowest numbered child (or self) node and the highest numbered child node, using DFS numbering. • This can be used to calculate support. • [0, 8] • v0 • [1, 5] • v1 • v6 • v7 • [7, 8] • [2, 2] • v8 • [8, 8] • v2 • v3 • [3, 5] • v4 • v5 • [4, 4] • [5, 5]

  50. ScopeList Join • ScopeLists are used to calculate support. • Let x and y be nodes with scopes sx = [lx, ux], sy = [ly, uy]. • sx contains syifflx ≤ ly and ux ≥ uy. • A scope list represents the entire forest.

More Related