1. Frequent Pattern Mining Toon Calders Bart Goethals ADReM research group

2. Outline • What is data mining? • Definition • local patterns vs global models • Supervised vs Unsupervised • What do we do? • Frequent set mining • More complex data types

3. What is data mining? “the use of sophisticated data analysis tools to discover previously unknown, validpatterns and relationships in large data sets.” $ $ $ Data Information

4. Supervised vs Unsupervised • Supervised: • data has been annotated • well-defined task: learn to annotate new data E.g.: examples of good/bad customers • Unsupervised: • only data has been given • no annotation • «  find knowledge » y n y x x x x x x x

5. Local vs Global • Local pattern: • tells something about a small subset of the data E.g. «  90% of the customers that purchase beer also buy chips » • Global model: • fits a global model to the data, a summary E.g. : there is a linear relationship between $ spent and the income of the customers

6. What do we do? • Pattern mining • Local • Unsupervised • Useful for • large datasets • exploration: «  what is this data like? » • Less suitable for • well-studied and understood problem domains

7. Outline • What is data mining? • Frequent set mining • Market Basket analysis • Association rules • Interestingness measures • Numerical attributes • More complex data types

8. Market Basket Analysis • Data: collection of transactions of customers: • Goal: find sets of products frequently occuring together

9. Applications • Supermarket • product placement • special promotions • Websearch • which keywords often occur together in webpages? • Health care • frequent sets of symptoms for a disease

10. Applications • Basically works for all data that can be represented as a set of examples/objects having certain properties • patient / symptoms • movies / ratings • web pages / keywords • basket / products • …

11. Algorithms • Computationally a very hard problem • with n products, 2nsets of products • Hundreds of algorithms have been proposed • for sparse/dense data • many rows/columns • data fits/does not fit in memory • …

12. Association Rules • Conditional probabilities XY (c%): if X is in the transaction, then there is a probability of c% that Y is in it as well. • Based on the frequent sets, associations can be computed easily: { Beer, Chips }  { Snack nuts } 75% { adrem.html, cnts.html }  { islab.html } 80% { rain }  { overcast } 100%

13. Interestingness Measures • Not all association rules are interesting • Domain knowledge pregnant  female, rain  overcast • Redundancy A  B (100%) then: AC  B, AD  B, … • Independence 70% buys product A: XA(70%), YA(70%) • Too many rules

14. Interestingness Measures • Incorporating background knowledge • e.g., via Bayesian network • only produce rules that deviate from background knowledge • Redundancies • Condensed representations: produce only a non-redundant subset of patterns

15. Interestingness Measures • Independence • statistical significance tests • X2 • Careful with conclusions !!1000 tests with significance level 0.05 …(Bonferroni correction) • Too many rules • Constraints • Top-k mining

16. Numerical Attributes • Association rule mining is also possible for numerical attributes • discretization: make continuous attributes ordinal • information loss • not appropriate if the order between the values is important • other methods: • recently new method based on rank correlation measures

17. Complex Patterns • Sets • Sequences • Graphs • Relational Structures • Generation and Counting of such patterns becomes much more complex too!

18. Sequences CGATGGGCCAGTCGATACGTCGATGCCGATGTCACGA

19. Patterns in Sequences • Substrings • Regular expressions (bb|[^b]{2}) • Partial orders • Directed Acyclic Graphs

20. Graphs

21. Patterns in Graphs

22. Rules f: 7 f: 8 f: 5 0.5 0.8 0.57 f: 4 f: 4 f: 4

23. Relational Databases

24. Patterns in RDBs • Queries • Query 1: Select L.drinker, V.barFrom Likes L, Visits VWhere V.drinker = L.drinkerAnd L.beer = ‘Duvel’

25. Patterns in RDBs • Query 2: Select L.drinker, V.barFrom Likes L, Visits V, Serves SWhere V.drinker = L.drinkerAnd L.beer = ‘Duvel’And S.bar = V.barAnd S.beer = ‘Duvel’

26. Patterns in RDBs • Association Rule: Query 1 => Query 2 If a person that likes Duvel visits bar, then that bar serves Duvel