1 / 32

04/01/2020

Parallel Algorithms for general Galois lattices building. Fatma BAKLOUTI , Gérard LEVY CERIA fatma.baklouti@dauphine.fr, gerardlevy@dauphine.fr Workshop WAS 2003. 1. 04/01/2020. Workshop WDAS 2003. Plan.  Knowledge Discovery in Databases (KDD).

jvolk
Download Presentation

04/01/2020

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Parallel Algorithms for general Galois lattices building Fatma BAKLOUTI , Gérard LEVY CERIA fatma.baklouti@dauphine.fr, gerardlevy@dauphine.fr Workshop WAS 2003 1 04/01/2020 Workshop WDAS 2003

  2. Plan Knowledge Discovery in Databases (KDD) One tool for data mining : Galois Lattices Problems and solutions :  Row-sharing  Column-sharing Conclusion 2 Workshop WDAS 2003 04/01/2020

  3. Knowledge Discovery in Databases (KDD) ’Knowledge Discovery in Databases’ (KDD) or ‘Data Mining’ (DM) Extraction of interesting (non-trivial, implicit, previously unknown and potentially useful) information (knowledge) or patterns from data in large databases or other information repositories [Fayyad et al., 1996] 3 Workshop WDAS 2003 04/01/2020

  4.  DM emergence factors  Wide Data bases volume – from Gbyte to Tbyte  Clientele report  Example  Analysis of a client basket in mass distribution  Which group or set of products were frequently bought by a client during a passage in a shop? Disposition of product on shelves.  Example : Milk and bread when a client buys milk, does he buy bread too ? 4 Workshop WDAS 2003 04/01/2020

  5.  Various applications  Medecine, Finances, Distribution, telecommunication …  Fields of research Data Base Statistics IHM Learning KDD Etc … Information Science 5 Workshop WDAS 2003 04/01/2020

  6. KDD General Process Text Picture Sound Data Data acquisition Data Preparation Table Selection,cleaning, Transformations, editing integration construction of attributes Description, Data Mining Structure, Explanation Evaluation, Simplification, Model Editing Knowledge Data base Concept Knowledge management 6 Workshop WDAS 2003 04/01/2020

  7.  Books :  Data Mining, • Han & Kamber (Morgan Kaufmann Pubs, 2001)  Mastering Data Mining, • Berry & Linoff (Wiley Computer Publishing, 2000) …  Interesting sites :  http://www.kddnuggets.com  http://www.crisp-dm.org : CRoss-Industry Standard Process for Data Mining - effort de standardization … 7 Workshop WDAS 2003 04/01/2020

  8. Galois Lattices  Using Galois Lattice (mathematical structure) for solving Data Mining problems.  References :  Birkhoff’s Lattice Theory: 1940, 1973  Barbut & Monjardet : 1970  Wille : 1982  Chein, Norris, Ganter, Bordat, …  Diday, Duquenne, …  Emilion, Lévy, Diday, Lambert  Basic Concepts : Context, Galois connection, Concept. 8 Workshop WDAS 2003 04/01/2020

  9. Galois Lattices - Definition Context = (O, A, I) :  O : finite set of examples  A : finite set of attributes  I : binary relation between O and A, (I  O x A) Example : a b c A O 1 1 1 1 2 1 1 3 1 1 9 Workshop WDAS 2003 04/01/2020

  10. Galois Lattices - Definition  Galois connection  Oi  O and Ai A, we define f et g like this :  f : P(O)  P(A) f(Oi) = {a  A / (o,a)  I,  o  Oi} intention  g: P(A)  P(O) g(Ai) = {o  O / (o,a)  I,  a  Ai} extension  f et g are decreasing applications  h =g · f and k = f · g, are :  Increasing O1 O2 h (O1)  h (O2) O1 h (O1) h (O1) = h · h (O1)  Extensive  Idempotent  h and k are closure operators.  (f,g) = Galois connection between P(O) and P(A) 10 Workshop WDAS 2003 04/01/2020

  11. Galois Lattices - Definition  Concept  Oi  O et Ai  A,  (Oi, Ai) is a concept iff Oi is the extension of Aiand Aiis the intention of Oi  Oi= g (Ai) and Ai= f(Oi)  L ={(Oi, Ai)  P(O)P(A) / Oi= g(Ai) et Ai= f(Oi)} : concepts set.  L: ordered set by the relationship ≤  (O1, A1) ≤ (O2, A2) iff O1 O2(or A2 A1).  Galois Lattice  T=(L, ≤) an ordered set of concepts. 11 Workshop WDAS 2003 04/01/2020

  12. Galois Lattices - Definition  Concept: Example  O1 = {6,7}  f(O1)= {a,c}  A1 = {a,c}  g(A1)= {1,2,3,4,6,7}  Remark: h(O1)= g · f(O1)= g(A1) ≠ O1  ({6,7} , {a,c})  L intention extension a 1 1 1 1 1 1 1 b 1 1 1 1 1 1 c 1 1 1 1 d 1 1 1 1 1 e 1 1 1 f g 1 h  ({1,2,3,4,6,7}, {a,c})  L 1 2 3 4 5 6 7 1 1 Because: 1 1 h({1,2,3,4,6,7}) = g · f({1,2,3,4,6,7}) = g ({a,c}) 1 1 1 1 1 = {1,2,3,4,6,7} 1 1 1 12 Workshop WDAS 2003 04/01/2020

  13. 1234567, a 123467, ac 123456, ab 12345, abd 12346, abc 12356, abe 1247, acf 1234, abcd 1235, abde 1236, abce 124, abcdf 135, abdeg 123, abcde 236, abceh 12, abcdef 13, abcdeg 23, abcdeh 1, abcdefg 2, abcdefh 3, abcdegh Ø, abcdefgh 13 Workshop WDAS 2003 04/01/2020

  14. Generalized Galois Lattices  Context : < I, F, d >  T = <F, , , ≤>  Tj= <Fj, j, j, ≤j> for all j de J, J = [1,n]  d: I  F  di = (di1,…, dij,…, din) : description of the individual i relatively to the attributes j of J. 1 2 j n 1  x  I Individuals I di1 dij din f (x) = ∧d(i) i  x Intention i   z  F dk1 dkj dkn k g (z) = { i  I | z ≤ d(i) } Extension 14 Workshop WDAS 2003 04/01/2020

  15. General Galois Lattice - Example F = F1 x F2 x F3 Size : short, medium, high 1 < 2 < 3 Weight : thin, fat 0 < 1 Age : child, adolescent, adult 1 < 2 < 3 F1 F2 F3 Size Weight Age 2 0 1 1 2 0 3 1 f {Cedric, Carine} = {1, 1, 2} Marc Cedric Céline Carine 1 2 3 2 Individuals I g{1, 1, 2}= {Cedric, Carine} 15 Workshop WDAS 2003 04/01/2020

  16. Ø, 313 4,312 3,203 34,202 24,112 134,201 234,102 1234,101 16 Workshop WDAS 2003 04/01/2020

  17. Problems  Large data volume: Partition data on different server nodes Process in parallel locally Group results on one (client) node Post-process Our tool: SDDS (Scalable Distributed Data Structures ) 17 Workshop WDAS 2003 04/01/2020

  18. Solutions : Column-sharing Row-sharing 1 2 3 1 2 3 1 1 C 2 2 C 3 3 4 4 5 5 3 1 2 1 1 1 2 3 1 2 3 2 C2 2 1 4 C1 C3 C4 3 3 2 5 4 4 3 5 5 18 Workshop WDAS 2003 04/01/2020

  19. Row-sharing M2 M1 C1 C2 T1=TG(C1) (X1, z1) T2=TG(C2) (X2 , z2) M g1(z) = g1(z1) = X1? g2(z) = g2(z2) = X2? X = X1U X2 z = z1∧ z2 T=TG(C) gj(z) = {i  Ij: z ≤ d(i) }, for j =1, 2 04/01/2020 . 19 Workshop WDAS 2003

  20. Example j  i  1 2 3 1 1 0 2 2 2 1 0 C 3 0 3 1 4 1 1 1 5 0 1 3 6 0 0 2 7 2 0 0 j  i  1 2 3 j  i  1 2 3 1 1 0 2 5 0 1 3 C2 C1 2 2 1 0 6 0 0 2 3 0 3 1 7 2 0 0 4 1 1 1 T1=GL(C1) T2=GL(C2) T = GL(C) is it egal to the horizontal product of lattices T1= GL (C1) and T2= GL (C2) ? 20 Workshop WDAS 2003 04/01/2020

  21. We apply an algorithm (here Bordat’s respectively lattice T1= GL(C1) and lattice T2= GL(C2). algorithm) to context C1and C2to build Graph of lattice T1= GL(C1) Graph of lattice T2= GL(C2) 21 Workshop WDAS 2003 04/01/2020

  22. Total number of closed pairs ( X , z ) of lattice T1=GL(C1) = 12. pair(1)= X={}, z=(2,3,3) pair(2)= X={1}, z=(1,0,2) pair(3)= X={2}, z=(2,1,0) pair(4)= X={3}, z=(0,3,1) pair(5)= X={4}, z=(1,1,1) pair(6)= X={1,4}, z=(1,0,1) pair(7)= X={2,4}, z=(1,1,0) pair(8)= X={3,4}, z=(0,1,1) pair(9)= X={1,2,4}, z=(1,0,0) pair(10)= X={1,3,4}, z=(0,0,1) pair(11)= X={2,3,4}, z=(0,1,0) pair(12)= X={1,2,3,4}, z=(0,0,0). Total number of closed pairs of T2=GL(C2) = 5 pair (1)= X={}, z=(2,3,3) pair(2)= X={5}, z=(0,1,3) pair(3)= X={6}, z=(2,0,0) pair (4)= X={5,6}, z=(0,0,2) pair(5)= X={5, 6, 7}, z=(0,0,0). 22 Workshop WDAS 2003 04/01/2020

  23. X1 z1  | z2 | X2 {} {5} (0,1,3) {5,6} (0,0,2) {5,6,7} (0,0,0) {7} (2,3,3) (2,0,0) { } {} {5} (0,1,3) {5,6} (0,0,2) {5,6,7} (0,0,0) {7} (2,0,0) (2,3,3) (2,3,3) {1} {1,5,6} (0,0,2) {1} (1,0,2) {1,5} (0,0,2) {1,5,6,7} (0,0,0) {1,7} (1,0,0) (1,0,2) {2,4,7} (1,0,0) {1,2,4} (1,0,0) {1,2,4} (1,0,0) {1,2,4,5} (0,0,0) {1,2,4,5,6} (0,0,0) {1,2,4,5,6,7} (0 ,0,0) {1,2,3,4,5,6,7} (0,0,0) {1,2,3,4} (0,0,0) {1,2,3,4} (0,0,0) {1,2,3,4,5} (0,0,0) {1,2,3,4,5,6} (0,0,0) {1,2,3,4,7} (0,0,0) X = X1 X2 z = z1 z2 {1,3,4,5,6} (0,0,1) {1,3,4} (0,0,1) {1,3,4} (0,0,1) {1,3,4,5} (0,0,1) {1,3,4,5,6,7} (0,0,0) {1,3,4,7} (0,0,0) {1,4} (1,0,1) {1,4} (1,0,1) {1,4,5} (0,0,1) {1,4,5,6} (0,0,2) {1,4,5,6,7} (0,0,0) {1,3,4,7} (0,0,0) {2} {2,7} (2,0,0) {2} (2,1,0) {2,5} (0,1,0) {2,5,6} (0,0,0) {2,5,6,7} (0,0,0) (2,1,0) {2,3,4,5} (0,1,0) {2,3,4} (0,1,0) {2,3,4} (0,1,0) {2,3,4,5,6} (0,0,0) {2,3,4,5,6,7} (0,0,0) {2,3,4,7} (0,0,0) {2,4} (1,1,0) {2,4} (0,1,0) {2,4,5} (0,1 ,0) {2,4,5,6} (0,0,0) {2,4,5,6,7} (0,0,0) {2,4,7} (1,0,0) {3} (0,3,1) {3} (0,3,1) {3,5} (0,1,1) {3,5,6} (0,0,1) {3,5,6,7} (0,0,0) {3,7} (0,0,0) {3,4,5} (0,1,1) {3,4} (0,1,1) {3,4} (0,1,1) {3,4,5,6} (0,0,1) {3,4,5,6,7} (0,0,0) {3,4,7} (0,0,0) {4} (1,1,1) {4} (1,1,1) {4,5} (0,1,1) {4,5,6} (0,0,1) {4,5,6,7} (0,0,0) {4,7} (1,0,0) 04/01/2020 Workshop WDAS 2003 23 Horizontal product of lattices T1 = GL (C1) and T2 = GL (C2)

  24. We apply BORDAT’s algorithm to the full context C. Graph of lattice T = GL(C) 24 Workshop WDAS 2003 04/01/2020

  25. Total number of closed pairs (X, z) of T = GL(C) =15. pair(1)= X={}, z=(2,3,3) pair(2)= X={1}, z=(1,0,2) pair(3)= X={2}, z=(2,1,0) pair (4)= X={3}, z=(0,3,1) pair(5)= X={4}, z=(1,1,1) pair(6)= X={5,}, z=(0,1,3) pair(7)= X={1,4}, z=(1,0,1) pair(8)= X={1,5,6}, z=(0,0,2) pair(9)= X={2,4}, z=(1,1,0) pair(10)= X={2,7}, z=(2,0,0) pair(11)= X={3,4,5}, z=(0,1,1) pair(12)= X={1,2,4,7}, z=(1,0,0) pair(13)= X={1,3,4,5,6}, z=(0,0,1) pair(14)= X={2,3,4,5}, z=(0,1,0) pair(15)= X={1,2,3,4,5,6,7}, z=(0,0,0). T = GL(C) is the horizontal product of lattices T1= GL(C1) and T2= GL(C2) 25 Workshop WDAS 2003 04/01/2020

  26. Column–sharing M1 M2 C1 C2 T1=TG(C2) (X1 , z1) T2=TG(C2) (X2 , z2) M f1 (X) = z1 ? f2(x) = z2 ? X = X1∩ X2 z = (z1, z2) T=TG(C) f1(X) = Ù {d1(i) : i Î X }, and f2(X) = Ù { d2(i) : i Î X }. 26 Workshop WDAS 2003 04/01/2020

  27. Example j  i  1 2 3 1 1 0 2 2 2 1 0 3 0 3 1 C 4 1 1 1 5 0 1 3 6 0 0 2 7 2 0 0 j  i  j  i  1 2 3 1 1 0 1 2 2 2 1 2 0 C1 C2 3 0 3 3 1 4 1 1 4 1 5 0 1 5 3 6 0 0 6 2 T2=GL(C2) T1=GL(C1) 7 2 0 7 0 T = GL(C) is it egal to the vertical product of lattices T1= GL (C1) and T2= GL (C2) ? 27 Workshop WDAS 2003 04/01/2020

  28. Graph of lattice T1= GL(C1) Graph of lattice T2= GL(C2) 28 Workshop WDAS 2003 04/01/2020

  29. Total number of closed pairs ( X , z ) of lattice T1 =GL(C1) = 8. pair(1) : X={}, z =(2,3) pair(2) : X={2}, z = (2,1) pair(3) : X ={3}, z = (0,3), pair(4) : X = {2,4}, z = (1,1) pair(5) : X= {2,7}, z =(2,0) pair(6) : X= {2,3,4,5}, z=(0,1), pair(7) : X = {1,2,4,7}, z=(1,0) pair(8) : X ={1,2 ,3,4,5,6,7}, z=(0,0). Total number of closed pairs ( X , z ) of lattice T1 =GL(C1) = 4. pair(1) : X = {5}, z=(3) pair(2) : X= {1,5,6}, z=(2) pair(3) : X= {1,3,4,5,6}, z=(1) pair(4) : X ={1,2 ,3,4,5,6,7}, z= (0). 29 Workshop WDAS 2003 04/01/2020

  30. X1 X2 {5} {1,5,6} {1,3,4,5,6} [1..7] Z1 z2 (3) (2) (1) (0) {} {} {} {} {} (2,3) (2,3,3) (2,3,2) (2,3,1) (2,3,0) {2} {2} {} {} {} (2,1) (2,1,3) (2,1,2) (2,1,1) (2,1,0) {3} {3} {} {} {3} X = X1 X2 z = (z1, z2) (0,3) (0,3,3) (0,3,2) (0,3,1) (0,3,0) {4} {2,4} {2,4} {} {} (1,1) (1,1,3) (1,1,2) (1,1,1) (1,1,0) {2,7} {2 ,7} {} {} {} (2,0) (2,0,3) (2,0,2) (2,0,1) (2,0,0) {5} {3,4,5} {2,3,4,5} {2,3,4,5} {5} (0,1) (0,1,3) (0,1,2) (0,1,1) (0,1,0) {1} {1,4} {1,2,4,7} {1,2,4,7} {} (1,0) (1,0,3) (1,0,2) (1,0,1) (1,0,0) {1,5,6} {1,3,4,5,6} [1..7] [1..7] {5} (0,0) (0,0,3) (0,0,2) (0,0,1) (0,0,0) 04/01/2020 Workshop WDAS 2003 30 T = GL(C) is the vertical product of lattices T1= GL(C1) and T2= GL(C2)

  31. Conclusion Generalized Galois Lattices. Problem of large data base can be perhaps resolved in our way. Sharing context into subsets. Possibility of building different architectures for station’s networks. 31 Workshop WDAS 2003 04/01/2020

  32. Thank you for Your Attention Fatma Baklouti Gérard LEVY fatma.baklouti@dauphine.fr gerardlevy@dauphine.fr 32 Workshop WDAS 2003 04/01/2020

More Related