MULTILEVEL PROCESSING OF LARGE GRAPH PROBLEMS

1. MULTILEVEL PROCESSING OF LARGE GRAPH PROBLEMS A. Brandt The Weizmann Institute of Science UCLA www.wisdom.weizmann.ac.il/~achi

2. Main Concepts Examples: Graph drawing Low-dimensional embedding Graph linear ordering Image segmentation Data clustering Image denoising • Most graph problem originate from fuzzy real-world problems, reformulated as optimization problems.

3. Main Concepts Examples: graph drawing, graph linear ordering, embedding, image segmentation, data clustering, denoising • Most graph problem originate from fuzzy real-world problems, reformulated as optimization problems. • Resulting optimization problems are ill posed and hard to solve • Approximately solved by spectral methods • Very fast multilevel spectral solvers are based on Algebraic MultiGrid (AMG). • Avoid spectral solvers: Better apply the multilevel (AMG-like) process directly to the optim. problem   The obtained solution is closer to optimal than the spectral solution

4. … Main Concepts Examples: graph drawing, graph linear ordering, embedding, image segmentation, data clustering, denoising • Most graph problem originate from fuzzy real-world problems, reformulated as optimization problems. • Apply a multilevel (AMG-like) processing directly to the fuzzy problem • Apply a multilevel (AMG-like) processing directly to the optimization problem • The fast multilevel processing can produce solutions far more adequate than the solution of the optimization reformulation as judged on collections of practical problems in terms of their real set of, sometimes contradictory, objectives.

5. Drawing Graphs Dorit Ron, Ilya Safro

6. Minimum Linear Arrangement Problem j i

7. Minimum Linear Arrangement Problem j i 1 2 3 4 5

8. Minimum Linear Arrangement Problem j i 1 2 3 4 5

9. Minimum Linear Arrangement Problem j i 1 2 3 4 5

10. Minimum Linear Arrangement Problem j i 1 2 3 4 5

11. Minimum Linear Arrangement Problem j i 1 2 3 4 5 The Spectral Method

12. The Spectral Method A, called the graph Laplacian,used in many graph problems Fast eigen solver: Algebraic Multigrid (AMG)

13. Grid Graph Point-by-point minimization of u= average of u's

14. point-by-point minimization of RELAXATION: random initial guess after 5 relaxation sweeps Fast elimination of high eigenvectors Remaining low eigenvectors are smooth Fast local ordering

15. fine grid

16. coarse grid fine grid All low eigenvectors can be accurately interpolated from their coarse-grid values

17. ALGEBRAIC MULTIGRID (AMG) Fine graph

18. ALGEBRAIC MULTIGRID (AMG) Fine graph Coarse variables - a subset Each variable strongly coupled to coarse variables Interpolation Derived by best fitting it to several relaxed vectors for all low eigenvectors Coarse matrix

19. AMG eigen-solver • Recursive: to increasingly coarser grid • Fast: linear time • Super-fast for calculating many eigenfunctions Low-dimensional embedding Electronic structure calculations Wave propagation • Calculating N eigen-functions of a differential operator in O(N logN) computer operations

20. Coarse-level variable =Weighted aggregate of fine-level variables INTERPRETATION: Interpolation: Part of belongs with Aggregate jincludes and the part of each Its “volume”:

21. In the linear arrangement problem: SPECTRAL: Find the best linear arrangement of the coarse aggregates INSTEAD: Alternating relaxation and arrangement that takes the lengths into account Recursive: multi-level

22. P=2: Multilevel approach vs. Spectral method ratio graphs The results of the multilevel approach were obtained without post-processing! Ilya Safro, Dorit Ron, A. Brandt: J. Graph Alg. Appl. 10 (2006) 237-258

23. Image Segmentation Eitan Sharon, Meirav Galun, Dahlia Sharon, Ronen Basri and Achi Brandt Eitan Sharon, Meirav Galun, Dahlia Sharon, Ronen Basri and Achi Brandt NATURE, 17 August 2006

24. The Pixel Graph Couplings {Wij} Reflect intensity similarity Low contrast – strong coupling High contrast – weak coupling

25. Segmentationminimize cut coupling Low-energy cut

26. Normalized-Cut Measure Minimize:

27. In the minimal normalized cut problem: SPECTRAL: • Very fast AMG eigensolver Find minimal normalized cutof the coarse aggregate graph INSTEAD:

28. Coarse-level variable =Weighted aggregate of fine-level variables INTERPRETATION: Interpolation: Aggregate jincludes and the part of each i.e., similarly colored neighboring pixels Coarse graph: edges = sums of pixel similarity

29. In the minimal normalized cut problem: SPECTRAL: • Very fast AMG eigensolver Find minimal normalized cutof the coarse aggregate graph INSTEAD: • Recursive to coareser levels • Very fast : few dozen operations per pixel • Finding a hierarchy of many low cuts BUT …

30. Normalized cuts Coarse couplings modified by aggregative properties • Boundary smoothness • Average color • Non-local matching • Variances • “Hairiness”

31. Specialized segmentation:Detecting Lesions Tagged Our results Data: Filippi

32. Graph problems Partition: min cut Linear arrangement, bandwidth, cutwidth Clustering VLSI placement, Routing Graph drawing Low dimension embedding Image segmentation Fuzzy problems General principle: Multilevel objectives not functional minimization

33. Graph problems Partition: min cut Linear arrangement, bandwidth, cutwidth Clustering VLSI placement, Routing Graph drawing Low dimension embedding Image segmentation Fuzzy problems General principle: Multilevel objectives not functional minimization

34. Clustering

35. A B C

36. k-means Spectral clustering … Diffusion map

37. Diffusion map Diffusion distance depends on the number of examples

38. Diffusion map “Diffusion distance” depends on the number of examples

39. Multiscale Clustering Bottom/up Aggregative Properties: • Average coupling

40. Multiscale Clustering Bottom/up Aggregative Properties: • Average coupling

41. Multiscale Clustering Bottom/up Aggregative Properties: • Average coupling • Direction

42. Bottom/up Multiscale Clustering Aggregative Properties: • Average coupling • Direction • Density Dan Kushnir: MSc thesis

43. Density-tuned Basic scheme Scale 6 Scale 6 Scale 7 Scale 7 Ya’ara Goldschmidt: PhD thesis

44. Multiscale Clustering Bottom/up Aggregative Properties: • Average coupling • Direction • Density • Low-dimension embedding

45. multiscale dimensionality • Data sets exhibit multiscale dimensionality on different scales of resolution:

46. Dimensionality Identification PCA: 2-dimensional

47. Multiscale Clustering Bottom/up Aggregative Properties: • Average coupling • Density • Direction • Low-dimension embedding Top/downsharpening FAST:low linear complexity

48. Fast Multi-Scale Clustering Application to Cold and Dark Matter Simulations D. Kushnir, M. Galun and A. Brandt, Pattern Recognition 39 (10), 2006.

49. Results – clustering CDM simulation • Both uniformity in density of the clusters and interesting characteristics of shape are detected. • Geometric object identification was also applied.

50. … Main Concepts Examples: graph drawing, graph linear ordering, embedding, image segmentation, data clustering, denoising • Most graph problem originate from fuzzy real-world problems, reformulated as optimization problems. • Apply a multilevel (AMG-like) processing directly to the fuzzy problem • The fast multilevel processing can produce solutions far more adequate than the solution of any optimization reformulation as judged on collections of practical problems in terms of their real set of, sometimes contradictory, objectives.