Parallel Graph Algorithms

1. Parallel Graph Algorithms Sathish Vadhiyar

2. Graph Traversal • Graph search plays an important role in analyzing large data sets • Relationship between data objects represented in the form of graphs • Breadth first search used in finding shortest path or sets of paths

3. Level-synchronized algorithm • Proceeds level-by-level starting with the source vertex • Level of a vertex – its graph distance from the source • Also, called frontier-based algorithm • The parallel processes process a level, synchronize at the end of the level, before moving to the next level – Bulk Synchronous Parallelism (BSP) model • How to decompose the graph (vertices, edges and adjacency matrix) among processors?

4. Distributed BFS with 1D Partitioning • Each vertex and edges emanating from it are owned by one processor • 1-D partitioning of the adjacency matrix • Edges emanating from vertex v is its edge list = list of vertex indices in row v of adjacency matrix A

5. 1-D Partitioning • At each level, each processor owns a set F – set of frontier vertices owned by the processor • Edge lists of vertices in F are merged to form a set of neighboring vertices, N • Some vertices of N owned by the same processor, while others owned by other processors • Messages are sent to those processors to add these vertices to their frontier set for the next level

6. Lvs(v) – level of v, i.e, graph distance from source vs

7. BFS on GPUs

8. BFS on GPUs • One GPU thread for a vertex • In each iteration, each vertex looks at its entry in the frontier array • If true, it forms the neighbors and frontiers • Severe load imbalance among the treads • Scope for improvement

9. Depth First Search (DFS)

10. Parallel Depth First Search • Easy to parallelize • Left subtree can be searched in parallel with the right subtree • Statically assign a node to a processor – the whole subtree rooted at that node can be searched independently. • Can lead to load imbalance; Load imbalance increases with the number of processors (more later)

11. Maintaining Search Space • Each processor searches the space depth-first • Unexplored states saved as stack; each processor maintains its own local stack • Initially, the entire search space assigned to one processor • The stack is then divided and distributed to processors

12. Termination Detection • As processors are independently, how will they know when to terminate the program? • Dijikstra’s Token Termination Detection Algorithm • Based on passing of a token in a logical ring; P0 initiates a token when idle; A processor holds a token until it has completed its work, and then passes to the next processor; when P0 receives again, then all processors have completed • However, a processor may get more work after becoming idle due to dynamic load balancing (more later)

13. Tree Based Termination Detection • Uses weights • Initially processor 0 has weight 1 • When a processor transfers work to another processor, the weights are halved in both the processors • When a processor finishes, weights are returned • Termination is when processor 0 gets back 1 • Goes with the DFS algorithm; No separate communication steps • Figure 11.10

14. Graph Partitioning

15. Graph Partitioning • For many parallel graph algorithms, the graph has to be partitioned into multiple partitions and each processor takes care of a partition • Criteria: • The partitions must be balanced (uniform computations) • The edge cuts between partitions must be minimal (minimizing communications) • Some methods • BFS: Find BFS and descend down the tree until the cumulative number of nodes = desired partition size • Mostly: Multi-level partitioning based on coarsening and refinement (a bit advanced) • Another popular method: Kernighan-Lin

19. Parallel partitioning • Can use divide and conquer strategy • A master node creates two partitions • Keeps one for itself and gives the other partition to another processor • Further partitioning by the two processors and so on…

20. Graph Coloring

21. Graph Coloring Problem • Given G(A) = (V, E) • σ: V {1,2,…,s} is s-coloring of G if σ(i) ≠σ(j) for every (i, j) edge in E • Minimum possible value of s is chromatic number of G • Graph coloring problem is to color nodes with chromatic number of colors • NP-complete problem

22. Parallel graph Coloring – General algorithm

23. Parallel Graph Coloring – Finding Maximal Independent Sets – Luby (1986) I = null V’ = V G’ = G While G’ ≠ empty Choose an independent set I’ in G’ I = I U I’; X = I’ U N(I’) (N(I’) – adjacent vertices to I’) V’ = V’ \ X; G’ = G(V’) end For choosing independent set I’: (Monte Carlo Heuristic) • For each vertex, v in V’ determine a distinct random number p(v) • v in I iff p(v) > p(w) for every w in adj(v) Color each MIS a different color Disadvantage: • Each new choice of random numbers requires a global synchronization of the processors.

24. Parallel Graph Coloring – Gebremedhin and Manne (2003) Pseudo-Coloring

25. Sources/References • Paper: A Scalable Distributed Parallel Breadth-First Search Algorithm on BlueGene/L. Yoo et al. SC 2005. • Paper:Accelerating large graph algorithms on the GPU usingCUDA. Harish and Narayanan. HiPC 2007. • M. Luby. A simple parallel algorithm for the maximal independent set problem. SIAM Journal on Computing. 15(4)1036-1054 (1986) • A.H. Gebremedhin, F. Manne, Scalable parallel graph coloring algorithms, Concurrency: Practice and Experience 12 (2000) 1131-1146.