1 / 47

Adaptive Mesh Applications

Adaptive Mesh Applications. Sathish Vadhiyar. Sources: - Schloegel, Karypis, Kumar. Multilevel Diffusion Schemes for Repartitioning of Adaptive Meshes. JPDC 1997 ( Taken verbatim ). Adaptive Applications. Highly adaptive and irregular applications

sage-gibbs
Download Presentation

Adaptive Mesh Applications

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Adaptive Mesh Applications Sathish Vadhiyar Sources: - Schloegel, Karypis, Kumar. Multilevel Diffusion Schemes for Repartitioning of Adaptive Meshes. JPDC 1997 (Taken verbatim)

  2. Adaptive Applications • Highly adaptive and irregular applications • Amount of work per task can vary drastically throughout the execution – similar to earlier applications, but.. • Has notions of “interesting” regions • Computations in the “interesting” regions of the domain larger than for other regions • It is difficult to predict which regions will become interesting

  3. AMR Applications • An example of such applications is Parallel Adaptive Mesh Refinement (AMR) for multi-scale applications • Adaptive Mesh – Mesh or grid size is not fixed as in Laplace/Jacobi, but interesting regions are refined to form finer level grids/mesh • E.g.: to study crack growth through a macroscopic structure under stress

  4. AMR Applications – Crack propagation • Such a system is subject to the laws of plasticity and elasticity and can be solved using finite element method • Crack growth forces the geometry of the domain to change • This in turn necessitates localized remeshing

  5. AMR Applications- Adaptivity • Adaptivity arises when advances crosses from one subdomain to another • It is unknown in advance when or where the crack growth will take place and which subdomains will be affected • The computational complexity of a subdomain can increase dramatically due to greater levels of mesh refinement • Difficult to predict future workloads

  6. Repartitioning • In adaptive meshes computation, areas of the mesh are selectively refined or derefined in order to accurately model the dynamic computation • Hence, repartitioning and redistributing the adapted mesh across processors is necessary

  7. Repartitioning • The challenge is to keep the repartitioning cost to minimum limits • Similar problems to MD, GoL and Nbody • The primary difference in AMR is that loads can drastically change; cannot predict; will have to wait for refinement, then repartition

  8. Structure of Parallel AMR

  9. Repartitioning • 2 methods for creating a new partitioning from an already distributed mesh that has become load imbalanced due to mesh refinement and coarsening • Scratch-remap schemes create an entirely new partition • Diffusive schemes attempt to tweak the existing partition to achieve better load balance, often minimizing migration costs

  10. Work Stealing • A method of load balancing • Each processor partnered with a neighbor • If its load goes below an application-level threshold, it sends messages to partner asking for work; i.e. tries to steal work • If partner does not have enough units, another partner is contacted • Can also be implemented using a master-worker approach; i.e., worker asking for more work from master

  11. Graph Representation of Mesh • For irregular mesh applications, the computations associated with a mesh can be represented as a graph • Vertices represent the grid cells; vertex weights represent the amount of computations associated with the grid cells • Edges represent the communication between the grid cells; edge weights represent the amount of interactions

  12. Graph Representation of Mesh • The objective is to partition across P processors • Each partition has equal amount of vertex weight • Total weight of the edges cut by the partition is minimized

  13. Scratch-map Method • Partitioning from scratch will result in high vertex migration since as the partitioning does not take the initial location of the vertices into account • Hence a partitioning method should incrementally construct a new partition as simply a modification of the input partition

  14. Notations • Let B(q) be the set of vertices with partition q • Weight of any partition q can be defined as: • Average partition weight: • A graph is imbalanced if it is partitioned, and:

  15. Terms • A partition is over-balanced if its weight is greater than the average partition weight times (1+ ) • If less, under-balanced • The graph is balanced when no partition is over-balanced • Repartitioning – existing partition used as an input to form a new partition

  16. Terms • A vertex is clean if its current partition is its initial partition; else dirty • Border vertex – adjacent vertex in another partition; those partitions are neighbor partitions • TotalV – sum of the sizes of the vertices which change partitions; i.e., sum of the sizes of the dirty vertices

  17. 3 Objectives • Maintain balance between partitions • Minimize edge cuts • Minimize TotalV

  18. Different Schemes • Repartitioning from scratch • Cut-and-paste repartitioning: excess vertices in an overbalanced partition are simply swapped into one or more underbalanced partitions in order to bring these partitions up to balance • The method can optimize TOTALV, but can have a negative effect on the edge-cut

  19. Different Schemes • Another method is analogous to diffusion • Concept is for vertices to move from overbalanced to neighboring underbalanced partitions

  20. Example(Assuming edge and vertex weights as equal to 1)

  21. Example (contd..)

  22. Analysis of the 3 schemes • Thus, cut-and-paste repartitioning minimizes TotalV, while completely ignoring edge-cut • Partitioning the graph from the scratch minimizes edge-cut, while resulting in high TotalV • Diffusion attempts to keep both TotalV and edge-cut low

  23. Diffusion Scheme: An Algorithm

  24. Multi-level Graph Repartitioning • Based on diffusive scheme • Multi-level graph repartitioning based on the k-way multilevel partitioning • Applying graph partitioning directly on a graph representation of a mesh can be time consuming since the graph can be very large

  25. K-way multilevel partitioning algorithm • Has 3 phases: coarsening, partitioning, refinement (uncoarsening) • Coarsening - a sequence of smaller graphs constructed out of an input graph by collapsing vertices together

  26. K-way multilevel partitioning algorithm • When enough vertices are collapsed together so that the coarsest graph is sufficiently small, a k-way partition is found • Finally, the partition of the coarsest graph is projected back to the original graph by refining it at each uncoarsening level using a k-way partitioning refinement algorithm

  27. K-way partitioning refinement • A simple randomized algorithm that moves vertices among the partitions to minimize edge-cut and improve balance • For a vertex v, let neighborhood N(v) be the union of the partitions to which the vertices adjacent to v belong • In a k-way refinement algorithm, vertices are visited randomly

  28. K-way partitioning refinement • A vertex v is moved to one of the neighboring partitions N(v) if any of the following vertex migration criteria is satisfied • The edge-cut is reduced while maintaining the balance • The balance improves while maintaining the edge-cut • This process is repeated until no further reduction in edge-cut is obtained

  29. Multi-level diffusion repartitioning • In the coarsening phase, only pairs of nodes that belong to the same partition are considered for merging • Hence, initial partition of the coarsest level graph is identical to the input partition that is being repartitioned • This makes the coarsening phase completely parallelizable, as coarsening is local to each processor

  30. Multi-level diffusion repartitioning • Uncoarsening phase contains 2 subphases • Multi-level diffusion • Multi-level refinement • Multi-level diffusion – balance is attempted on the coarsest graph

  31. Multi-level diffusion repartitioning • A vertex is visited in a random order • If it is in overbalanced partition and is neighbors with nonoverbalanced partitions, moved to that nonoverbalanced partition that produces greatest improvement in edge-cut • Vertex is migrated even if this results in increased edge-cut • The process repeats until no balancing progress is made

  32. Multi-level diffusion repartitioning • But it may not be possible to balance the graph at the coarsest level since there may not be many vertices of small weight • Hence graph needs to be uncoarsened one level to increase the number of smaller weight vertices • The process is repeated on the next coarsest graph and so on until some threshold

  33. Multi-level diffusion repartitioning • After graph is balanced, multi-level diffusion ends, and multilevel refinement begins on the current graph • Here, the emphasis is on improving the edge-cut

  34. Multi-level diffusion repartitioning • Each border vertex is visited randomly to potentially migrate based on the following refinement phase vertex migration criteria • The selected partition is the vertex’s initial partition from the input graph; the move does not increase the edge-cut; the balance is maintained • The edge-cut is reduced while maintaining the balance • The balance is improved while maintaining the edge-cut

  35. Summary • Multi-level diffusion repartitioning algorithms are made up of three phases: graph coarsening, multilevel diffusion, multilevel refinement • Coarsening phase results in a series of contracted graphs • Multilevel diffusion phase balances the graph using the very coarsest graphs • Mutlilevel refinement phase seeks to improve the edge-cut disturbed by the balancing process

  36. Illustration

  37. Space Filling Curves for Partitioning and Load Balancing

  38. Space Filling Curves • The underlying idea is to map a multidimensional space to one dimension where the partitioning is trivial • There are many different ways • But a mapping for partitioning algorithms should preserve the proximity information present in the multidimensional space to minimize communication costs

  39. Space Filling Curve • Space filling curves are quick to run, can be implemented in parallel, and produce good load balancing with locality • A space-filling curve is formed over grid/mesh cells by using the centroid of the cells to represent them • The SFC produces a linear ordering of the cells such that cells that are close together in a linear ordering are also close together in the higher dimensional space

  40. Space Filling Curve • The curve is then broken into segments based on the weights of the cells (weights computed using size and number of particles) • The segments are distributed to processors; thus cells that are close together in space are assigned to the same processor • This reduces overall amount of communication that occur, i.e., increases locality

  41. SFC representation • One method is to define recursively – curve for a 2^k x 2^k grid composed of four 2^(k-1) x 2^(k-1) curves • Another method is using bit interleaving – the position of a grid point along the curve can be specified by interleaving bits of the coordinates of the point • The interleaving function is a characteristic of the curve

  42. Z-curve or Morton ordering • The curve for a 2^k x 2^k grid composed of four 2^(k-1) x 2^(k-1) curves, one in each quadrant of the 2^k x 2^k grid • Order in which the curves are connected is the same as the order for 2x2 grid

  43. Graycode Curve • Uses same interleaving function as Z-curve • But visits points in the graycode order • Graycode – two successive values differ in only one bit • The one-bit gray code is (0,1)

  44. Graycode Curve • The gray code list for n bits can be generated recursively using n-1 bits • By reflecting the list (reversing the list) [1,0] • Concatenating original with the reflected [0,1,1,0] • Prefixing entries in the original list with 0, and prefixing entries in the reflected list with 1 [00,01,11,10]

  45. Graycode Curve • 3-bit gray code: 000,001,011,010,110,111,101,100

  46. Hilbert Curve • Hilbert curve is a smooth curve that avoids the sudden jumps in Z-curve and graycode curve • Curve composed of four curves of previous resolution in four quadrants • Curve in the lower left quadrant rotated clockwise by 90 degree, and curve in lower right quadrant rotated anticlockwise by 90 degree

  47. SFCs for AMR • All these curve based partitioning techniques can also be applied for adaptive mesh by forming hierarchical SFCs

More Related