1 / 20

2.4 Parallel Performance Enhancements

2.4 Parallel Performance Enhancements. In this section, we will discuss the following topics: A. New add-on product Parallel Performance for ANSYS B. Distributed Domain Solver (DDS) C. Algebraic Multigrid Solver (AMG). Parallel Performance Enhancements Overview.

yasuo
Download Presentation

2.4 Parallel Performance Enhancements

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. 2.4 Parallel Performance Enhancements • In this section, we will discuss the following topics: A. New add-on product Parallel Performance for ANSYS B. Distributed Domain Solver (DDS) C. Algebraic Multigrid Solver (AMG) Training Manual 001419 15 Aug 2000 2.4-1

  2. Parallel Performance EnhancementsOverview • Driven by user requirements of higher accuracy and fidelity in solution • e.g. mesh refinement and adaptive meshing • Desire to solve assemblies instead of individual component analysis • e.g. assembly contact problems Training Manual 001419 15 Aug 2000 2.4-2

  3. Parallel Performance EnhancementsA. Parallel Performance for ANSYS • A new, add-on product for shared memory and distributed memory environments • Offers powerful new solvers enabling quick, accurate solutions to large models using multiple processors • Algebraic MultiGrid (AMG) solver • Solves static/ transient nonlinear analyses using multiple processors (up to 8) on a single system (shared memory parallel) • Distributed Domain Solver (DDS) • Solves large static / transient nonlinear analyses over multiple systems (Distributed memory parallel) as well as multiple processors on a single machine (Shared memory parallel) or any combination Training Manual 001419 15 Aug 2000 2.4-3

  4. Parallel Performance Enhancements B. DDS What is DDS? • Breaks large problems (up to 10 million DOFs) into smaller domains (1000 to 10000 DOFs) automatically • Compatibility among domains obtained by solving for interface variables (Lagrange multipliers) Training Manual 001419 15 Aug 2000 2.4-4

  5. 14 12 10 8 Speed up ratio 6 4 2 0 0 2 4 6 8 10 12 14 16 18 20 22 24 Number of CPUs Parallel Performance Enhancements … DDS ...What is DDS? • Transfers and factorizes the subdomains on slave machines using direct solver • Master machine retrieves and assembles subdomain solutions; solves for interface variables using an iterative solver and computes results for entire model Training Manual 001419 15 Aug 2000 2.4-5

  6. Parallel Performance Enhancements… DDS Why DDS? • Highly scalable • More processors / less elapsed time • Example below shows a 3.5 million-DOF SOLID92 model • 2020 subdomains on an SGI Origin 2000, 12GB memory Speed-up = 21.0 Training Manual 001419 15 Aug 2000 2.4-6

  7. Parallel Performance Enhancements … DDS Memory / Disk requirements • 2 to 4 times more memory than PCG; however, this is not a problem for distributed memory architecture. • Memory required is a sum of all master & individual slave machine memories • In general Master machine will need large memory Training Manual 001419 15 Aug 2000 2.4-7

  8. Parallel Performance Enhancements … DDS - Under the Hood • DDS has 2 components: • Domain decomposer • Embedded in ANSYS • Divides domain into n subdomains • Creates scratch.dds, file.dds, and file.erot • Issues ‘mpirun’ command and launches appropriate ansdds.e57 executable • ANSDDS.E57 • A stand-alone, MPI enabled executable • Computes solution for subdomain on the slave processor • Writes out a file called scratch.u, which is later retrieved by the Master to calculate element results Training Manual 001419 15 Aug 2000 2.4-8

  9. Parallel Performance Enhancements… DDS • System requirements • Network must be homogeneous (same operating system) • Message Passing Interface (MPI) used to communicate • Master (where the job is submitted) • “Performance Parallel for ANSYS” add-on required • ANSYS 5.7 must be installed (including ansdds.e57) • Installation of MPI • 256 MB ram / 10 GB disk required • Slave • Installation of MPI on all slave machines • ansdds.e57 executable must be installed Training Manual 001419 15 Aug 2000 2.4-9

  10. Parallel Performance Enhancements … DDS How to use DDS • Specify “Parallel Peformance for ANSYS” add-on when starting ANSYS • ansys57 -pp • Choose DDS Solver • EQSLV,DOMAIN • Specify information about slave processors • DDSOPT command* *DDSOPT command covered in Systems Training Training Manual 001419 15 Aug 2000 2.4-10

  11. Parallel Performance Enhancements … DDS How to use DDS (cont'd) • Solve • Postprocessing • You get a results file as usual • /PNUM,DOMAIN,ON will display domains by colors / numbers Training Manual 001419 15 Aug 2000 2.4-11

  12. Parallel Performance Enhancements … DDS Solver Are there any modeling restrictions for using DDS? • Structural static/transient only (linear or nonlinear) • Symmetric matrices • “h” elements only • No coupling / constraint equations • No inertia relief Training Manual 001419 15 Aug 2000 2.4-12

  13. Parallel Performance Enhancements … DDS Solver Training Manual 001419 15 Aug 2000 2.4-13

  14. Parallel Performance Enhancements C. AMG Solver What is AMG solver? • A preconditioned conjugate gradient solver similar to PCG solver • The preconditioner used in AMG solver is derived using Algebraic MultiGrid technique • MultiGrid techniques derive a preconditioner that is very close to [K]-1 by working on a coarser mesh of the FE model supplied • Algebraic MultiGrid methods work on a coarsened version of the full [K] matrix instead of the mesh (that is mesh independent) Training Manual 001419 15 Aug 2000 2.4-14

  15. Parallel Performance Enhancements… AMG Solver Why do we need AMG solver? • Sensitivity to ill-conditioning • Much less sensitive to ill-conditioned problems than PCG • Will get solutions in fewer iterations than PCG for ill-conditioned problems • Expected to perform as well as PCG for well conditioned problems • Scalability • Up to 5 times for 8 processors • Scales much better than PCG • Used in shared memory parallel (single machine with multiple processors) only Training Manual 001419 15 Aug 2000 2.4-15

  16. Parallel Performance Enhancements … AMG Solver Scalability Training Manual 001419 15 Aug 2000 2.4-16

  17. Parallel Performance Enhancements … AMG Solver How to use AMG solver • Specify “Parallel Peformance for ANSYS” add-on when starting ANSYS • ansys57 -pp • Specify number of processors: • /CONFIG,NPROC,N • or config57.ans • or use the macro SETNPROC • Choose AMG Solver • EQSLV,AMG,Toler • Tolerance defaults to 1e-8 similar to PCG • Solve Training Manual 001419 15 Aug 2000 2.4-17

  18. Parallel Performance Enhancements … AMG Solver • When to use AMG solver • Structural Static & Transient analyses • Nonlinear analyses • Large aspect ratio elements, reduced integration elements • Models with combination of shells/ solids/ beams • Shared memory parallel machines • When not to use AMG solver • Non-structural problems (it works but is less efficient) • Models made of only shell63 elements do not seem to be as cpu efficient as PCG Training Manual 001419 15 Aug 2000 2.4-18

  19. Parallel Performance Enhancements … AMG Solver Memory / Disk requirements • 1.3 to 2 times more memory than PCG solver • Rule of thumb is 130 MB per 100,000 dof for solid92s • Memory required is also a function of number of processors used (overhead) • Files created during AMG solution are very similar to PCG and about the same size Training Manual 001419 15 Aug 2000 2.4-19

More Related